Supporting Exam Revision via Google Talk

Page created by Carolyn Lowe
Supporting Exam Revision via Google Talk
Session T1G
     Supporting Exam Revision via Google Talk and
                  Examopedia Wiki
                                                       Manish Malik
                                    University of Portsmouth,

Abstract - This work highlights the benefits of tapping         Today’s students see themselves as consumers and the
into one of many informal learning activities in which          university as a provider of service [3]. Many universities
students engage, namely use of past papers for exam             have set generic guidelines with regards to exam revision
revision. Exams can be both stressful and isolating for         strategies and study skills [4] and [5]. Providing support for
students. This paper presents a novel approach to exam          exam revision has become necessary in the current climate
revision via a ‘wiki’ and ‘Google talk’ which the author        of ‘top-up’ fees within the UK Higher Education (HE)
calls ‘Examopedia’. This approach benefits both from            sector.
cooperative and constructivist learning. We show how
students form an online community of practice around            Exam papers are widely used by the students for exam
this informal learning activity providing peer-support          revision. This was also one of the most dominant strategies
for each other. It is well known that peer interactions,        adopted by the students in this study. It is by no means the
online or face to face, can encourage deep learning.            only strategy used for exam revision [6]. Other resources
Using Examopedia, students can prepare together for an          used include lecture notes/slides, lectures, tutorials, revision
exam. When using Examopedia on their own, students              lectures, personal and or shared notes [4] and [5]. This work
found conflicting answers a problem. However, the tutor         focuses only on the use of past exam papers as a student
used Examopedia with the students to give them timely           strategy for exam revision. We are aware that students may
guidance. Examopedia gives a new opportunity to                 however have used other resources in parallel to
facilitate revision through timely formative feedback           Examopedia.
and guidance. It also allows ‘Just in time’ teaching
where needed. We find Examopedia to be a blend of               Many tutors just provide solutions and or simply the
teacher and student centric learning activities. We             marking scheme of past papers for students to aid them in
present analysis of both quantitative and qualitative           their revision. The ‘contentious learner’ is not happy with
data collected for two groups of students (N=38 and             just the ‘correct answers’ to past papers [3] and tends to
N=58) at Level M and Level 1 of their engineering               value feedback on their own work. Wikis provide a valuable
degree.                                                         asynchronous platform for interaction to develop, despite
                                                                the space and time boundaries that separates people. Google
Index Terms – Online Collaborative and Cooperative              talk is a synchronous tool that can be used to address issues
Learning, Exam revision and stress, Formal and Informal         that are urgent and need to be resolved quickly.
Learning, Teacher centric and student centric
                                                                The remainder of this paper is organized as follows.
                      INTRODUCTION                              Presented first are the details of a novel service, called
Exams are known to cause anxiety in some students. This         ‘Examopedia’, which uses a wiki and Google talk. Followed
may lead to psychological problems and reduced                  by an account of how changing technology has had an
performance in students [1]. Peer support and interaction is    impact on shaping Examopedia. This is followed by the
proven to help achieve deep learning in students [2]. This      research questions to which this paper provides answers.
paper describes a blend of cooperative and constructivist       Following which, presented are the details of the experiment
approaches where a large group of students collaboratively      and the results. Finally, this is followed by some discussion
solve past papers using the Examopedia website. The aim of      and the conclusion.
this work is to study the impact of cooperative and                                     EXAMOPEDIA
collaborative exam revision on student learning and also on
the issue of anxiety and stress associated with exams. So far   Using past exam papers for exam revision is seen mainly as
up to 490 undergraduate and postgraduate students have          an informal approach, i.e. one in which academics and
used ‘Examopedia’ within the Department of Electronic and       tutors do not engage. Other informal approaches include
Computer Engineering at the University of Portsmouth, UK.       students studying in groups and sharing notes etc. Usually,
                                                                there is no direct formal input from the lecturers during the
                                                                revision period other than maybe a revision lecture.

978-1-4244-6262-9/10/$26.00 ©2010 IEEE                                   October 27 - 30, 2010, Washington, DC
                               40th ASEE/IEEE Frontiers in Education Conference
Session T1G
Examopedia is an exam revision service that the author runs                   question text and image/data etc live in the first row of a
for his students at level 1, 2 and M at his institution.                      multi row table (See table 1). This table also has 3 columns
Examopedia allows formal input into one of the most                           in total. The rows other than the first row are identified as
popular informal revision practice i.e. using past papers.                    contributor rows, one for each contributor. Usually 3
This approach is both open and contributory for staff and                     answers for the same questions are sufficient to provide
students alike. Of course not every student uses past exam                    different point of views, but where needed students are free
papers for their revision, but most do. We feel that                          to add a row to add their contributions. The first column is
Examopedia can be used by majority of students at any                         for question number/contributor number, the second for
stage of their revision. Although using Examopedia is an                      question text/data or contributions and the final comment is
optional activity and not all students contribute to all the                  for tutor only feedback. Students are allowed to write just in
answers on the site, it was found that a vast majority of the                 the middle column.
students still use it.
                                                                              One criticism directed towards Examopedia is that it leads
                                                                              increased workload for the tutor and how to manage an
                                                                              increased sense of immediacy in students with regards
                                                                              responses from the tutor. The solution to both these issues
                                                                              lies in advertising and scheduling the responses as and when
                                                                              these are made. The author advertised 2 responses per day
                                                                              before the exam.

                                                                                        USE OF GOOGLE SITES & GOOGLE TALK

                                                                              The wiki used in the first two years of operation of
                             FIGURE 1                                         Examopedia was a Twiki wiki, which worked fine in the
    EXAMOPEDIA: ITERATIVE CYCLES SHOWING STUDENT ATTEMPTS IN                  main but the ‘what you see is what you get’ edit interface
         SOLVING PAST PAPERS AND TUTOR/PEER FEEDBACK                          was not reliable enough. There are newer versions of Twiki
                                                                              available with a growing support community. However,
In Examopedia students learn by reading shared                                when the University of Portsmouth decided to subscribe to
contributions from other students and through feedback                        ‘Google Apps’ for student email and collaboration tools the
from the tutor(s) on these contributions. This iteration goes                 author also decided to use Google Sites for Examopedia.
on and it unfolds opportunities for student-student and                       This proved both useful and problematic. Google Sites
student-teacher interactions which can be hard to replicate                   proved very user friendly collaborative website authoring
in face-to-face environments throughout the exam revision                     tool. Also, what helped was the fact that the student
period. This is cyclic interaction is best shown in figure 1.                 accounts were already created as part of the university
                                                                              Google Apps move. The down side was that the students
                             TABLE I                                          had their accounts on the ‘’ domain and
                                                                              some of the staff had their accounts on the ‘’
Q. No. ‘n’       Text, images and data relevant       Tutor comments/
                 to the question to be solved by      Feedback and            domain of ‘Google Apps’ within the University of
                 the students.                        Guidance                Portsmouth. It turned out that having staff and student
Contributor 1    This is where student 1 answer       This is where tutor     accounts on the same domain, till this day, is better for staff-
                 the questions with text , images,    responds to student 1   student collaboration. This is due to the logging in process
                 diagrams, calculations, links etc.
                                                                              being simpler if both staff and students have a common
Contributor 2    This is where student 2 answer       This is where tutor     Google Apps domain.
                 the questions with text , images,    responds to student 2
                 diagrams, calculations, links etc.
Contributor 3    This is where student 3 answer       This is where tutor     Google Talk is a simple chat tool, placed on Examopedia as,
                 the questions with text , images,    responds to student 3   which was used for synchronous one-to-one tutoring.
                 diagrams, calculations, links etc.                           Google Talk widget, an embedded script that launches the
…                …                                    …
                                                                              Google Talk application, can be invoked by the students to
Contributor N    This is where student N answer       This is where tutor
                                                                              chat to the tutor when both are online. This tool was used
                 the questions with text , images,    responds to student     one or two days before the exam to help students with any
                 diagrams, calculations, links etc.   N                       last minute questions and clarifications. This tool was only
                                                                              used on the day before the exam.
Examopedia uses a wiki to host past exam paper questions.
The author arranges questions as complete exam papers for
different years. Questions are posted on an editable wiki                                        RESEARCH QUESTIONS
page along for students to read and contribute to. The author
                                                         It was intended to find out the benefits of Examopedia to the
found it useful to give some structure to these pages. Each
                                                         students who used it for their exam revision. To this end a
978-1-4244-6262-9/10/$26.00 ©2010 IEEE                                       October 27 - 30, 2010, Washington, DC
                               40th ASEE/IEEE Frontiers in Education Conference
Session T1G
questionnaire was developed to test some of the hypothesis         The site usage data could be misleading as students used
about the benefits of Examopedia. The questionnaire was            shared computers when at the university, hence we say
developed from the data in qualitative comments about use          minimum 28 students. As shown in figure 1 the ‘average
of Examopedia made prior to 2009 as well as from the               time on page’ spent on the main exam revision page
evidence gathered from the research literature. The research       gradually rose to a maximum of 5 minutes and 6 seconds in
questions that this paper addresses are listed below:              the period before the exam. It is noteworthy, that the
                                                                   ‘average time on page’, stabilised four days before the exam
1.   How many students used Examopedia, how and when?              and remained around the 3.5 minute mark before dropping
2.   What impact does the use of Examopedia has on                 back to 0 seconds, 2 days after the exam. This stabilisation
     student learning and exam related anxiety with respect        happened after the bulk of new users had started to use the
     to full time HE students preparing for their tests or end     site. The majority of the edits on the site took place between
     of unit exams?                                                10-14 days before the exam. During the constant ‘average
                                                                   time on page’ period the number of edits were 4, out of an
                EXPERIMENT BACKGROUND                              all time total of 18 edits. The earliest edit was recorded 15
The response rate for the questionnaires was 52% over all          days before the exam.
and 53% for level M students and 51% for Level 1 students.
The level M students were preparing for a course on                II. Level M usage data
Microwave technologies and Systems and the Level 1
students were preparing for an introductory course on ICT.         The survey data and the site usage data both suggest that
The author is the tutor on both these courses. The exam for        there were at least 20 students (100% respondents) that were
the Level 1 students is a 1 hour in-class test with 10 short       users of the site including 13 active contributors. Again the
answer questions and that for the Level M students is an end       site usage data can be misleading as many students would
of semester 2 hour exam with 4 questions with several parts.       have used the same computers whilst at the university. As
                                                                   shown in figure 2 the ‘average time on page’ spent on the
The questionnaire used to collect the data had 25 Likert           main exam revision page gradually rose from 0 seconds to a
style questions and two free text areas for students to inform     maximum of 6 minutes and 45 seconds. It is noteworthy,
either about the survey itself or the issues with Examopedia.      that the ‘average time on page’ became almost constant for
The survey was carried out using ‘Google Forms’.                   three days before the exam and remained around the 6
                                                                   minute mark before dropping back to 0 seconds the day
Data relating to the number of edits, time of edits, number        following the exam. This stabilisation in usage happened
of unique and returning visitors, ‘average time on page’ and       after the bulk of new users had started to use the site. The
site usage on specific dates for specific pages was at hand to     same is true for the majority of the edits on the site. During
use in the analysis. Google Analytics was used to generate         the stabilised ‘average time on page’ period the number of
this data.                                                         edits were 50, out of an all time total of 62 edits. The
                                                                   earliest edit was recorded one month before the exam.

I. Level 1 usage data

The survey data suggest that there were 25 students users
(or 83% of the respondents) of the site including 9 active
contributors. But the site usage data suggests only 5 active
contributors or editors and a minimum of 28 unique site
                                                                                              FIGURE 2
                                                                   AVERAGE TIME ON PAGE FOR ‘LEVEL M’ MAIN EXAM PAGE VS DATES (DATE
                                                                                           OF EXAM -26-JAN)

                                                                   III. Level M student questionnaire responses

                                                                   80% respondents agreed, of which 40% strongly, that
                                                                   revising ‘had never been so interactive’. 100% respondents
                                                                   thought, of which 65% strongly agreed, that Examopedia
                                                                   ‘helped them revise topics’. 85% respondents of which 40%
                                                                   strongly agreed that by using Examopedia they ‘felt more
                           FIGURE 1
                                                                   prepared for the Exam’. 95% respondents, of which 45%
                        OF TEST -09-DEC)                           strongly agreed, that Examopedia ‘helped them identify
                                                                   topics for revision’. 95% respondents, of which 55%

978-1-4244-6262-9/10/$26.00 ©2010 IEEE                                   October 27 - 30, 2010, Washington, DC
                               40th ASEE/IEEE Frontiers in Education Conference
Session T1G
strongly agreed, that the collaboration enabled by
Examopedia ‘helped them revise better’. For the categories      Guidance from the tutor was of the following nature:
reported so far there were no disagree/strongly-disagree        • Point out mistakes in solutions and how to improve (11
responses. 70% respondents, of which 35% strongly agreed,           counts)
that Examopedia ‘helped reduce exam related stress’.            • Encouragement and confirmation of correct
                                                                    contributions (8 counts)
80% respondents thought, of which 30% strongly agreed,          • Point out what could gain them more marks in the exam
that Examopedia ‘helped them access the collective                  if they got a similar question type (8 counts)
intelligence of fellow students’. 95% respondents thought, of   • Hints to solve a question (6 counts)
which 40% strongly agreed, that Examopedia provided             • Identification of sources to read further to solve a
‘valuable peer and tutor feedback’. 95% respondents                 question (6 counts)
thought, of which 45% strongly agreed, that Examopedia          • Asking students to use two contributions to create a
provided ‘different view of fellow students about the same          new answer (5 counts)
things’. 85% respondents thought, of which 30% strongly
                                                                • Drawing links between two contributions (4 counts)
agreed, that Examopedia provided ‘critiques on work shared
                                                                • Statement of facts where necessary (3 counts)
on the site’. 80% respondents thought, of which 45%
                                                                • Explaining procedures (3 counts)
strongly agreed, that Examopedia helped ‘increase their
depth of understanding’. 70% respondents agreed, of which       • Asking students for their understanding or encouraging
20% strongly agreed, that Examopedia helped ‘increase my            them to solve or as questions (3 counts)
motivation to learn’. 90% respondents thought, of which         • Doing Just in time teaching [9] (3 counts)
35% strongly agreed, that Examopedia helped ‘broaden
their knowledge of some topics’. 90% respondents thought,                              DISCUSSIONS
of which 55% strongly agreed, that Examopedia helped
‘them to think broader about the question before answering      The work presents qualitative and quantitative data in order
it’.                                                            to answer the two research questions listed above.
     60% respondents thought, of which 35% strongly             Questionnaire and site usage data shed light on how the
agreed, that Google Sites was ‘easy and user friendly to        students used this service. Going only by the survey data
use’. 25% respondents thought, of which 10% strongly            more than 80% of the student used Examopedia in some
agreed, that Examopedia ‘assumed that they knew too             way. While majority of the students only preferred to read
much’. And finally 100% respondents, of which 65%               the contributions from the site there were around 32% Level
strongly agreed, that they will use Examopedia ‘again           M students who made regular contributions to the site. This
where possible’.                                                figure was lower for Level 1 students (15%). The ‘average
                                                                time on page’ figures show that students engage with exam
Sample free text comments from questionnaire:                   papers only in the 15 to 20 days before the exam. The usage
• Examopedia was very interactive and enlightening and          became more consistent in the few days before the exam.
   went long way to help clear doubts                           This is both expected and common with majority of students
• Activate it in the Christmas holidays                         as prior to this period most students are busy reading other
• Only a few people contribute, most did not try to solve       course material, deadlines for assignments and or attending
   any questions                                                lectures and labs and may not all be revising for exams.
• Very useful to read what others are thinking about some
   particular question.                                         The idea of solving past exam papers in an open and
• Some of the answers can be confusing                          contributory fashion has some resemblance with the popular
                                                                online encyclopaedia, Wikipedia [7]. Researchers have
IV. Level M edits and counter edits                             studied the roles of contributors who contribute to
Different contributions resulting from the 62 edits on the      Wikipedia. In Wikipedia the ‘experts’ are the seasoned
site from different people were categorised as follows:         contributors who may have started as ‘novices’ at some
• textual explanations/answers (19 counts)                      point [7]. The majority of the users on Wikipedia simply
• new or improved textual explanations (12 counts)              read the contributions from other people. The equivalent of
• Asking a question or requesting Just in time teaching         ‘Expert(s)’ in Examopedia, are the tutors who facilitate
     [9] (7 counts)                                             student contributions and guide them to improve their
                                                                answers. This can be seen from the nature of comments
• Solution to numerical and analytical questions
                                                                made by the tutor as detailed in the previous section. The
     (5 counts)
                                                                nature of student contributions clearly suggest that
• addition of a figure or a diagram to aid explanation (3
                                                                Examopedia is cooperative and collaborative both.
                                                                Examopedia has resemblance with cooperative learning
• addition of equations to better explain (2 counts)            where there is a division of labour and individual
• Different ways of solving the same analytical/numerical       contributions [8]. This division is hidden or tacit due to the
     question (2 counts)
978-1-4244-6262-9/10/$26.00 ©2010 IEEE                                   October 27 - 30, 2010, Washington, DC
                               40th ASEE/IEEE Frontiers in Education Conference
Session T1G
pre-structuring done on the wiki using the table shown in        contributions made revision an interactive and less stressful
table 1.                                                         process. The site encourages student centric learning as
                                                                 guidance is provided on specific topics that students have
The interactions in Examopedia begin as individual               actually shared. The site also is a teacher centric tool to
contributions to separate questions from past papers. Over       deliver Just-in-time teaching if needed. Many students
time, students get feedback from the tutor(s) (guide on the      confirmed that the service helps address the issue of anxiety
side) and new contributions from peers appear on                 and stress related with exam. There were less active
Examopedia. As can be seen from the nature of student and        contributors and more passive users for Examopedia.
tutor contributions there is minimal amount of discussion        However, the majority of the students did benefit from it
and negotiation of meaning between students but there is a       and are willing to use it again. There are technical issues
lot of guidance given to student to improve their answers. It    that we have learnt lessons from and have decided to make
is the ‘guide on the side’ that gives it a constructivist and    the necessary changes to further improve the student
collaborative feel. The students value the presence of an        experience in using the service.
expert in of case confusing contributions from some
students. Without the tutor presence Examopedia could lead
to confusion (as indicated in free text comments) and its use                           ACKNOWLEDGMENT
drop significantly eventually.                                   The author is thankful to the comments received from Phil
                                                                 Barker who evaluated the work for the Higher Education
Moreover, as the contribution–feedback iterative cycle           Academy’s Engineering Subject Centre in the consideration
unfolds, gap in student knowledge on different topics            of National Teaching Award which the author won along
become known to the tutor. This knowledge easily triggers        with three other winners in 2009.
another role for the tutors, that of delivering ‘Just-in-time
teaching’ to the participating students [9]. This is also                                    REFERENCES
evident from the nature of student and tutor contributions in
                                                                 1.   Neuderth, S., Jabs, B. and Schmidtk, A., “Strategies for reducing test
section IV.                                                           anxiety and optimizing exam preparation in German university
                                                                      students: a prevention-oriented pilot project of the University of
Some students however did show concern about other                    Wu¨rzburg”, Journal of Neural Transmission, 116, 6, 2009, 785-790.
students benefiting from their work – ‘…most did not try to      2.   Biggs, J., B., “Approaches to the Enhancement of Tertiary Teaching”,
solve any question…’. This feeling is understandable as the           Higher Education Research & Development, 8, 1, 1989, 7-25.
participation to the site was purely optional. In cooperative
                                                                 3.   Revision and Exam Skills, Loughborough University, UK, 2007,
learning activities division of labour is very important to  [Last accessed 30
avoid such feelings. Most cooperative learning thus takes             March 2010]
place in small groups. The fact that the entire class was        4.   Revision and Exam Skills, University of Leicester, UK, 2010
working on the same task may have resulted in many not      
contributing. Were there enough questions for everyone to             [Last accessed 30 March 2010]
contribute? This is something we will investigate in future.     5.   Higgins, R., Hartley, P. and Skelton, A., “The Conscientious
                                                                      Consumer: reconsidering the role of assessment feedback in student
Finally, there were a number of students who were                     learning.” Studies in Higher Education, 27, 1, 2002, 53-64.
dissatisfied with Google Sites. This was identified earlier on   6.   Van Etten, S., Freebern, G. and Presley, M., “College Students’
and captivate demos were launched for helping students                Beliefs about Exam Preparation”, Contemporary Educational
learn the processing of logging on to the site for editing it.        Psychology, 22, 1994, 192–212.
Students found it difficult to log on to a Google site which     7.   Bryant, S.L., Forte, A., and Bruckman, A., “Becoming Wikipedian:
existed on ‘’ domain since their accounts were             Transformation of Participation in a Collaborative Online
in ‘’. This was not impossible to do but as it           Encyclopedia”, In Proceedings of the 2005 international ACM
                                                                      SIGGROUP Conference on Supporting Group Work, Sanibel Island,
was a new thing for everyone, people generally struggled.             Florida, USA, November 6-9, 2005, GROUP '05.
For future versions the author will use Google docs
embedded in Google Sites to avoid the problem student            8.   Slavin, R. E., “Cooperative Learning”, Review of Educational
                                                                      Research, 50, 2, 1980, 315-342.
                                                                 9.   Novak, G. M., Patterson, E. T., Gavrin, A. D. and Christian, W., “Just
                       CONCLUSIONS                                    in time teaching”, American Journal of Physics, 67, 10, 1999, 937-
We have presented in this paper a novel technique of using a
wiki as an exam revision site, called Examopedia.                                     AUTHOR INFORMATION
Examopedia was used mainly during 15-20 days in the run          Manish Malik, Principal Lecturer, Faculty Learning and
up to the exam by a majority of students. The use of the site    Teaching Coordinator, University Learning and Teaching
was consistent nearer to the exam when most students are         Fellow, Department of Electronic and Computer
engaged with preparing with the exam. Multiple edits of the      Engineering,   University     of    Portsmouth,    UK
same page show how contributions, guidance, further    
978-1-4244-6262-9/10/$26.00 ©2010 IEEE                                   October 27 - 30, 2010, Washington, DC
                               40th ASEE/IEEE Frontiers in Education Conference
You can also read