Examinations That Support Collaborative Learning: The Students' Perspective

Page created by Manuel Brady
Examinations That Support Collaborative Learning: The Students' Perspective
Examinations That Support Collaborative Learning

Examinations That Support
Collaborative Learning:
The Students’ Perspective
By Georg W. Rieger and Cynthia E. Heiner

We used surveys and classroom                     niversity instructors in-     Moreover, although individual
observations to examine student                   creasingly use interactive    exams produce a uniquely intense
reactions to two-stage exams,                     engagement and social/        engagement with the material, that
where students first do the exam                  collaborative      learning   engagement provides little or no
individually and then redo it           methods in their science classes to     contribution to learning—defined as
collaboratively. Our results show       achieve better learning outcomes        acquiring new ideas—because of the
why both students and instructors       (National Research Council, 2012).      lack of timely and useful feedback
appreciate this examination             Such methods result in deeper en-       (Black & Wiliam, 1998).
format: Two-stage collaborative         gagement by the students and in-           The two-stage exam is a relatively
examinations are relatively easy to     corporate more formative assess-        simple way to solve these problems.
implement, have a high potential        ment to support learning. A number      In a two-stage exam, students first
for learning, and support the           of research-based methods, such         complete and turn in the exam indi-
collaborative learning approach         as peer instruction (Mazur, 1997),      vidually and then, working in small
used in many sciences classes.          think-pair-share (Johnson, Johnson,     groups, answer the exam questions
A look at survey data from an           & Smith, 2011), and cooperative         again. This makes the exam itself a
introductory physics class shows        group problem solving (Heller &         valuable learning experience while
that a vast majority of students        Hollabaugh, 1992; Heller, Keith, &      also sending a consistent message to
(76%) had a positive opinion of         Anderson, 1992), share some basic       the students as to the worth of col-
this exam format (expressed in          features that are recognized to sup-    laborative learning. We see indica-
236 comments) whereas only 10%          port learning across a wide range of    tions that the use of this exam format
expressed an overall negative           contexts. These features include in-    goes beyond ensuring consistency
opinion in 30 negative statements.      tense engagement by students, col-      across course components, in that it
Most of the positive comments           laborative learning where students      positively impacts how students ap-
relate to how this benefits learning.   develop their thinking, and imme-       proach the other collaborative com-
In this article, we describe how to     diate feedback through the inter-       ponents in the course. The two-stage
implement two-stage exams, discuss      actions with their peers (National      exam accomplishes this while still
advantages and disadvantages, and       Research Council, 2012). In this ar-    providing summative assessment of
present the students’ view.             ticle we discuss an exam format—        individual performance.
                                        two-stage exams—that uses these            Collaborative tests have been
                                        same features.                          used for some time in a variety of
                                           Frequently, collaborative learning   formats (see summaries in Leight,
                                        and formative assessment will be        Saunders, Calkins, & Withers,
                                        used in classroom instruction, but      2012; Zipp, 2007). The two-stage
                                        the course exams will remain in the     format discussed in this article
                                        traditional format in which students    (sometimes referred to as group test
                                        solve problems in isolation and         Cortright, Collins, Rodenbaugh, &
                                        only receive feedback several days      DiCarlo, 2003; or pyramid exam,
                                        later. Exams send very powerful         Cohen & Henle, 1995) has also
                                        messages, and such an exam format       been used in the past, in particular
                                        does not support the message that       in team-based learning as part of
                                        collaborative learning is important.    the readiness assurance process (see

                                                                                               Vol. 43, No. 4, 2014      41
e.g., http//:www.teambasedlearn-         free-riders (see discussion in Zipp,       ing the time for discussions and
     ing.org). This process, which uses       2007). Our survey results, however,        agreeing on a solution.
     scratch-and-win type testing cards       indicate that this occurs only in a
     during the group part to reveal          small number of groups.                     As an example, the two-stage
     the answers to all questions, fol-          This article was inspired by see-    exam given in our introductory phys-
     lows up on the assigned reading,         ing both the success of two-stage       ics course (N = 178) had a total dura-
     and provides a low-stakes way to         exams and how popular they have         tion of 90 minutes that was split as 55
     ensure that students have the back-      been with both students and instruc-    minutes for individual effort (Stage
     ground knowledge necessary for           tors across the Faculty of Science at   1) and 30 minutes for group effort
     the problem-solving activities that      the University of British Columbia      (Stage 2), with 5 minutes in between
     follow. However, administering           (UBC). This exam format was first       for making the switch from Stage 1
     high-stakes examinations such as         introduced in the UBC Faculty of        to Stage 2. During the switch, in-
     midterm or final examinations in a       Science 3 years ago and is now          structors and teaching assistants first
     two-stage format is still relatively     being used in at least 20 science       collected the individual exam copies,
     rare. Stearns (1996), for example,       courses. The faculty members value      and then students were instructed to
     mentions increased student per-          the widespread intense engagement       sit with their predetermined group
     formance on the (individual) final       by their students during the second     members (3–4 students per group).
     exam in a research method and            stage of the exam, and as discussed     In some courses, these groups are
     statistics class after taking the mid-   below, students see them as valuable    preformed (e.g., same as collab-
     term exams in a two-stage format,        learning experiences. Next, we de-      orative groups in class or groups put
     as well as decreased dropout rates,      scribe how to implement two-stage       together by the instructor), whereas
     higher enjoyment of the course,          exams, discuss their benefits, and      in other courses, students are free to
     and increased collaborative skills.      present the students’ view.             choose their groups. Once the groups
     Only a few studies have attempted                                                were assembled, the second part of
     to measure the benefits of two-stage     Implementation of two-                  the exam was distributed. Generally
     exams on learning in science: In a       stage exams                             the switch can be done in less than
     recent study, Gilley and Clarkston       The particular format of the two-       5 minutes—even in large classes,
     (2014) reported knowledge gains          stage exam we use is relatively easy    if there is at least one instructor or
     (increases in student learning, i.e.,    to implement and has worked well        teaching assistant for 50 students.
     the original acquisition of knowl-       in UBC science courses.                     A two-stage exam in a 50-minute
     edge by students) due to the collab-                                             lecture time slot is doable, but hav-
     orative part of the exam in a science    • Stage 1 (individual, between 3/4      ing a 90-minute time slot is easier.
     course on natural disasters, whereas       and 2/3 of the examination time):     In some courses, instructors have
     other studies in biology (Leight et        This is a standard formal exami-      replaced their 50-minute in-class
     al., 2012) and physiology (Cortright       nation students complete work-        midterm exams with 90-minute eve-
     et al., 2003) have focused on the          ing alone.                            ning exams, so that similar content
     retention of content. A positive im-     • Stage 2 (small groups, remainder      can be covered. Concerns about the
     pact on student motivation, reduced        of the examination time): The         length of an exam can be addressed
     test anxiety, increased collaborative      group portion begins after all        by repeating only the conceptual
     skills, and improved perception of         individual exams are collected.       questions of the individual part in
     the course were also mentioned in a        Students work in groups of three      the group portion and/or by turning
     number of other studies (see refer-        or four students on (mostly) the      short-answer questions of the indi-
     ences in Gilley & Clarkston, 2014;         same problems as in the individ-      vidual part into multiple choice or
     Leight et al., 2012; Zipp, 2007).          ual portion (Figure 1). They must     ranking tasks in the group portion;
     Potential limitations of two-stage         come to a consensus on the an-        see Figure 1 for examples.
     exams are a reduced number of              swers and hand in one copy with           Grades from the individual and
     questions on the tests (to make time       the names and student ID num-         the group portion are combined for
     for the group portion) and a slightly      bers of all group members. Be-        the total examination mark, weight-
     higher administrative effort. In addi-     cause students have already seen      ed between 75% to 90% for the
     tion, differences in group composi-        each problem during Stage 1,          individual portion and 25% to 10%,
     tion may limit the effectiveness of        solving the same problems again       respectively, for the group portion.
     this approach in groups with one           in Stage 2 usually takes much         The group exam score has no effect
     dominant student or in groups with         less time than in Stage 1, includ-    on the differentiation between stu-

42   Journal of College Science Teaching
Examinations That Support Collaborative Learning

dents (i.e., a student’s performance           resulted in an average increase of the           0.5% from the midterm and 0.7% for
relative to the class), yet even the           midterm mark due to the group por-               the final exam, where the standard
small weight of the group portion              tion of 3.3% and an average increase             deviation of course grade distribu-
provides sufficient motivation for             in the final exam score due to the               tion was 9.7%.
students to take this part seriously.          group portion of 1.6 %. The resulting               On the basis of the collective ex-
For example, an 85/15 (individual/             impact on the average course grade               perience at UBC across the science
group) split used in our physics class         of the group part of the exams was               disciplines of physics, chemistry,

 Examples of questions taken from a two-stage exam for physics.

  Most questions will be the same for the individual and the group part. If questions are modified, it is usually to reduce the number
  of detailed calculations, which do not promote discussions, and replace with prompts to “explain your reasoning.” Additionally,
  one or two more challenging questions may be added.

  INDIVIDUAL PART                                                     GROUP PART
  A train is approaching the train station at velocity v0 = 15 m/s (Changed to ranking) A train is approaching the train station at
  relative to the ground in still air. The train operator sounds the velocity v0 relative to the ground in still air. The train operator
  train whistle, which emits a note with frequency f0 = 2500 Hz.     sounds the train whistle, which emits a note with frequency f0.

  The sound of the whistle is heard by different observers:           The sound of the whistle is heard by different observers:

  The train operator hears a frequency fA;                            The train operator hears a frequency fA;

  a person standing on the station platform watching the train a person standing on the station platform watching the train
  approach hears a frequency fB;                               approach hears a frequency fB;

  the operator of a second train approaching the station from the the operator of a second train approaching the station from the
  other direction with velocity v2 = 10 m/s hears a frequency fC. other direction with velocity v2 hears a frequency fC;

  What are the frequencies fA, fB, and fC?                            a passenger traveling on a slower train that has just been over-
                                                                      taken by the first train (and sees the first train move farther
                                                                      away) hears frequency fD.

                                                                      Rank the frequencies heard by the observers (fA, fB, fC, fD) in order
                                                                      from the highest to the lowest frequency.
  The graph shows the velocity vs. time graph of a harmonic (Replace part c)
  oscillator.                                               a) same
                                                            b) same
                                                            c) Sketch the potential energy curve as a function of time. As-
                                                               sume that we have a horizontal harmonic oscillator.

  a) the angular frequency
  b) the maximum displacement
  c) the phase constant and the equation describing the position
  as a function of time.

                                                                                                                   Vol. 43, No. 4, 2014       43
biology, math, statistics, Earth and
                                                FIGURE 2
     ocean sciences, computer science,
     and land and food systems, we would       Percentage of students with generally positive, negative, or neutral
     further recommend the following:          opinions about two-stage exams (N = 123 students). General positive
                                               means students found the exam format to be good or helpful for
     • Students are told on the first day      learning. Neutral/other means that students did not express a clear
       of classes that examinations will       positive or negative opinion, or commented on other things. General
                                               negative means that student had overall negative comments about the
       be conducted in this format and,
                                               exam format.
       more important, why this is done
       in this way.
     • A policy is implemented that
       the group score cannot be lower
       than the individual mark. This
       will address concerns about fair-
       ness. In practice, it affects only
       a few high-performing students
       as groups perform equal or bet-
       ter than individual students in al-
       most all cases.
     • Clear instructions are given dur-
       ing the individual-to-group tran-
       sition. For example, students
       should remain seated while their
       individual exam copies are col-
       lected. Remind and check that all
       names and student numbers are
       listed on the group exam.
     • Students are discouraged from
       working on their own during the        add a few observations from several      exam? Even a casual observation of
       group portion and all members          science classes at UBC.                  the two situations reveals the differ-
       are encouraged to be involved in          During the high-stakes environ-       ence: We routinely see nearly 100%
       discussing every problem. Teach-       ment of an examination, students are     engagement during the group part
       ing assistants and instructors can     heavily invested in figuring out the     of the exams, presumably because
       help with forming groups and           correct answers. After the individual    of the high-stakes situation of an
       encouraging collaborative work,        portion, all students are well pre-      examination. As confirmed through
       but this is seldom needed.             pared to discuss their approach in a     both observations and student self-
                                              group. In these discussions, students    reports, most groups discuss the
         Overall, it does not take much ef-   get immediate feedback on their          questions until all members agree on
     fort to run a two-stage exam. From       solutions from their peers, which        an answer; even during open-book,
     our experience, creating the group       might help them clarify their think-     two-stage exams, it was very unusual
     portion of the exam is easy because it   ing (Cortright et al., 2003, Gilley &    to see students looking through the
     is largely identical to the individual   Clarkston, 2014, Rao, Collins, & Di-     book to find the answers instead of
     exam, and the additional marking         Carlo, 2002). Weaker students could      discussing them and figuring them
     time of the group copies is minor be-    benefit from the explanation that is     out themselves. Those students who
     cause most solutions are correct. To     targeted to their difficulties, higher   are usually too shy to speak up dur-
     our knowledge, no instructor at our      achieving students might benefit         ing in-class activities will defend
     institution who has tried two-stage      from explaining concepts to others,      their answers vigorously during
     exams has abandoned this approach.       and everyone may well benefit from       the second stage of the exam. By
                                              critically evaluating others’ ideas.     comparison, the discussions during
     Benefits of two-stage exams                 One may argue that these same         normal in-class activities, such as
     Here we offer some thoughts on           benefits are also present in “nor-       clicker question discussions, do not
     why collaborative exams can in-          mal” in-class collaborative learn-       have nearly the same intensity. This
     crease learning and retention and        ing activities; so why do this on an     is probably because the stakes are

44   Journal of College Science Teaching
Examinations That Support Collaborative Learning

lower, and it is not necessary for stu-           In our introductory physics class,           Impact on student opinions
dents to reach an agreement because           we noted an additional beneficial                For illustrative purposes, we exam-
they usually submit their own (indi-          effect of a two-stage midterm exam:              ine in detail how two-stage exams
vidual) answers. The students also            It increased the engagement during               impacted student opinions in one
know that they will receive expert            in-class collaborative activities fol-           course; however, these results are
feedback from the instructor fol-             lowing the examination. Although                 similar to what has been seen in
lowing the discussion, so they don’t          students regularly participated in               other science courses.
have to evaluate as carefully what            peer discussions of clicker questions               We gave both the midterm and
their colleagues are saying. Finally,         and worksheet problems before the                final exams in a two-stage format in
students are better prepared to carry         midterm and the instructors ex-                  the aforementioned calculus-based
out peer discussions in a two-stage           plained the benefits of collaborative            introductory physics course. The
exam than they are during lecture             learning, it appears that the two-stage          students filled out a 20-question
because (a) they have studied for the         exam convinced the students (more)               online survey after the final exam;
exam, and (b) each student is forced          of the value of peer discussions. It is          four questions probed their views on
to think deeply about the questions           also possible that, after the midterm            the exams. Of the 179 students, 123
during the individual portion of the          exam, students think of the in-class             completed the survey. Eighty-seven
exam before the discussion starts in          activities as more directly related to           percent supported the use of the
the group portion.                            exams.                                           two-stage exam format for midterms,

  Coding scheme and results as applied to students’ written comments regarding their experience with two-
  stage exams in Physics 101.

                                                                                                               No. of times
   Overall        Detailed                                                                                     mentioned
   code           code       Description of code                                                               (N = 123 students)

   General        G-E        Good, enjoy, benefit, great, liked, useful, OK, interesting                       56
   (Total: 236)   H          Helpful                                                                           30

                  C          Increased confidence                                                              9

                  LE         Good learning experience, good way to review exam                                 21

                  LE-D       Learning from: discussions with others, hearing other approaches, comparing       48
                             with others, explaining yourself, collaborating

                  IF         Immediate feedback: good to know if right or wrong                                34

                  IF-LM      Immediate feedback: learning from mistakes                                        16

                  GD-pos     Positive mention of group working together, group members, meeting
                             friends, group preparation, cooperation, and references to grade boost            22

   Neutral/       Misc       Random comments not fitting into the above categories as well as                  15
   other                     suggestions

   General        NEG-gen    Negative mention of group not working so well together, not everyone
   negative                  pulling their own weight, hard to explain to others, and concerns about unfair    15
                             grade boost to weaker student, not fair for the individual

                  NEG-em     Dislike, frustrating, not helpful, feeling sad or depressed, less confident       15

                                                                                                              Vol. 43, No. 4, 2014    45
whereas 74% supported the use for
                                                FIGURE 3
     both midterm and final exams. A
     possible reason for the difference        Student survey results on group decision making (N = 123). Students
     could be that students view the mid-      were asked: “During the group exam, my group usually____.” Full
     term as being part of learning and        answer choices (left to right): “discussed EACH question until ALL
     perhaps feedback on their studying,       members agreed on an answer and explanation,” “took a VOTE and
     whereas they see the final exam as        if unanimous moved on, otherwise discussed the question until all
                                               members agreed on an answer,” “took a VOTE and used the MAJORITY
     a kind of “certification,” similar to
                                               to determine the answer,” “USED the answers from the ONE PERSON in
     many instructors. Many students see
                                               the group who knew the most physics.”
     this course as their final exposure to
     physics, so although students may
     see the second-stage feedback on
     what they did wrong for the mid-
     terms as productive, they may not
     appreciate it as much for the final,
     where there is no hope of using the
     feedback for future improvement. To
     explore this further, one would need
     to conduct interviews with students.
        The survey included a question
     in which students were asked to
     describe their experience with the
     group exam in one or two sentences.
     All students who completed the
     survey answered this question. As
     shown in Figure 2, most students had
     a generally positive opinion.
        The detailed analysis and coding
     scheme we developed for classifying
     the comments is shown in Table 1.          always commented why I chose            These comments give us insight
     Many students’ responses fell into         the answer that I did and our        into why students generally value
     multiple categories; from the 123          group would discuss it. I think      two-stage exams: they felt it was
     students, we coded 283 comments.           I was also very lucky to meet        a good learning experience, and a
     The comments were coded inde-              kind people during lecture.” (H;     good way of reviewing the exam,
     pendently by each researcher and           MISC; LE-D; GD-pos)                  they learned from discussing with
     then compared. The interrater reli-      • Student C: “It was sort of de-       other students and hearing other
     ability for the comments was 95%,          pressing to know what you got        students’ approaches, they enjoyed
     with differences in the coding being       wrong right after writing the        working together, and they valued
     discussed until an agreement was           exam. I think it ends up being       the immediate feedback in the group
     reached. A few examples of student         worth it, though, because you        part.
     comments and coding are as follows:        learn from your mistakes and the        Students also expressed concerns
                                                way classmates explain things        about the exam format in 30 negative
     • Student A: “It was a good experi-        could be easier to understand at     comments, half relating to group work
       ence since going over the exam           times than the way it’s explained    and the other half to the emotional im-
       with my peers reassured me               in the textbook.” (NEG-em G-E;       pact of getting immediate feedback.
       about my answers. As well, I was         IF-LM; LE-D)                         However, of the 15 students who
       able to learn from my mistakes         • Student D: “The group exam           criticized the group work, only six
       through the group exam.” (G-E;           was interesting and a good op-       students rated their overall experience
       C; IF-LM)                                portunity to go over the answers     on the two-stage exam as negative.
     • Student B: “It was surprisingly          and talk about the questions. Just   Three of the six students mentioned
       very helpful. I would say I con-         did not like when some members       concerns about “weak students un-
       tributed as much as I could.             did not do anything.” (G-E, LE,      fairly gaining marks.” Nine students
       When I got a different answer I          NEG-gen)                             were critical of the group work but

46   Journal of College Science Teaching
Examinations That Support Collaborative Learning

still had an overall positive experience   the overall coherence of any course            1: Group versus individual problem
(as Student C). Fifteen students com-      that is using techniques of collabor-          solving. American Journal of
mented that it was “sad” or “depress-      ative learning and formative assess-           Physics, 60, 627–636.
ing” to learn about their mistakes,        ment, as well as allowing students          Johnson, D. W., Johnson, R. T., &
but nine students still had an overall     to learn while completing the exam.            Smith, K. A. (2011). Lecturing
positive view.                             We therefore highly recommend                  with informal cooperative learning
    In the survey students were            this exam format to any instructor             groups. In J. Cooper & P. Robinson
specifically asked about how their         looking to add a formative element             (Eds.), Small group learning in
group reached consensus. The               to their summative assessments. n              higher education: Research and
results are presented in Figure 3.                                                        practice (p. 46). Oklahoma City,
Clearly, most students worked on           Acknowledgments                                OK: New Forums Press.
the group exam in the intended col-        The authors gratefully acknowledge the      Leight, H., Saunders, C., Calkins,
laborative way. Only three students,       Carl Wieman Science Education Initiative       R., & Withers, M. (2012).
two of whom commented about bad            (CWSEI) for funding and support. We            Collaborative testing improves
dynamics in their group, claimed to        thank Carl Wieman for assistance with the      performance but not content
have “used the answers from the one        preparation of the manuscript. We also         retention in a large-enrollment
person in the group who knew the           thank all the CWSEI Science Teaching           introductory biology class. CBE—
most physics.” These responses sup-        and Learning Fellows for providing             Life Sciences Education, 11,
port our observations of classwide         information on their experiences with          392–401.
participation in the second stage of       two-stage exams, in particular Brett        Mazur, E. (1997). Peer instruction: A
the exam and the intensity of the          Gilley and Bridgette Clarkston.                user’s manual. Upper Saddle River,
physics discussions that ensue.                                                           NJ: Prentice Hall.
                                           References                                  National Research Council. (2012).
Summary                                    Black, P., & Wiliam, D. (1998).                Discipline-based education
Two-stage exams are valuable in-              Assessment and classroom                    research: Understanding
structional tools that offer a com-           learning. Assessment in Education:          and improving learning in
bination of formative learning and            Principles, Policy and Practice, 5,         undergraduate science and
assessment. They can easily be                7–74.                                       engineering. Washington, DC:
implemented in many courses and            Cohen, D., & Henle, J. (1995, July).           National Academies Press.
are popular with students and fac-            The pyramid exam. UME Trends,               Available at http://www.nap.edu/
ulty members who use them. Sur-               10, 2, 15.                                  catalog.php?record_id=13362
veys show that this exam format is         Cortright, R. N., Collins, H. L.,           Rao, S. P., Collins, H. L., & DiCarlo,
popular with students for the right           Rodenbaugh D. W., & DiCarlo,                S. E. (2002). Collaborative
reasons—students recognize the                S. T. (2003). Student retention             testing enhances student learning.
value of immediate feedback that              of course content is improved               Advances in Physiology Education,
takes place and the learning that             by collaborative‐group testing.             26, 37–41.
results. The exam format is similar           Advances in Physiology Education,        Stearns, S. A. (1996). Collaborative
to the collaborative in-class activi-         27, 102–108.                                exams as learning tools. College
ties and therefore strengthens the         Gilley, B. H., & Clarkston, B. (2014).         Teaching, 44(3), 111–112.
link between exams and the peer               Collaborative testing: Evidence of       Zipp, J. F. (2007). The impact of two-
instruction activities in class. We           learning in a controlled in-class           stage cooperative tests. Teaching
have noted an increase in engage-             study of undergraduate students.            Sociology, 35, 62–76.
ment during in-class peer activities          Journal of College Science
after a group midterm exam. Fur-              Teaching, 43(3), 83–91.                  Georg W. Rieger (rieger@phas.ubc.ca) is
ther studies are necessary to estab-       Heller, P., & Hollabaugh, M. (1992).        an instructor 1 in the Department of Phys-
lish that this is mainly a result of          Teaching problem-solving                 ics and Astronomy and a member of the
the two-stage exam. It would also             through cooperative grouping.            Carl Wieman Science Education Initiative
be interesting to find out if students        Part 2: Designing problems and           (CWSEI), University of British Columbia
acquire better group skills through           structuring groups. American             (UBC), British Columbia, Vancouver, Cana-
participation in the group part of            Journal of Physics, 60, 637–644.         da. Cynthia E. Heiner was a teaching and
the exam. The experience in our            Heller, P., Keith, R., & Anderson, S.       learning fellow in the CWSEI at UBC at the
science faculty has shown that the            (1992). Teaching problem solving         time the article was written and is now at
two-stage approach contributes to             through cooperative grouping. Part       the Free University Berlin in Germany.

                                                                                                        Vol. 43, No. 4, 2014        47
You can also read