Adopting and adapting the mini-CEX as an undergraduate assessment and learning tool
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Practical
Assessment
Adopting and adapting
the mini-CEX as an
undergraduate
assessment and learning
tool
Faith Hill and Kathleen Kendall , School of Medicine, University of Southampton, UK
I
n Southampton, we intro- Medicine1 and evaluated by examiners rate student perfor-
duced an innovative clinical John Norcini.2,3,4 This paper de- mance along several predeter-
assessment to the final year scribes how we adopted and mined dimensions, and provide
of our BM programme in 2004. adapted the mini-CEX for use in an immediate feedback to students.
This was designed to replace the undergraduate context. We This process enables observation
long-case assessment at the end address the feasibility and and feedback by different exam-
of attachments, and to increase acceptability of introducing this iners in a variety of settings,
Examiners rate the opportunities for formative new assessment tool into an across a range of patient prob-
and summative assessment of established programme. lems. For the mini-CEX at South-
student students in clinical settings. The ampton, students are assessed on
performance innovation was modelled on the The mini-CEX involves a short, three different occasions in each
along several mini-CEX (clinical evaluation focused observation of students of five different clinical attach-
predetermined exercise) developed by the interacting with patients. Using ments, across a wide range of
American Board of Internal scannable standardised forms, clinical problems of varying com-
dimensions
244 Blackwell Publishing Ltd 2007. THE CLINICAL TEACHER 2007; 4: 244–248
student comments. Student par- We were aiming
ticipants in the evaluation were to improve not
recruited through e-mail invita-
tions, and purposive sampling was
only our
used in order to ensure that a summative
diverse range of participants was assessments
included from across the region, but also the
and from a range of attachments. learning
Staff were also sampled purpo-
sively to include course co-ordi-
experience of
nators and others with our students
responsibility for student assess-
ment, both in the NHS and in the
School of Medicine. An initial
interview schedule, containing
questions about experiences and
perceptions of the mini-CEX was
piloted and revised in response to
feedback. The interviews, lasting
between 20 and 75 minutes, were
conducted by an independent and
experienced researcher and were
professionally transcribed. The
student comments were collected
from feedback forms following
each attachment. The data was
plexity. The setting tends to be based on a wide range of data analysed systematically using the
ward-based (68 per cent in sources, including: framework approach. This ap-
2004/5) and the patients repre- proach outlines a well-defined
• 2,340 completed forms (156 procedure for data analysis that
sent a broad spectrum of ages.
students) (2004/5) involves classifying and organis-
Apart from Obstetrics and Gynae-
cology, there is only a small • 1,146 completed forms (177 ing data according to key themes
students) (2005/6) and issues. It is recognised as a
gender disparity (in 2004/5, 54
method particularly suited to
per cent of patients were male).
• 33 tape-recorded, semi-struc- applied research.5
tured individual interviews
Students have a different
(2004/5) ADOPTING AND ADAPTING
examiner for each assessment and
are observed with different • 4 tape-recorded group inter- THE MINI-CEX FOR
patients. Students are expected to views with both staff and UNDERGRADUATE
initiate the assessment, but each students (2004/5) ASSESSMENT
attachment centre is responsible
• 19 tape-recorded individual
for providing examiners and Since the mini-CEX was a new and
interviews with staff and stu-
selecting patients. Pre-carbonised unfamiliar assessment tool, the
dents (2005/6)
multi-copy forms allow both school recognised that its intro-
examiner and student to retain a • Students’ comments on feed- duction would be likely to provoke
copy of the assessment, and for back forms some concern among both stu-
additional copies to be forwarded dents and staff. Indeed, a number
This article focuses on the
by the centres to the School of staff were initially sceptical
qualitative aspects of the evalua-
Office. The forms are then scanned about the introduction of the
tion, describing the key issues
centrally and the results entered mini-CEX. Therefore, prior to its
identified during the adoption of
into the students’ records. implementation, the school em-
the mini-CEX and how the mini-
CEX was adapted in response to phasised that an evaluation would
We have made a number of be undertaken to identify prob-
these issues. The quantitative
changes to the mini-CEX since its lems as they arose. In addition, a
data, including an analysis of
introduction in response to the videotape about the mini-CEX,
reliability, will be reported in full
findings of a rigorous evaluation which included mock consulta-
in a separate paper.
conducted in 2004/5 and 2005/6. tions, was produced and circu-
The evaluation used both quali- lated. Information about the
The qualitative data included
tative and quantitative methods, mini-CEX was also presented at a
student and staff interviews, and
Blackwell Publishing Ltd 2007. THE CLINICAL TEACHER 2007; 4: 244–248 245Many of the number of meetings attended by and feedback was indicated more inations. In addition, some stu-
concerns both staff and students. clearly in the guidelines. dents reported that the mini-CEX
has increased their overall level of
expressed by Many of the concerns ex- The mini-CEX forms used in study time and motivation to learn
staff and pressed by staff and students were 2004/5 and the calculations for because of the recurrent nature of
students were practical – including, for example, an overall pass mark also raised three mini-CEX’s in each attach-
practical the availability of examiners as some issues. There were concerns ment. It is no longer feasible for
well as the time and organisation that the 9-point rating scale students to miss any stage of an
required. However, in practice, being used was too broad. Objec- attachment, or to rely heavily on
these problems did not arise. For tions were also raised about the library-based learning at the ex-
example, it proved possible to agreed calculations for an overall pense of work with patients.
find enough clinical examiners for pass mark that would exempt Examples of the students’ com-
all the students, even in the first students from further mini-CEX ments are given in Table 3.
year. In addition, time and or- assessment in their final examin-
ganisational issues were found to ations. There was a perception A particularly important influ-
be less, rather than more, prob- that the assessment criteria were ence on students’ learning was
lematic than with long-case too lenient, possibly allowing a the opportunity to be observed
assessment. small number of students to pass and receive immediate feedback
their mini-CEX’s inappropriately. on multiple occasions from a
Staff and students also ques- We therefore modified the form range of examiners. Students felt
tioned the reliability of using a (see Table 1) and strengthened that the feedback from the
single examiner in the mini-CEX the criteria for passing (see examiners on their clinical skills
compared to the paired scheme Table 2). These changes appear to helped them to identify their
that had been used for long-case have been successful. Examiners strengths and weaknesses, and to
assessment. Examiners working on and students prefer the new use this information to improve
their own might be reluctant to 6-point scale. The number of and enhance their skills. The
give a poor mark, particularly as students failing to earn exemp- mini-CEX also provides a useful
they had to give immediate feed- tions (and therefore repeating opportunity to identify student
back to students. There was also their mini-CEX’s in their final problems at an early stage within
concern about the standard ex- exams) has increased to the level an attachment. Students valued
pected, and students felt there we would normally expect. highly the one-to-one ‘teaching’
was a poor correlation between associated with the mini-CEX and
the comments to students during THE MINI-CEX AS A felt this was unlikely to have
feedback and the written mark. LEARNING TOOL occurred without the assessment.
Finally, some staff and students
felt that there was a great deal of In introducing the mini-CEX at Concerns were expressed by
variation across examiners and Southampton we were aiming to some examiners that the focused
attachments in the time spent improve not only our summative history taking in the mini-CEX
observing students and giving assessments but also the learning might encourage students to view
feedback. experience of our students patients in a narrow, de-contex-
through formative assessment. tualised way. The importance of
In response to these concerns, We knew from the literature that taking a more holistic approach to
renewed emphasis was placed on assessment drives student learn- patient problems was expressed in
staff development, and each of the ing,6 and were expecting that particular by some examiners from
speciality co-ordinators devised introducing this innovative form the child health attachment.
plans for disseminating good of assessment would lead to At Southampton, we are intending
practice in the use of the assess- important changes in the way our to address this concern by the
ment. Increased attention was students learn. The qualitative increased use of student portfolios
paid to quality control, and more evaluation showed this to be the to record full patient histories.
detailed guidelines were provided case. For example, students are
for examiners. The expected stan- now very aware that they will be CONCLUSIONS AND
dard was described more explicitly. observed closely and marked on RECOMMENDATIONS
The guidelines now clearly state their history taking, physical
that students should achieve the examination and interaction with We are continuing to monitor and
level that would be expected of patients. For many students, this evaluate the mini-CEX. On the
Renewed a safe and competent doctor at means paying far more attention basis of our findings to date, we
emphasis was the start of the Foundation Pro- to their clinical skills and spend- believe that this assessment is
placed on staff gramme. Finally, the time that ing more time practising focused feasible and acceptable to both
should be allowed for observation history taking and physical exam- students and staff. Despite initial
development
246 Blackwell Publishing Ltd 2007. THE CLINICAL TEACHER 2007; 4: 244–248Table 1. Revised mini-CEX form There was a
perception that
the assessment
criteria were
too lenient
Students valued
scepticism, almost all students learning opportunities provided ing in a clinical setting. In par- highly the
and examiners at Southampton by the mini-CEX are considerable; ticular, there is immediate feed- one-to-one
are now in favour of this innova- and students are provided with back to students who have been
tive form of assessment. The valuable opportunities for learn- observed interacting with pa- ‘teaching’
Blackwell Publishing Ltd 2007. THE CLINICAL TEACHER 2007; 4: 244–248 247ensure that the wide range of
Table 2. Revised mini-CEX marking criteria examiners are marking to the same
Completing the form – using the scale: standard. It is also necessary to
The full range of the rating scale will be used. Comparison will be made with ensure that chosen assessment
a safe and competent doctor at the start of the F1/PRHO year. It is expected criteria can discriminate effec-
that some rates below ‘meet expectations’ will be in keeping with your level tively between competent and
of experience, and you should be aware that a borderline score represents a under-performing students. Final-
referral, rather than a fail. ly, it is very important to consider
Referral (requires repeat mini-CEX at the BM Final exam in the specialties in the mini-CEX in the context of the
which you are referred): overall assessment of students: it
offers a valuable addition to tra-
Five or more borderline or below expectation scores in any of the seven ditional assessments but is not
elements across 3 mini-CEX assessments within an attachment
designed to assess the complete
or range of knowledge, skills and
Three borderline or below expectation scores in any one element within an attitudes required of a modern
attachment (to identify students who have a difficulty in a particular area) medical graduate.
REFERENCES
Table 3. Student comments 1. American Board of Internal Medicine
Clinical Competence Program. The
‘As learning tools they have been fantastic. As an actual occasion to get
mini-CEX: a quality tool in evaluation.
someone... to sit down and properly listen to you and give you genuinely guidelines and implementation strate-
informed feedback... it’s some of the best teaching I’ve had on any of my gies from program directors. Work in
attachments.’ progress. Philadelphia, PA:ABIM Clin-
ical Competence Program, June 2002.
‘I think it makes me do a lot more clinical work on the wards – a lot more –
because I know somebody is going to be watching me examine people, so I 2. Norcini J, Blank L, Arnold G, Kimball
had better know how to do it properly.’ H. The mini-CEX (clinical evaluation
exercise): a preliminary investigation.
‘I found that one of the best features of them actually is that you know once Ann Intern Med 1995;123:795–799.
a week or whenever you get them, you will have a good teaching session and
3. Norcini J, Blank L, Arnold G, Kimball
it’s useful being watched because the criticism they give you is useful and
H. Examiner differences in the mini-
there’s not so many opportunities you get a Consultant to watch you doing a
CEX. Adv Health Sci Educ 1997;2:
full examination on something.’ 27–33.
4. Norcini J, Blank L, Duffy D, Fortna G.
The mini-CEX: a method for assessing
tients. As a summative assess- adapted in response to local con- clinical skills. Ann Intern Med
ment, it is much preferred by both cerns and specific settings. In our 2003;138:476–481.
staff and students when compared experience, any school thinking of 5. Ritchie J, Spencer L. Qualitative data
to the long-case assessment used undertaking the mini-CEX with analysis for applied policy research.
in the past. undergraduates needs to consider In: Bryman A., Burgess RG, eds.
Analyzing qualitative data. London:
a number of issues. First, to ensure
Routledge 1994:173–194.
We are therefore clearly in consistency it is essential to
favour of adopting the mini-CEX. invest in an extensive staff 6. Newble DI, Jaeger K. The effect of
assessments and examinations on the
However, in order for it to be development programme for po- learning of medical students. Med
adopted successfully in an under- tential mini-CEX examiners. Staff Educ 1983;17:1.
graduate context, it needs to be need very clear guidelines to
248 Blackwell Publishing Ltd 2007. THE CLINICAL TEACHER 2007; 4: 244–248You can also read
