Rethinking assessment in education: The case for change - CSE LEADING EDUCATION SERIES
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
02 CSE
LEADING
EDUCATION
SERIES
APRIL 2021
Rethinking assessment
in education: The case
for change
B IL L LUC A S
Acknowledgements Contents
All my colleagues at Rethinking Assessment, especially Rosie 2 Introduction
Clayton, Peter Hyman, Rachel Macfarlane and Al McConville;
to the Edge Foundation for its support; and to the team at UCL 3 The wrong kind of nets for catching
School of Management for their research. young people’s strengths
Many along the way whose work has influenced my thinking
about assessment, including: Michelle Anderson, Ken Baker,
5 An education system fit for purpose?
Geoff Barton, Ron Berger, Sarah-Jayne Blakemore, Roy 12 The problem with educational
Blatchford, Esther Care, Christine Cawsey, Guy Claxton,
Paul Collard, Art Costa, Angela Duckworth, Carol Dweck,
assessment today
Charles Fadel, Sharon Foster, Michael Fullan, Valerie Hannon, 19 Revisiting the purposes of assessment
John Hattie, Lois Hetland, Rosemary Hipkins, David Howes,
Tony Mackay, Geoff Masters, Jonnie Noakes, James Pellegrino, 23 Promising practices from across
David Perkins, Mario Piacentini, Sandra Milligan, Andreas the world
Schleicher, Ellen Spencer, Michael Stevenson, Louise Stoll,
Stéphan Vincent-Lancrin, Dylan Wiliam. 33 Visible progress
38 References
ISSN 1838-8566 ISBN 978-1-925654-57-8
© 2021 Centre for Strategic Education, Victoria.
The Centre for Strategic Education welcomes usage of this
publication within the restraints imposed by the Copyright Act.
Where the material is to be sold for profit then written authority
must be obtained first. Detailed requests for usage not specifically
permitted by the Copyright Act should be submitted in writing to:
The Centre for Strategic Education
Mercer House, 82 Jolimont Street,
East Melbourne VIC 3002
Produced in Australia by Centre for Strategic Education
Mercer House, 82 Jolimont Street, East Melbourne VIC 3002
Editorial Team: Tony Mackay, Keith Redman,
Murray Cropley, Andrew Miller
Rethinking assessment
in education: The case
for change
B ILL L U C A S
Introduction
This paper is the second in the CSE evidence-based in teaching and learning, we
Leading Education Series and a are failing to keep up with the science of
collaboration between CSE in Australia assessment, preferring to rely on outdated,
and Rethinking Assessment in England. outmoded and unsubtle methods.
Across the world assessment is not Our young people require all of us working
working. We are not evidencing the kinds in education to establish greater clarity
of dispositions and capabilities that about the uses of assessment in education,
society increasingly wants. Educational linked to a greater understanding of the
jurisdictions are placing too much reliance science of assessment.
on high-stakes, standardised testing. They
We need nothing less than a paradigm shift
are testing the wrong things in the wrong
in our understanding about how best to
ways. High-stakes assessment is having
create assessment systems that use more
a damaging impact on the health and
effective ways of evidencing the full range
wellbeing of students and it is not giving
of student progress.
universities, colleges or employers the kind
of information they want. Assessment is In addition, we want to move rapidly from
out of sync with curriculum and pedagogy. theoretical debate to practical prototyping
Where we have become increasingly and implementation.
Rethinking assessment in education: The case for change / 2
The wrong kind of nets for
catching young people’s
strengths
To solely use standardised achievement tests is like casting a net into the sea – a
net that is intentionally designed to let the most interesting fish get away. Then, to
describe the ones that are caught strictly in terms of their weight and length is to
radically reduce what we know about them. To further conclude that all the contents
of the sea consist of fish like those in the net compounds the error further. We need
more kinds of fish. We need to know more about those we catch. We need new nets.
(William T Randolph, Commissioner of Education, Colorado1)
Metaphors abound in education. From In the UK, for example, GCSE exams
the Greeks via the Romans we took the routinely fail 33 per cent of all sixteen
idea that a child’s mind was a tabula rasa year olds. The students who do not make
or blank slate. Children, the comparison the grade have become known as the
suggests, know nothing and bring nothing; ‘forgotten third’ (Association of School
all is dependent on the experiences that and College Leaders, 2019). The system
adults offer them. They are empty vessels has sifted ‘sheep’ from ‘goats’, but the
waiting for those more knowledgeable than public, the shepherd in this analogy, has
them to fill up their minds. little understanding of what it all means
and the goats, the third who ‘fail’, are
There are many other
left with nothing much to show for their
metaphors we might draw
ATAR is a ladder in an on that are more cheerfully
compulsory schooling.
educational game of expansive: a search for hidden In Australia, the Australian Tertiary
snakes and ladders, treasure; an odyssey; discovery; Admission Rank (ATAR) is a kind of
whose higher rungs challenge. Randolph net, too. The score out of 100
hold out a promise of When it comes to assessment,
gives Australian youth a certain kind of
success, which turns out weight and length and then produces
William Randolph’s
a rank order. ATAR is a ladder in an
to equate to abstract thoughtlessly designed net
educational game of snakes and ladders,
rather than to real- seems an apt image for our
whose higher rungs hold out a promise
world intelligence. times. For, in different ways,
of success, which turns out to equate
educational assessment
to abstract rather than to real-world
systems across the world
intelligence.
have become very good at weighing and
measuring students, without reflecting on The Randolph net metaphor originated
whether the assessments they are making in the USA, where, notwithstanding
are relevant, meaningful or useful, and considerable varieties in provision between
without considering the consequences of states, it is reasonable to assume that it
the assessment process. has currency as a provocation beyond
Colorado.
3 / CSE Leading Education Series #02 April 2021
Words matter too. The nouns and verbs we Each of these words says something
use in connection with assessment come about the kind of learning imagined, the
freighted with semantic and educational method by which it might be assessed, the
baggage. perspective from which such assessment is
made and the validity or value that might
achievement, attainment, assessment,
be placed on the approach being described.
baccalaureate, badge, (balanced) score-
card, competition, curriculum vitae, As we unpack the practices of assessment,
demonstration, diploma, evidence, it may be helpful to stay close to the
exam, exhibition, expedition, feedback, words which have least baggage, such
illustration, interview, observation, as ‘evidence’ or ‘record’ (noun and verb)
passport, performance, portfolio, and phrases like ‘track the progress of’.
presentation, profile, project, publication, Too often we invest the scores and grades
qualification, record, score, score-card, used in end of school qualifications with a
task, test, transcript, viva … scientific validity they do not possess.
achieve, attain, assess, curate, We need new educational nets to catch
demonstrate, display, evidence, more of the capabilities young people need
examine, exhibit, illustrate, measure, to thrive today.
present, qualify, record, score, test,
track (the progress of) …
Rethinking assessment in education: The case for change / 4
An education system
fit for purpose?
The pervasive obsession with academic grades and degrees, and corresponding elite
rewards at the expense of other people … results in narrow learning that severely
distorts what people learn and need in the 21st century.
Michael Fullan, 2021, p 8
Across the world there has been growing The beginnings of a global
discontent with the content of school
curricula, ever since the arrival of the curriculum
twenty-first century with its attendant Today there are a dozen or so well regarded
millennial symbolism. In addition to models of what contemporary curricula
traditional subjects such as literacy, maths should look like. The World Economic
and science, it is widely argued that Forum (2015) is widely cited (see Figure 1).
schools need to focus on what students can
Whether framed as foundational literacies,
do and who they are becoming.
competencies or character qualities, it is
increasingly recognised that, in our digital
age, there are more core literacies than
Figure 1. 16 skills for the twenty-first century (World Economic Forum, 2015)
21st century skills
Foundation literacies Competencies Character qualities
How students apply core How students approach How students approach
skills to everyday tasks complex challenges their changing environment
1. Literacy 7. Critical thinking/ 11. Curiosity
problem solving
2. Numeracy 8. Creativity 12. Initiative
3. Scientific 9. Communication 13. Persistence/grit
literacy
4. ICT literacy 10. Collaboration 14. Adaptability
5. Financial
literacy 15. Leadership
6. Cultural and
civic literacy 16. Social and cultural
awareness
LIFELONG LEARNING
5 / CSE Leading Education Series #02 April 2021
Figure 2. Center for Curriculum is the one developed by the Center for
Redesign 4D Framework 1.02 Curriculum Redesign (see Figure 2).
There are many variants of such
contemporary curricula – of which the one
Knowledge Guy Claxton and I developed, Educating
‘What we know and understand’
Interdisciplinarity Ruby: What Our Children Really Need
Traditional (ie, Mathematics)
Modern (ie, Entrepreneurship) to Learn (2015), focusing on Confidence,
Themes (ie, Global Literacy)
Curiosity, Collaboration, Communication,
Creativity, Commitment and Craftmanship
is one. The approach adopted in New
21st
Century Pedagogies for Deeper Learning3 (Character,
Skills Learner Character Citizenship, Collaboration, Communication,
‘How we use ‘How we behave
what we know’ and engage in Creativity and Critical Thinking) is another.
the world’
We are building on the 3Rs of old to develop
Creativity Mindfulness
Critical Thinking Curiosity the 6 or 7Cs of today.
Communication Courage
Collaboration Resilience
Ethics Whether people like or do not like
Leadership
the framing of these dispositions as
Meta-Learning twenty-first century skills, or students as
‘How we reflect and adapt’
Metacognition twenty-first century learners (I do not,
Growth Mindset
finding the phrases vague, misleading
© Centre for Curriculum Redesign
and somewhat evangelical), there is now
substantial common ground as to what
we once thought. Whether we use words these dispositions or wider skills are.
like ‘competencies’ or ‘character ‘(or both) Importantly this consensus includes
there are certain important dispositions or perspectives from educationalists, parents,
capabilities for living a good life, and for psychologists and researchers, as well as
being a good learner, which schools have employers.
a role in cultivating.
Table 1 is an overview of different
Another model with a broadly similar evidence-based lists of such dispositions
framing of a contemporary curriculum (Lucas, 2019).
Table 1. Dispositions for a lifetime of learning (Lucas, 2019)
European Key Pellegrino and Gutman and Heckman and Lamb et al, 2017
Competences for Hilton, 2012 Schoon, 2013 Kautz, 2013
Lifelong Learning,
2007
• Communication • Critical thinking • Motivation • Perseverance • Critical thinking
in mother tongue • Information literacy • Perseverance • Self-control • Creativity
• Communication in • Reasoning • Self-control • Trust • Metacognition
foreign languages
• Innovation • Metacognitive • Attentiveness • Problem-solving
• Digital strategies
competence • Intellectual • Self-esteem and • Collaboration
openness • Social self-efficacy • Motivation
• Learning to learn competencies
• Work ethic • Resilience • Self-efficacy
• Social and civic • Resilience to adversity
competences • Conscientiousness • Conscientiousness
and coping • Openness
• Sense of • Positivity • Perseverance
• Creativity to experience
initiative and • Communication
entrepreneurship • Empathy
• Collaboration
• Cultural • Humility
• Responsibility
awareness and • Tolerance of
expression • Conflict resolution diverse opinions
• Engaging
productively
in society
Rethinking assessment in education: The case for change / 6
A generic term, expansive education, Of course curriculum is only one of
has been developed by the author and the three core elements of education
colleagues at the University of Winchester, systems, the other two being pedagogy or
(Lucas, Claxton and Spencer, 2013) to instruction (how the curriculum is taught)
describe the kinds of dispositions that are and assessment (how performance and
desirable for success at school and in life, progress is evidenced). Of significance for
and how these can be interleaved into the this report is the chronic disjoint between
subject disciplines of the school timetable. curriculum, pedagogy and assessment
with, as yet, no focus or guidance on
Research by the Brookings Institution
the teaching or assessment of these
(Care et al, 2016) has shown that,
competencies/dispositions.
across the world, such dispositions
are gradually beginning to filter their Interestingly, the same Brookings research
way into schools, with 36 countries shows a kind of league table of progress in
mentioning them explicitly, 76 countries developing more expansive curricula in
identifying skills related to them, 51 countries and states across the world (see
locating them within the curriculum Table 3).
and 11 mapping their progression over
While the research does not cover every
the lifetime of formal schooling. The
educational jurisdiction in the world, it is
scope and sequence documents of the
worth noting that the countries and states
Australian capabilities are an example of
making most progress in implementing
the last of these categories. Recently the
new thinking about contemporary
Brookings Institution, using the Center
curricula according to this report are
for Curriculum Redesign model, has
Australia, British Columbia (Canada),
produced an overview of the prevalence
Singapore, Finland, Hong Kong, Victoria
of dispositions/competencies in different
(Australia) and New Zealand.
educational jurisdictions (see Table 2).
Table 2. The prevalence of Center for Curriculum Redesign competencies,
(Taylor et al, 2020)
Competency Inclusion Identification Progression Pedagogy Assessment
Creativity 21 12 5 0 0
Critical thinking 21 11 6 0 0
Skills
Communication 22 11 5 0 0
Collaboration 21 10 6 0 0
Mindfulness 17 10 5 0 0
Curiosity 17 7 3 0 0
Character
Courage 9 5 5 0 0
Resilience 15 8 6 0 0
Ethics 18 10 4 0 0
Leadership 10 7 4 0 0
learning
Metacognition 14 7 5 0 0
Meta-
Growth mindset 14 6 5 0 0
7 / CSE Leading Education Series #02 April 2021
Table 3. The frequency of competencies/dispositions across jurisdictions
(Taylor et al, 2020)
Jurisdiction CRE CRI COM COL MIN CUR COU RES ETH LEA MET GRO Total
Australia (Federal) 3 3 3 3 3 3 3 3 3 3 3 3 36
British Columbia (Canada) 3 3 3 3 3 3 3 3 3 3 3 3 36
Singapore 3 3 3 3 2 3 3 3 3 2 2 3 33
Finland 3 3 3 3 3 3 3 2 3 3 3 32
Hong Kong (China SAR) 2 3 3 3 3 1 3 3 2 2 3 3 31
Victoria (Australia) 3 3 1 3 3 1 3 3 3 3 26
New Zealand 2 2 2 2 2 2 2 2 2 2 2 22
Portugal 2 2 2 2 2 2 2 2 16
Chinese Taipei (aka Taiwan) 2 2 2 2 2 2 2 14
Denmark 2 1 1 1 1 1 1 1 1 1 11
England (UK) 2 1 1 1 2 1 1 1 1 11
Scotland (UK) 1 1 1 1 1 1 1 1 1 1 1 11
South Korea 2 2 2 1 2 1 1 11
Alberta (Canada) 1 1 1 1 1 1 1 1 1 1 10
New Brunswick (Canada) 1 2 2 1 1 1 1 1 10
New South Wales (Australia) 1 1 1 1 1 1 1 1 1 1 10
Massachusetts (USA) 1 1 1 1 1 1 1 1 1 9
Ontario (Canada) 1 1 1 1 1 1 1 1 1 9
China 1 2 2 1 2 1 9
USA (Federal) 1 1 1 1 1 1 9
Japan 1 1 1 1 1 5
Russia 1 1 1 1 4
Total 38 38 38 37 32 27 19 29 32 21 26 25
Researchers looked at the frequency of mentions in curriculum documents across 5 categories - Competency inclusion, Competency
identification, Competency progressions, Competency pedagogies and Competency assessments. A “3” (blue) indicates that the jurisdiction
has identified that competency in 3 categories, a “2” (orange) indicates that the competency was identified in two categories and so on.
Differing approaches to that there is a middle way; that these are
false binary positions.
teaching and learning
In such a mid-position we might be asking
In terms of the kinds of pedagogy/ questions such as:
instruction needed today to develop both
the foundational literacies and the kinds of What kind of knowledge is it important
desirable dispositions listed in Table 1, the for all young people to have?
educational world has become unhelpfully What kind of dispositions is it important
polarised. for all young people to acquire?
One group, broadly those who might see How can we ensure that young people
themselves as traditional, tends to argue acquire and apply useful knowledge in
for the teacher’s role in transmitting a range of settings?
knowledge and to favour didactic methods.
The other, typically seen as progressive, How can we teach young people to
argues for student-led approaches, such as work across subject disciplines, as
problem-based learning. With Guy Claxton happens in the real world, ensuring that
(Claxton and Lucas, 2015) I have suggested they have the necessary building blocks
in place?
Rethinking assessment in education: The case for change / 8
How can we ensure that important A misunderstanding of the role
dispositions for learning and for life are
best cultivated in a range of disciplinary of skills in learning
contexts? There is much nonsense talked about skills
How can we develop strength, breadth today.
and depth in learning to facilitate its By those who see the acquisition of
transfer across contexts? knowledge as the main purpose of
Which pedagogies work best for education, an emphasis on skills is
promoting deep learning? often portrayed as an attempt to dumb
down or distract schools from their core
How best to assess knowledge and purpose.
evidence dispositions?
By those who see dispositions and
As argued so far, there is emerging capabilities as being centrally important,
agreement as to the answers to the first two there is a temptation to hold fast to
questions, with continuing discussions bigger concepts, such as creativity or
about the other six. collaboration, without recognising that
There are many other questions we they are in reality made up of aspects of
could pose with regard to pedagogy or knowledge and clusters of skills.
instruction, and many sources of evidence Knowledge and dispositions are not
on which educators can draw (Hattie, polar opposites, just different ways
2008; Committee on Developments in the of categorising what we can learn.
Science of Learning, 2000; Coe et al, 2020). The ‘currency’ of both is skills. Skills
Importantly, any teacher reaching for such are what matter in life. Skills are the
guidance will need to consider not just ‘connective tissue’ between knowledge
which teaching methods promote success and dispositions. As we practise a skill
in terms of typical examinations, but in different contexts we become more
which methods also cultivate competent, confident and capable, until
the kinds of dispositions or it becomes a disposition, something we
Skills are what matter capabilities young people need. are disposed to do. Some examples, from
in life. Skills are the Ideally, methods that effectively simple to more complex, might include
‘connective tissue’ promote both outcomes will be
planning an essay;
between knowledge chosen.
and dispositions. delivering a speech;
critiquing an argument;
having a good idea when you need one;
tying your shoelace;
9 / CSE Leading Education Series #02 April 2021 trying different approaches when faced Learning 2:
with a tricky problem; Learning is the ability to consciously
using your common sense when your modify understandings, beliefs and actions
satnav takes you to a cul-de-sac not yet in response to evidence, experience, and
updated in its software; reflection.
reading the mood of those with whom Schooling is one of many environments in
you are working; which humans develop the capability to
exercise judgement and control over what
facilitating a workshop where you are a
they learn, how they learn, and what they
content expert; or
intend to do with what they have learned.
facilitating a workshop when you have
Assessment is the means by which
only a basic knowledge of the context
individuals receive useful information
but can transfer facilitation skills learned
about the development of their capabilities
in other contexts to the task at hand.
as learners over time.
And, yes, recalling (Elmore, 2019, p 333)
decontextualised information
Assessment influences In this paper it is the second of these two
in a pencil and paper
not just what gets taught examination is a skill, but not conceptions that we shall be exploring.
but how it gets taught one that adults need to use
much in a digital age.
The tail that wags the dog
The deeper your knowledge and the more
you practise your skills in a variety of So, to assessment. Almost anyone who has
contexts, the more capable you become. worked in education knows that what gets
Dispositions are clusters of skills which assessed by and large gets taught. You can
have been practised so well that they have a bold and expansive curriculum, but
have become habitual; you are routinely as the time of examinations draws close,
disposed to deploy them; and skills are the the focus shifts to those aspects of the
mechanism by which knowledge is applied curriculum which will be assessed. This is
and dispositions are lived out. especially true in upper secondary schools
as students reach the age when they move
on to university, vocational training or
Learning 2.0 employment. The decisions are complex
for young people as they navigate their
Richard Elmore (2019) helpfully next steps, and the means by which such
summarises the consequences for school of decisions are made are often by ‘high-
two very different conceptions of learning, stakes assessment’.
as follows.
Assessment influences not just what gets
Learning 1: taught but how it gets taught. If Teacher
Learning is the ability to recall and deploy X uses a particular teaching method for
information and algorithms accurately and science with her class and students do
appropriately. well in their assessments, while Teacher Y
uses a different method and her class does
Schooling is the mechanism by which we
less well, then, assuming the classes share
organise social and status consistent with
similar enough characteristics, schools and
this definition of learning.
school systems will begin to draw lessons
Assessment is the means by which we from this. Reasonably enough they will
define, measure, evaluate, and confer ‘merit’, suggest that when teaching science the
consistent with this definition of learning. methods chosen by Teacher X are the ones
to use.
Rethinking assessment in education: The case for change / 10Most education systems
are seriously in need of
attention if they are to
be fit for purpose.
At first sight this is an intelligent system’s Most education systems are seriously in
response, but what if assessments in need of attention if they are to be fit for
science privilege decontextualised purpose. Curricula are changing, debates
recall of scientific theory and simplistic are at least being had about pedagogy
memorisation of scientific facts, which (Griffin, McGaw and Care, 2012; Vincent-
neither encourage students to think and Lancrin, et al, 2019) but, despite some
work like scientists nor equip them to promising initiatives, assessment needs
go on to deeper study of science and some serious rethinking.
its uses in society? What if thinking
Eight years ago Geoff Masters suggested
about assessment is not keeping up with
that the ‘field of educational assessment
advances in the learning sciences? In these
is currently divided and in disarray’ in
cases such a response would be dumb.
Australia (Masters, 2013, p 1). I suspect
The dog in the sub-heading of this section that this is still the case in Australia and
is the school system and the tail that wags still the case for the majority of educational
it is, of course, assessment. Also, mixing jurisdictions across the world today.
my metaphors, the tails which seem to wag
Let us turn now to what is wrong with
so many school systems across the world
educational assessment in more detail.
are the fishing nets with which this section
began.
11 / CSE Leading Education Series #02 April 2021The problem with educational
assessment today
The measurement of deep learning must be always informed by a wealth of
underlying assessment evidence that captures the complete picture of who students
are, what they know and whether they are prepared to use that knowledge to advance
their lives and others.
(Joanne McEachen, Assessment for Deep Learning, 2017, p 12)
There are many aspects of educational The risk is that schools create students
assessment today which are failing. These dependent on direct instruction,
fall into the four broad areas of cramming, drilling and coaching,
reliant on expert instruction by teachers
what is assessed (focus);
who are expected to guide learners
how it is assessed (methods); through a carefully prescribed body of
the impact of the assessment process knowledge, assessed in predictable ways.
(consequences); and (p 14)
the uses made of the assessment
(validity).
An assessment focus that is
Of course there is also a fifth
too shallow and too narrow
challenge: the degree to which
Complex, higher order whatever we might want Currently, the knowledge that is typically
skills are rarely assessed to measure can be reliably assessed is from a narrow range of subjects,
assessed. rarely explored in depth and almost never
in ways that recognise interdisciplinary. Practical knowledge
the subtleties involved. In a recent review (2020a)
and skill is not much assessed in general
Many dispositions or Sandra Milligan and colleagues
education, and individuals rather than
cut across all of these
capabilities known to teams remain the focus. Complex, higher
categories elegantly when they
be important in life are suggested that
order skills are rarely assessed in ways that
not assessed at all. recognise the subtleties involved (Darling-
Without a focus on mastery of Hammond, 2017). Many dispositions or
generic capabilities, assessment capabilities known to be important in life
and teaching practices tend to privilege are not assessed at all.
memorisation, essay writing, individual Assessments frequently require recall
mastery of set content and solving of of content but rarely demand the kind
problems with formulaic solutions. of deep thinking, problem solving or
application needed in the real world.
Rethinking assessment in education: The case for change / 12 Traditional areas, literacy, maths and Assessment methods that
science continue to require considerable
content to be tested, while newer are too blunt
areas such as citizenship, engineering, Most tests used in schools still rely on
sustainable development and ethical paper and pencil. They examine aspects
understanding are only briefly explored. of knowledge and routine skills. They
Except in a very few countries (Finland test students’ ability to remember and
and Singapore are examples) there is little write about something, rather than apply
or no interdisciplinary assessment.4 or do the thing they have been learning.
Concepts and skills are tested in individual
Practical knowledge and skill is rarely subjects and only very rarely across
assessed even in those subjects where disciplines.
it once used to be a central component,
such as science. While tests often purport to be criterion-
based, many countries effectively revert
Students’ capabilities in planning and to norm-referencing either because of the
undertaking extended investigations are scale used (the ATAR in Australia, for
rarely assessed. example), or the external moderation by
Although the ability to collaborate an accountability body that keeps levels
with others is widely valued in the of achievement very similar year on year
workplace it is only acknowledged at (as with GCSE in England). Even where
school on the sports field or in music tests are explicitly criterion-based, grades
and drama performances. often relate to syllabus content, rather
than to more carefully sequenced learning
While dispositions or capabilities are progressions.
becoming more visible in curricula
they are rarely assessed; at a global Traditional assessment methods
level PISA’s innovative domain tests typically fail to measure the high-
of collaborative problem-solving and level skills, knowledge, attributes
creative thinking are exceptions, as is and characteristics of self-directed
the State of Victoria’s testing of critical and collaborative learning that are
and creative thinking. increasingly important for our global
economy and fast-changing world.
(Griffin, McGaw and Care, 2012, p v–vi).
Figure 3. A continuum of assessment methods, adapted from Darling-Hammond
(2017), p 6
Shallow/Narrow Deeper/Wider
Traditional tests Tests with Performance tasks Extended tasks Longer, deeper
pencil and paper open-ended (1 day–1 week) (1–4 weeks) investigations
often multiple- items and short requiring extended involving exhibitions
choice for routine performance problem solving formulation/ (2–3 months)
skills tasks carrying out of in multiple
inquiries and modalities
presentation of
findings
13 / CSE Leading Education Series #02 April 2021A recent High Resolves report (2020) Assessments need not be done in this
proposes the concept of ‘strings-based way, as ‘Measuring progress provides a
assessment’ (High Resolves, 2020, p 16) deliberate counterpoint to the traditional
to exemplify the kind of blend or ‘strings’ practice of measuring achievement
of immersive, repeated practices and at specific time points’ (Hipkins and
real-world applications that may be Cameron, 2018, p 22).
useful in evidencing high-order skills
in citizenship education. The range of
possible assessment methods educational Consequences that
jurisdictions might choose from is actually are unhelpful
wide (see Figure 3).
In any assessment system there are
Students are tested at set intended and unintended consequences,
times rather than when they but it would seem fundamental to assume
fundamentally, most are ready, often to meet the that an essential principle should be, as the
assessments fail to needs of the next educational USA’s Gordon Commission on assessment
capture the degree to provider or, frequently in 2013 noted, that assessment systems
which students have ineffectively, of employers. should ‘do no harm’.
These inflexible encounters
progressed over time Sadly, the consequences of the focus and
with assessment ignore
the huge variety of student methods of many, especially high-stakes
achievement levels, where ‘in any given assessments, are well-documented and
year of school, the most advanced learners harmful in a number of ways, including
in areas such as Reading and Mathematics leading students to conclude that they
can be as much as five or six years ahead are failures (Education Policy Institute,
of the least advanced learners’ (Masters, 2019);
2013, p 3), the fact that ‘attainment is only
demotivating students to the extent that
loosely related to age’ (Wiliam, 2007, p 248)
they may not stay on at school or find
and the differing levels of maturity found
employment (Milligan et al, 2020a);
in any cohort on account of birth dates.
making it less likely that students will
More fundamentally, most assessments fail
see themselves as learners and want to
to capture the degree to which students
continue learning throughout their lives
have progressed over time. Instead they
(Tuckett and Field, 2016)
… provide snapshots of achievement at
causing negative impact on young
particular points in time, but they do
people’s wellbeing (Howard, 2020);
not capture the progression of students’
conceptual understanding over time, exacerbating inequity (Au, 2016);
which is at the heart of learning. This reducing performance through anxiety,
limitation exists largely because most especially for students of lower ability
current modes of assessment lack an (von der Embse et al, 2018);
underlying theoretical framework of
how student understanding in a content increasing irrelevance to employers
domain develops. (Harvard Business Review, 2015);5
(Pellegrino, Chudowsky and Glaser, 2001, distracting from the huge importance of
p 27–28). assessment for learning and assessment
as learning (Birenbaum et al, 2015);
Rethinking assessment in education: The case for change / 14 misunderstanding and undervaluing Dubious validity for many users
wider skills and dispositions by not
measuring them (Heckman and Kautz, Assessment serves many purposes,
2013), and perpetuating the myth that including the following.
soft skills are easy to acquire and of less It certifies, selects and credentials
value than so-called hard skills such as students for universities and colleges.
core literacies;
It is a sifting mechanism for employers.
inviting a lack of trust in teacher
It gives teachers information on the
judgement in some jurisdictions
progress of their students.
(Harlen, 2005; Coe et al, 2020) which,
in an unhelpfully reinforcing loop, It gives students actionable feedback
can lead to lower levels of teacher on their progress and suggests potential
assessment ‘literacy’. next steps.
In The Testing Charade (2015), Koretz Across the world, however, there is a crisis
reminds us of the danger of Campbell’s of validity, with growing dissatisfaction
law, that from each of the main users.
the more any quantitative social Universities and colleges
indicator is used for social decision-
Universities and colleges find the grades
making, the more subject it will be to
or scores they are provided with too crude
corruption pressures and the more
to be helpful, so that many are creating
apt it will be to distort and corrupt
consortia to work with schools to provide
the social processes it is intended to
more rounded information. The Mastery
monitor (p 38) … When test scores
Transcript Consortium,6 the New York
become the goal of the teaching process,
Performance Standards Consortium7 and
they both lose their value as indicators
the Comprehensive Learner Record,8 in
of educational status and distort the
the USA, and New Metrics for Success,
educational process in undesirable
in Australia,9 are indicators of a growing
ways.
unease with the status quo.
(p 39)
The National Academy of Education (2021) Employers
points out that, to avoid unintended and Employers are frustrated that the
sometimes unfair consequences, we need to current crop of academic and vocational
qualifications leave them under-informed
Communicate clearly (and often) about potential employees (Education
the intended purposes and uses of Council, 2020; Confederation of Business
particular assessments as well as any Industry (CBI), 2019). Many employers
relevant context. are now qualification-blind in their
(p 11) recruitment. In England, Rethinking
Assessment has identified many examples
of, predominantly, larger organisations
that operate in this way, including Apple,
Bank of America, BBC, the Civil Service,
Clifford Chance, Google, The Guardian,
Hilton, Microsoft, Penguin Random House,
PricewaterhouseCoopers (PWC) and
Starbucks.
15 / CSE Leading Education Series #02 April 2021Many employers now develop their When passing tests is high stakes,
own approaches to assessing potential teachers adopt a teaching style which
employees. Often these are ‘strength- emphasises transmission teaching of
based’ aptitude tests, looking to see what knowledge, thereby favouring those
capabilities and values candidates have to students who prefer to learn in this
better enable them to work productively way and disadvantaging and lowering
with others, seeking to establish a more the self-esteem of those who prefer
balanced scorecard than mere exam grades. more active and creative learning
experiences.
As Professor Tristram Hooley,
Chief Research Officer of the (Harlen and Deakin Crick, 2002, p 4)
Teachers ... are
Institute of Student Employers
concerned variously Wherever you are in the world, the
in England, puts it, COVID-19 pandemic has provided
about the way that
a dramatic interruption of normal
tests privilege certain Most employers don’t worry if a
candidate knows a little bit less assessment activity. PISA’s 2021 tests are
subjects over others, currently rescheduled until 2022. Across
about theories of population
especially ‘academic’ the world, school examinations for 18-year-
migration or the nineteenth
over practical, and century novel. But they will olds or 19-year-olds have been cancelled,
how an emphasis on care a lot about candidates’ postponed or simplified.12 In many cases
memorisation can lead ability to learn, to think on these changes have required students to
rely on teacher-assessed grades. While
to shallower and less their feet, to be resilient in the
this can be seen as a positive development
enjoyable learning, face of knock backs, and so
(inviting innovation in methods), in
on.10
especially at upper practice it has caused additional stress
secondary level The old narrative of working
among teachers who may not yet be
hard, getting good grades
assessment literate enough to undertake
at school, going to a good
such testing without an appropriate
university and securing a well-paid job
infrastructure of moderation and training,
is increasingly fractured. Employers are
along with equitable appeals processes.
becoming aware that, ‘when it comes to
predicting job performance, aptitude tests
Students
are twice as predictive as job interviews,
Students are increasingly unsettled. In
three times as predictive as job experience,
one part of their world they have moved
and four times as predictive as education
from an era of television programs to be
level’.11
watched at set times, to unlimited on-
Teachers demand consumption of You-Tube, TikTok
and streaming services; from books which
Teachers have had rising degrees of
needed to be learned, to an Internet which
dissatisfaction with the status quo since
can be searched. Not so their examinations,
the millennium. They are concerned
which mostly require pencil and paper
variously about the way that tests privilege
completion on a set date and considerable
certain subjects over others, especially
feats of memory.
‘academic’ over practical, and how an
emphasis on memorisation can lead to When it comes to high-stakes assessment,
shallower and less enjoyable learning, there is widespread and ongoing stress
especially at upper secondary level. This among students, as this blog13 on the website
was evident in England two decades ago. of Ofqual (The Office of Qualifications
Rethinking assessment in education: The case for change / 16and Examinations Regulation) in England Can dispositions be measured?
highlights. In The Testing Charade, Daniel
Koretz quotes an alarming letter from New In the last few decades we have made
York principals to parents. real progress in understanding how best
to evidence dispositions more generally
We know that many children cried (Soland et al, 2013; Darling-Hammond,
during or after testing, and others 2017; Siarova et al, 2017; Care et al, 2018).
vomited or lost control of their bowels In some cases real progress is being made
or bladders. Others simply gave up. One in developing useful standard measures of
teacher reported that a student kept specific aspects of some key dispositions,
banging his head on the desk … for example of ‘grit’ (Duckworth and
(Koretz, 2017, p 2) Quinn, 2009).
Interestingly, it is through tests like PISA
Educational jurisdictions that we have been able to make significant
An educational jurisdiction’s performance breakthroughs in our understanding of
is also judged through international how two key dispositions/competencies,
assessments. Assessments are used as collaborative problem solving14 and
a means by which society rates, often creative thinking15, can be assessed in an
in very limited ways, the performance online test. (I have been involved in helping
of its schools. Using tests such as the to shape the second of these two tests.)
Programme for International Student
We have been assisted in this process by
Assessment (PISA), Trends in International
advances in assessment technology. For
Mathematics and Science Study (TIMSS)
example, evidence-centred design,
and Progress in International Reading
a way of creating assessments that better
Literacy Study (PIRLS), the success of
demonstrate how test-takers’ inferences
individual jurisdictions can be compared
are made and their reasoning is developed
internationally. These have a powerful
as they approach assessment tasks, is a
impact on both what is tested and how it is
promising approach.
evidenced, but that is beyond the scope of
this discussion.
17 / CSE Leading Education Series #02 April 2021Table 4. Methods of evidencing progress in creativity
(Lucas and Spencer, 2017, p 160)
Student Teacher Real-world Online
Real-time feedback Criterion-referenced Expert reviews Reliable, validated
Photos grading Gallery critique online tests
Self-report Rating of products Authentic tests Digital badges
questionnaires and processes eg displays, E-portfolios
Logs/diaries/journals Structured interviews presentations,
Performance tasks interviews,
Peer review podcasts,
Group critique Capstone projects films
Badges Exhibitions
Portfolios
In my own work in the UK and in In 2016 the journal Applied Measurement
Australia, working with schools and in Education compiled a special issue
school systems and drawing on a wider focusing on the assessment of so-called
OECD study (Vincent-Lancrin, et al, 2019) 21st century skills.17 It focused on four
with which I was involved exploring the types of dispositions: collaborative problem
assessment of creativity, I have found that solving; complex problem solving; digital
a clear understanding of what creativity is, and information literacy; and creativity,
along with an understanding of learning to which I contributed our research at the
progression, is a necessary starting point. University of Winchester, (Lucas, 2016).
Then, provided a range of different In the spirit of scientific enquiry, the issue
perspectives are acknowledged, it is focused on both what we do know and
possible to provide students and teachers what we do not yet fully understand. It
with robust evidence of progress over time offered some promising approaches, some
(see Table 4). of which are already being used by PISA.
Importantly, we need multimodal Just as these days few contest the notion
assessment to gain an accurate picture, of the learning sciences as a valid lens to
using perspectives from at least three explore teaching, so we need a similar shift
columns in Table 4. in building the science of assessment.
I’ll say more about this in the final section.
However, we have a way to go yet. As
Daniel Willingham reminded us in 2013,
in his blog,
we’re far from agreed-upon measures.
Just how big a problem is that? It
depends on what you want to do. If you
want to do science, it’s not a problem at
all. It’s the normal situation.16
Rethinking assessment in education: The case for change / 18Revisiting the purposes
of assessment
Assessments must fully represent the competencies that the increasingly complex
and changing world demands. The best assessments can accelerate the acquisition of
these competencies if they guide the actions of teachers and enable students to gauge
their progress.
(Gordon Commission, 2013, p 7)
Over the last few decades we have Twenty years ago the Committee on
progressively lost our way with the Foundations of Assessment in the
educational assessment. What we assess USA (National Research Council, 2001)
grows ever further away from what we considered the degree to which advances
want young people to be able to know, do, in the cognitive sciences were impacting
be and become in the complex world in on educational assessment. The central
which they live today. The focus of most problem it identified is that ‘most widely
systems is on summarising used assessments of academic achievement
rather than understanding, are based on highly restrictive beliefs
We seem happier using recalling rather than applying, about learning and competence not fully
noticing deficiencies rather in keeping with current knowledge about
numbers rather than
than celebrating strengths. human cognition and learning’ (p 1). The
narratives, keener on We seem happier using report explores many of the then-current
judging rather than numbers rather than narratives, kinds of assessment and exposes these
prompting improvement. keener on judging rather than to rigorous scrutiny. It concludes with a
prompting improvement. vision of assessment that has still not been
realised in education:
Previous attempts to In the future envisioned by the
committee, educational assessments
rethink assessment will be viewed as a facilitator of high
The Assessment Reform Group in the levels of student achievement. They
UK made ground-breaking progress in its will help students learn and succeed in
exploration of the value of assessment school by making as clear as possible
for learning between 1996 and 2010, to them, their teachers, and other
something which has subsequently spread education stakeholders the nature of
across the world, albeit always in tension their accomplishments and the progress
with the tendency of systems to prefer of their learning.
summative data, (Birenbaum et al, 2015). (p 292)
For a long while (Black and Wiliam, 1998)
we have known that formative assessment In 2006 a group of states in Canada worked
is effective in promoting improvements in with Lorna Earl and Steven Katz to, as they
student learning. described it, ‘rethink classroom assessment
19 / CSE Leading Education Series #02 April 2021with purpose in mind’. The document In Australia, at the same time as the
provides a framework for thinking about Gordon Commission, the Australian
the purposes of assessment. Its reminders Council for Educational Research (ACER)
about the distinctions between assessment undertook a review of educational
for, as and of learning are clear and succinct: assessment, (Masters, 2013). Geoff Masters
reminds us that ‘the fundamental purpose
Assessment for learning is designed of assessment is to establish where
to give teachers information to learners are in their learning at the time
modify and differentiate teaching and of assessment’ (p 5–6). In a thoughtful
learning activities. It acknowledges and prescient overview Masters points
that individual students learn to the folly of age-related testing, the
in idiosyncratic ways, but it also failure of assessment to enable effective
recognizes that there are predictable differentiation in teaching, the way in
patterns and pathways that many which it supports ‘traditional approaches
students follow. to schooling, including the assembly-
Assessment as learning is a process line model, whole-class teaching, age-
of developing and supporting based curricula’ (Masters, 2013, p 4), its
metacognition for students. Assessment ignoring of important life skills such as
as learning focusses on the role of collaborative working, and its pedestrian
the student as the critical connector use of technology. Not much has changed
between assessment and learning. in the eight years following the ACER review.
Assessment of learning is summative The COVID-19 pandemic, as well as
in nature and is used to confirm requiring educational jurisdictions to
what students know and can do, rethink their approach to assessments, at
to demonstrate whether they have least temporarily, has forced educators to
achieved the curriculum outcomes, think more carefully about issues of equity
and, occasionally, to show how they are as we recover from the pandemic.
placed in relation to others.
Assessments, if used properly, can
(Earl and Katz, 2006, p 13–14) help us to mitigate the impacts of the
The Gordon Commission in the COVID-19 pandemic for years to come.
USA in 2013 made a number of key If used improperly, assessments may
recommendations, about designing and waste precious instructional time and
implementing assessment that supports resources, worsen inequities, reinforce
a more ambitious and expansive vision misperceptions as to sources of inequity,
of education. It is vitally important, the and impede sound education policy.
Commission argued, that assessments (National Academy of Education, 2021, p 13).
best represent the kind of learning New Metrics for Success18 at Melbourne
students will need to thrive in the world University in Australia, the Brookings
that awaits them beyond graduation Institution in the USA19 and Rethinking
(p 8) … Assessments must advance Assessment20 in England are three
competencies that are matched to the examples of organisations trying to find
era in which we live. Contemporary saner solutions to many of the issues noted
students must be able to evaluate the in these earlier attempts to reimagine
validity and relevance of disparate assessment. Also we are starting to move
pieces of information and draw with an urgency and a focus on practical
conclusions from them. solutions.
(p 9)
Rethinking assessment in education: The case for change / 20Many schools that have been
particularly successful in reducing
opportunity and achievement gaps for
traditionally marginalized students
– producing high graduation and
college success rates – have adopted
mastery-oriented performance-based
assessments that build higher order
thinking and performance skills,
collaboration and communication
skills, motivation and engagement,
and a host of co-cognitive skills such
as self-regulation, executive function,
resilience, perseverance and growth
mindset (p 121) … Performance
assessments that encourage higher
order thinking, evaluation, reasoning,
and deep understanding are themselves
tools for learning.
(p 122)
Along with an understanding of the
The science of assessment purpose of any activity to evidence
Over the last two decades there has been a performance and progress in education,
step change in the way in which evidence these kinds of observations form the
about learning is used by schools and bedrock of the assessment sciences that
within school systems; the science of will help us take better decisions. To return
learning is a widely accepted concept in to the metaphor of fishing with which I
education. began this paper, the science of assessment
will help us to cast the right kinds of nets.
However, despite some occasional
thoughtful publications (National Research
Council, 2001), there has not been the
same culture shift among teachers and
Characteristics of high-quality
policy-makers with regard to the science assessment systems
of assessment. A Google search on ‘science In the last decade a significant number of
of learning’ produces 1,840,000,000 results, reviews (Lai and Viering, 2012; Conley
while one on ‘science of educational and Darling-Hammond, 2013; Bennett,
assessment’ yields 655,000,000, the 2013; OECD, 2013; Masters, 2013; Soland,
majority of which on closer scrutiny turn Hamilton and Stecher, 2013; Hill and
out to be about the assessment of science Barber, 2014; Siarova, Sternadel and
education. Mašidlauskaitė, 2017; Care et al, 2018;
In a recent paper exploring implications O’Connell, Milligan and Bentley, 2019;
for educational practice of the science Care, Anderson and Kim, 2019; Milligan et
of learning (Darling-Hammond et al, al, 2020b) have looked at the implications
2020) it is noteworthy that assessment for systems wanting to move towards the
does not merit a discrete section (despite assessment of deeper learning, what Elmore
the fact that the paper is written by a calls ‘Learning 2’ (See page 10).
team including an assessment expert). While analysing implications at a system
Notwithstanding this, there are clear level is complex, and needs to take
messages to be heeded. into account the differences between
21 / CSE Leading Education Series #02 April 2021the political intentions of educational Authenticity
jurisdictions, the convergence of thinking Increasing interest in strengths-based
across these reviews – combined with the approaches, especially from employers.
slowness with which their suggestions
have been taken up – reminds us of how The need to design better performance-
difficult it is to change assessment systems. based assessments.
Common themes from evidence on high- A move towards scenario-based,
quality assessment systems include the authentic assessment.
following. A move towards assessments of
investigations over longer time periods.
Purpose and consequence
The importance of understanding the Some interest in assessment on demand.
purpose any assessment is intended to Increased opportunities for student
serve. involvement and agency in the process.
A growing recognition of assessment as Progression and improvement
a tool for improvement at individual,
The benefits of assessment for and as
school and system level.
learning.
The tensions that exist between
The need for multimodal approaches
summative and formative approaches.
to assessment, incorporating data from
The many unhelpful consequences of a number of sources.
high-stakes assessment.
Quality infrastructure
Depth and breadth A better understanding of when to use
A need to evidence high-order thinking assessment of, for and as learning.
skills reliably.
The need for new assessment
A requirement for better definitions of partnerships.
dispositions and associated learning
Enhanced teacher capacity in
progressions.
assessment literacy and moderation.
The growing visibility of dispositions
The desirability of international
in the curricula of educational
benchmarking.
jurisdictions.
Three things emerged at the metalevel.
The desirability of assessments
Systems need to decide what they value
being pedagogically sensitive and
before they determine what they seek
educationally valuable.
to evidence. An increasingly scientific
The complexity of designing ways approach to the field of educational
of fairly evidencing student progress assessment is required. While the direction
within dispositions. of travel is increasingly clear, progress
A growing interest in the concept of towards it is glacially slow.
mastery.
The need for flexibility to ensure that
the full range of abilities can be fairly
assessed.
A focus on collaborative rather than just
individual performance.
Rethinking assessment in education: The case for change / 22Promising practices from
across the world
Mā te kimi ka kite, Mā te kite ka mōhio, Mā te mōhio ka mārama.
Seek and discover. Discover and know. Know and become enlightened.
(Maori saying)
There are many examples from which An overarching question here concerns just
we can learn as we rethink the purposes how much knowledge we think students
and practices of assessment. As the need to acquire at school and what kind
Education Council (2020) reminded us in of knowledge that is. Across the world,
its review of senior secondary pathways and reinforced by PISA’s focus, literacy,
into work in Australia, we need to learn numeracy and science are generally
from those who are actually trying things considered to be foundational.
out, ‘Demonstration projects need to have If the answer to this question is ‘less than
greater influence on the traditional core of we do now’, as many of us believe to be the
how we measure educational success’. case, then we will need to understand any
possible consequences for the curriculum
in schools of reducing the core focus of
Interrogating practices assessment.
At Rethinking Assessment (RA) in At a more nuanced level we might want
England, we have been exploring a number to look at the science curriculum to see
of questions to better understand the which concepts are more relevant than
nature of the problems with others given the size of the field; with
which we are grappling, as we maths we might wish to re-emphasise
An overarching question explore promising international content, prioritising, for example, statistics
here concerns just how demonstrations of what might over some aspects of trigonometry.
much knowledge we be adopted in England. We We might want to weigh the benefits
think students need to have developed two sets of of interdisciplinary knowledge, either
questions, one to do with expressed as, for example, STEM (Science,
acquire at school and
knowledge and skills (see Technology, Engineering and Maths) or
what kind of knowledge Table 5), and another relating STEAM (Science, Technology, Engineering,
that is. to dispositions and skills (see Arts and Maths), or through an
Table 6). organisation of the curriculum into projects
Skills, as I have argued earlier, are the requiring more than one discipline. Both
connective tissue between knowledge and of these approaches are increasingly part
dispositions. of university life21 but surprisingly absent
from schools.
23 / CSE Leading Education Series #02 April 2021You can also read
