Preparing for a Renaissance in Assessment - Peter Hill and Michael Barber December 2014 - Pearson

 
Preparing for a Renaissance in Assessment - Peter Hill and Michael Barber December 2014 - Pearson
Preparing for a Renaissance
in Assessment
Peter Hill and Michael Barber

December 2014
Preparing for a Renaissance in Assessment
Peter Hill and Michael Barber
ABOUT PEARSON                                       or send a letter to Creative Commons, 559
Pearson is the world’s leading learning             Nathan Abbott Way, Stanford, California
company. Our education business combines            94305, USA.
150 years of experience in publishing with
the latest learning technology and online           Sample reference: Hill, P. and M. Barber (2014)
support. We serve learners of all ages around       Preparing for a Renaissance in Assessment,
the globe, employing 45,000 people in more          London: Pearson.
than seventy countries, helping people to
learn whatever, whenever and however they           ABOUT THE AUTHORS
choose. Whether it’s designing qualifications in
the UK, supporting colleges in the US, training
school leaders in the Middle East or helping
students in China learn English, we aim to help
people make progress in their lives through
learning.
                                                    Dr Peter Hill has held senior positions in
INTRODUCTION TO THE                                 education in Australia, the USA and Hong
SERIES                                              Kong, including as Chief Executive of the Vic-
The Chief Education Advisor, Sir Michael Barber,    torian Curriculum and Assessment Board,
on behalf of Pearson, is commissioning a series     Chief General Manager of the Department
of independent, open and practical publications     of School Education in Victoria, Australia, Pro-
containing new ideas and evidence about             fessor of Leadership and Management at the
what works in education. The publications           University of Melbourne, Director of Research
contribute to the global discussion about           and Development at the National Center on
education and debate the big ‘unanswered’           Education and the Economy in Washington
questions in education by focusing on the           DC, Secretary General of the Hong Kong
following eight themes: Learning Science,           Examinations and Assessment Authority and
Knowledge and Skills, Pedagogy and Educator         Chief Executive of the Australian Curriculum,
Effectiveness, Measurement and Assessment,          Assessment and Reporting Authority.
Digital and Adaptive Learning, Institutional
Improvement, System Reform and Innovation,          He is currently a consultant advising on system
and Access for All. We hope the series will         reform in the areas of curriculum, assessment
be useful to policy-makers, educators and all       and certification. He has published numerous
those interested in learning.                       research articles and co-authored with Michael
                                                    Fullan and Carmel Crévola the award-winning
CREATIVE COMMONS                                    book, Breakthrough, published by Corwin Press.
Permission is granted under a Creative Com-
mons Attribution 3.0 Unported (CC by 3.0)
licence to replicate, copy, distribute, transmit
or adapt all content freely provided that attri-
bution is provided as illustrated in the refer-
ence below. To view a copy of this licence, visit
http://creativecommons.org/licenses/by/3.0

ii
Standards (from 1997 to 2001). Before joining
                                                   government he was a professor at the Institute
                                                   of Education at the University of London.
                                                   He is the author of several books including
                                                   Instruction to Deliver, The Learning Game:
Sir Michael Barber is a leading authority on       Arguments for an Education Revolution and How
education systems and education reform.            to Do the Impossible: A Guide for Politicians with
Over the past two decades his research             a Passion for Education.
and advisory work have focused on school
improvement, standards and performance;            Michael has recently been appointed as Chair
system-wide reform; effective implementation;      of the World Economic Global Advisory
access, success and funding in higher education;   Forum.
and access and quality in schools in developing
countries.                                         ACKNOWLEDGEMENTS
                                                   We would like to recognise the significant
Michael joined Pearson in 2011 as Chief            contribution of Simon Breakspear to the
Education      Advisor,     leading    Pearson’s   conceptualisation of this paper and Jacqueline
worldwide programme of research into               Cheng for working with us to develop the
education policy and efficacy, advising on and     paper. We would also like to put on the
supporting the development of products and         record our gratitude to Carmel Crévola,
services that build on the research findings and   Michael Fullan, Doug Kubach and many
playing a particular role in Pearson’s strategy    colleagues within the Pearson North America
for education in the poorest sectors of the        assessment community; to Maria Langworthy,
world, particularly in fast-growing developing     Tony Mackay, Geoff Masters, Roger Murphy
economies.                                         and Jim Tognolini, for the time they took to read
                                                   drafts and for their many valuable suggestions
Prior to Pearson, Michael was a Partner at         for improving the text; and Lee Sing Kong for
McKinsey & Company and Head of McKinsey’s          writing the foreword. Finally, thanks to Peter
global education practice. He co-authored          Jackson and Tanya Kreisky for their editorial
two major McKinsey education reports: How          work; to Olivia Simmons and Liz Hudson for
the World’s Most Improved School Systems Keep      managing production of the final version; and
Getting Better (2010) and How the World’s          to Splinter for the design.
Best-Performing Schools Come Out on Top
(2007). He is also Distinguished Visiting Fellow   Pearson © 2014 The contents and opinions
at the Harvard Graduate School of Education        expressed in this report are those of the
and holds an honorary doctorate from the           authors only. Figures reprinted with permission.
University of Exeter.
                                                   ISBN: 9780992422653
Michael previously served the UK government
as Head of the Prime Minister’s Delivery Unit
(from 2001 to 2005) and as Chief Adviser to
the Secretary of State for Education on School

                                                                                                   iii
CONTENTS
FOREWORD by Lee Sing Kong                                                            1
EXECUTIVE SUMMARY                                                                    3
 Setting the scene                                                                   3
 Assessment: a field in need of reform                                               3
 Transforming assessment                                                             6
 A framework for action                                                             10

1. SETTING THE SCENE                                                                11
  The educational revolution                                                        12
  Key elements of the education revolution                                          22
  When will the ‘revolution’ happen, and how?                                       24

2. ASSESSMENT: A FIELD IN NEED OF REFORM                                            25
  Assessment for certification and selection purposes                               26
  Assessment for accountability purposes                                            27
  Assessment for improving learning and teaching                                    37
  Assessment as the lagging factor                                                  40

3. TRANSFORMING ASSESSMENT                                                          41

  Transforming formal assessment programmes                                         41
  Transforming assessment as part of the ongoing process of learning and teaching   50
  Rethinking, aligning and rebalancing assessment                                   57

4. A FRAMEWORK FOR ACTION                                                           64
  1. Think long-term                                                                65
  2. Build partnerships                                                             65
  3. Create the infrastructure                                                      66
  4. Develop teacher capacity                                                       66
  5. Allow variation in implementation                                              66
  6. Adopt a delivery approach                                                      66
  7. Communicate consistently                                                       67
  8. Apply the change knowledge                                                     67
  Drawing together the threads                                                      69

REFERENCES                                                                          72

                                                                                         v
FOREWORD

Assessment is a very complex topic. As            systems, a fundamental issue that must be first
this essay articulates, it is meant to monitor    clearly articulated is ‘What is the purpose of
or to measure what students have learnt.          education in this new world that we live and
For validity and reliability, and to minimise     work in?’ Only when we can articulate with
subjectivity, standardised tests are often        clarity the purpose of education in terms of
adopted and marks are awarded, followed by        the learning outcomes that the education
a process in which test scores are converted      process aims to achieve can we then articulate
into grades. The grades are then recognised       what an assessment renaissance implies so
as measures of students’ learning attainment.     that the ‘what’ and ‘how’ of assessment can be
But what assessment actually means is seldom      crystallised.
articulated. Is it a measure of the body of
knowledge that a student has acquired, or is it   For an assessment renaissance to be
also a measure of other attributes?               meaningful, it also needs a total cultural shift
                                                  within society to accept the different ‘what’ and
Institutes of higher education have often found   ‘how’ of assessment. The current mindset of
such assessment grades to be so lacking in        assessment is all about test scores, irrespective
substance for admission purposes that many of     of whether the meaning of the test scores is
these institutes have introduced other modes      well clarified. In realising the outcomes of the
of assessment so as to gauge the other desired    assessment renaissance, there may not always
attributes of their candidates. The complexity    be a test score to contend with. It may just be a
of assessment is further compounded by the        series of qualitative descriptions of the extent
way in which test scores are utilised. Apart      to which a student may have demonstrated
from being considered for entry into further      various attributes that cannot be quantified.
education, they are also used for the purpose     Can society accept such assessment outcomes?
of accountability of schools or the system, as
well as the performance of teachers.              Going forward, assessment will remain
                                                  a complex issue, no matter what form
In the twentieth century, the standardised test   the assessment renaissance may take. It
approach could be valid and reliable, though      is here that the importance of research
never perfect. However, in the twenty-first-      and development into assessment issues
century landscape, where the demands go           cannot be overemphasised. If the ‘what’ and
beyond just knowledge and technical skills,       the ‘how’ can be conducted with clarity of
there is, indeed, a need for an ‘assessment       meaning, and considered valid and reliable
renaissance’ so that the desired attributes       with minimal subjectivity, and if society at
can be meaningfully monitored or measured.        large can be educated about the need for
However, in this new world, where there are       such a renaissance, then there will be light at
so many ‘drivers’ that are impacting education    the end of the tunnel. I believe this will take

                                                                                                 1
FOREWORD

time, but the journey must start immediately. I
congratulate the authors for writing this think
piece, which sets out so clearly where we have
come from and where we need to go.

Professor Lee Sing Kong
Vice President for Education Strategies,
Nanyang Technological University Director,
National Institute of Education, Nanyang
Technological University (2006–14)

2
EXECUTIVE SUMMARY

SETTING THE SCENE                                   and systems that are already operating in or
                                                    contemplating moving towards some of the
Three core processes lie at the heart of            directions indicated, but this is inevitably a
schooling:                                          sporadic process.

 1 c urriculum (deciding what students             ASSESSMENT: A FIELD IN NEED OF
    should learn);                                  REFORM
 2 learning and teaching; and
 3 a ssessment (monitoring student learn­          The primary purpose of educational assess­
    ing).                                           ment is to seek to determine what students
                                                    know, understand and can do. While that
When well executed, they work together in           would seem a relatively straightforward
a symbiotic fashion, and all other activities       intention, in the real world of policy and
function in support of this triad. Of the three,    practice, educational assessment is complex
this essay focuses primarily on assessment,         and frequently controversial.
but we are aware that it is not possible to
talk about changes in the field of assessment       This essay reviews the key purposes of
without relating them to a much wider set of        assessment, namely its use in formal assess­-­
changes taking place in education.                  ment programmes for the purposes of
                                                    certification, selection and accountability; and
The educational revolution                          its formative use in classrooms and schools
We believe that two game-changers are at            for improving learning and teaching. We have
work that will shake the very foundations of        also sought to illustrate why assessment,
the current paradigm of school education. The       when used for these purposes, is so often
first is the push of globalisation and new digit­   controversial, difficult and a barrier to change.
al technologies, which are sweeping all before      The key challenges we have highlighted are
them. The second is the pull inherent in the        summarised in Table ES.2, which contrasts
realisation that the current paradigm is not        what we ideally want from formal assessment
working as well it should any more. Even the        programmes with what we typically get.
top-performing systems in the world have hit
a performance ceiling.

Key elements of the education revolution
Table ES.1 summarises what we see as six
key changes that characterise this revolution.
The seeds of each of these key changes are
everywhere to be seen. There are schools

                                                                                                  3
Table ES.1 Key features of the education revolution.

                   Overthrown and
Key element                                         Replaced by
                   repudiated

 1. C
     apacity to   Practices reflecting an          • Practices that build on prior learning and
    learn          assumption that students           reflect a belief in the potential for all students
                   commence school tabula             to learn and achieve high standards, given high
                   rasa and with an innate and        expectations, motivation and sufficient time and
                   fixed capacity to learn and        support
                   profit from formal education

 2. The           Curricula that emphasise         • A greater emphasis on deep learning of big
    curriculum     memorisation of unrelated          ideas and organising principles
                   facts and breadth at the         • More explicit and systematic attention to cross-
                   expense of depth                    curricular skills, capabilities, understandings and
                                                       dispositions that support lifelong learning and
                                                       living in the Knowledge Society of the twenty-
                                                       first century

 3. E ducation    The school as the focus of       • The student as the focus of educational policy
    policy         educational policy                 and concerted attention to personalising
                                                      learning

 4. O
     pportunity   Current age and time-bound       • Students able to progress at different rates and
    to learn       parameters:                        with time and support varied to meet individual
                   • age–grade progression            needs
                   • 9.00–4.00 school hours         • Significantly increased access to care and
                   • open 200/365 days a year        education to better align with the realities of
                                                      modern living and working
                                                    • Greater use of the home, the community and
                                                      other settings as contexts for 24/7 learning

 5. Teaching       Predominantly teacher/text       • Increasing reliance on sophisticated tutor/
                   instruction, with schools and      online instruction with greater differentiation
                   classrooms as the physical         in educator roles and the creation of learning
                   and organisational places for      partnerships between and among students,
                   all formal learning and with       teachers and families, with the teacher as the
                   the classroom teacher as the       ‘activator’
                   imparter of knowledge

 6. Teacher       Teaching as largely under-       • Teaching as a true profession with a distinctive
    quality        qualified and trained, heavily     knowledge base, a framework for teaching with
                   unionised, bureaucratically        well-defined common terms for describing
                   controlled ‘semi-profession’       and analysing teaching and strict control by the
                   lacking a framework and            profession itself on entry into the profession
                   a common language to
                   describe and analyse teaching
Table ES.2 Assessment: a field in need of reform.

The ideal                       The norm

 Assessments that can           • Assessments unable to assess accurately at either end of the ability
 accommodate the full range       distribution, or away from critical cut-scores
 of student abilities           • Assessments within tiered credentials or tiered assessments, with
                                  resulting problems of cost, logistics, cross-tier comparability and
                                  capping of student aspirations

 Assessments that provide       • Over-reliance on grades or levels that reveal little about what the
 meaningful information on         student can do
 learning outcomes              • Feedback to schools on student performance typically provided
                                   too late and too broad-brush to be of value in improving learning
                                   and teaching
                                • Assessments used to generate a single score for each student
                                  which is then further summarised at the school or system level as
                                  a percentage meeting a nominated cut-score – a volatile statistic,
                                  hiding more than it reveals about performance, particularly shifts
                                  in performance on either side of the cut-score. Alternatively,
                                  summarised as a mean score unadjusted for intake and other
                                  characteristics beyond the control of the teacher or school

 Assessments that               • Tests and examinations dominated by questions assessing low-level
 accommodate the full range       cognitive processes and failing to capture such valued outcomes as
 of valued outcomes               practical, laboratory and field work, speaking and listening, higher-
                                  order cognitive processes and a range of inter- and intra-personal
                                  competences (so-called ‘twenty-first century skills’)

 Assessments that support       • Assessment policies that pay little or no attention to formative
 students and teachers in         assessment and to providing teachers with the tools and the
 making use of ongoing            capacity to use it on a daily basis
 feedback to personalise        • An absence of validated learning progressions, efficient processes
 instruction and improve          for collecting and analysing data and easy-to-use assessment tools
 learning and teaching

 Assessments that have          • Assessments that carry undue weight in high-stakes decision-
 integrity and that are used      making, increasing the risks of cheating and ‘gaming’ the system
 in ways that motivate
 improvement efforts and
 minimise opportunities for
 cheating and ‘gaming’ the
 system
PREPARING FOR A RENAISSANCE IN ASSESSMENT

TRANSFORMING ASSESSMENT                            classrooms), as part of the ongoing process of
                                                   learning and teaching.
This chapter describes ways in which new
thinking and new digital technologies are          Developers of next-generation learning
transforming assessment and overcoming             systems don’t start with preconceived notions
current barriers and limitations. We begin by      of any of these components but completely
considering how these changes affect formal        rethink the whole delivery process and how
assessment programmes, such as those used          to best assist teachers to connect all of the
for certification/selection and accountability     elements so that they operate seamlessly. We
purposes, and then move to consider                can follow the logic of these systems with the
assessment as part of the ongoing process of       aid of the diagram in Figure ES.1.
learning and teaching. Finally, we indicate how
a better balance between various purposes          Curriculum
of assessment and a closer alignment of            Starting at the top of Figure ES.1 is the
assessment with curriculum and teaching can        curriculum, but one looking quite different to
be achieved as a result of the radical changes     curriculum documents of the past, consisting
in thinking and practice made possible by          of online interactive multidimensional maps at
these developments.                                several different scales that can be interrogated
                                                   in different ways, depending on one’s focus or
Transforming formal assessment                     query.
programmes
Increasingly, formal assessment programmes         Assessment
serving certification, selection and account­      Going clockwise around the diagram, the
ability purposes are being administered online,    next element is assessment. Yes, personalised
not only as part of a broad trend within           learning systems move straight from the
modern society but also, more particularly,        curriculum (deciding what students need to
because the online assessment environment          learn) to assessment, because effective learning
offers a number of major advantages once           and teaching require that one begin with the
the technical problems of access have been         students and their individual starting points.
addressed. These include:
                                                   Resources
    • a ssessing the full range of abilities;     In generating instructional sequences, learning
    • providing meaningful information on         tasks and associated assessment activities,
      learning outcomes;                           next-generation learning systems will embed
    • assessing the full range of valued          or search out the resources that most closely
      outcomes;                                    match students’ learning needs, accessing both
    • maintaining the integrity of assessments.   purpose-built, commercially available materials
                                                   and the rapidly expanding collections of public-
Transforming assessment, as part of the            domain and creative-commons resources.
ongoing process of learning and teaching
We then consider assessment undertaken at          Data management and analysis
the point of learning, at the teacher–student      It was not so long ago that almost all
interface typically (although not necessarily in   information about students and their learning

6
EXECUTIVE SUMMARY

Figure ES.1 Next-generation learning system.

                                            Curriculum

              Personalised
                                                                        Assessment
               instruction

                                               Next -
                                             generation
                                              learning

              Professional
                                                                        Resources
                learning

                                         Data management
                                           and analysis

was contained within teachers’ books of             and detailed feedback into the learning and
marks, attendance registers, student record         teaching process.
cards and student reports. Next-generation
learning systems will create an explosion in        Professional learning
data because they track learning and teaching       In next-generation learning systems, the
at the individual student and lesson level every    teacher retains the key role in fostering the
day in order to personalise and thus optimise       learning for each student, but the job itself
learning. Moreover, they will incorporate al­       changes. Learning systems of the future will
gorithms that interrogate assessment data           free up teacher time currently spent on
on an ongoing basis and provide instant             preparation, marking and record-keeping

                                                                                               7
Table ES.3 Transforming assessment.

The ideal                         How new thinking and technologies can help

 Assessments that can             • Use of adaptive testing to generate more accurate estimates
 accommodate the full range         of student abilities across the full range of achievement while
 of student abilities               reducing testing time

 Assessments that provide         • Online environments to facilitate:
 meaningful information on           – the administration of multiple versions of the same test in order
 learning outcomes                     to obtain information on performance across a much wider
                                       range of the curriculum
                                     – the collection and analysis in real time of a wide range of
                                       information on multiple aspects of behaviour and proficiency
                                       and
                                     – more immediate, detailed and meaningful reporting to specific
                                       stakeholder groups, such as via smartphone/tablet devices and
                                       through the creation of e-portfolios
                                  • Advances in the application of data analytics and the adoption
                                    of new metrics to generate deeper insights into and richer
                                    information on learning and teaching

 Assessments that                 • Automated marking to overcome obstacles to the more
 accommodate the full range         widespread use of essay and other open-response format
 of valued outcomes                 questions
                                  • Platforms to support the delivery of a new generation of
                                     assessments specifically designed to assess deep learning and a
                                     range of inter- and intra-personal competences and character
                                     traits

 Assessments that have            • The adoption of (1) more cumulative approaches to approaches to
 integrity and are used in ways     assessment for selection purposes, with opportunities to re-sit; and
 that motivate improvement          (2) intelligent accountability systems that utilise multiple indicators
 efforts and that minimise          of performance, that are designed to incentivise improvement and
 opportunities for cheating         that avoid the creation of win–lose consequences for stakeholders
 and ‘gaming’ the system            for outcomes not fully under their control

 Assessments that support         • Sophisticated online intelligent learning systems to integrate the
 students and teachers in           key components involved in effective instruction and to support
 making use of ongoing              a new generation of empowered teachers in reliably assessing
 feedback to personalise            a much wider range of outcomes, using instant and powerful
 instruction and improve            feedback on learning and teaching to deliver truly personalised
 learning and teaching              instruction
EXECUTIVE SUMMARY

and allow a greater focus on the professional          1 the teacher–student interface (tradi-
roles of diagnosis, personalised instruction,             tionally the classroom);
scaffolding deep learning, motivation, guidance        2 the school; and
and care. This is the combination of activities        3 the system.
that John Hattie describes as ‘teacher as
activator’ (2009: 17).                                The most important level is the teacher–
                                                      student interface, because this is where
Personalised instruction                              learning takes place and where there is the
With all the above in place, it is then possible to   greatest need for assessment data to enable
talk confidently about personalised instruction,      a truly personalised approach to learning and
which is the final and most crucial component         teaching. We would argue that the other levels
of Figure ES.1. By personalised instruction,          and purposes of assessment should be built
we mean instruction that is adjusted on a             on the assessment carried out at this level.
daily basis to the readiness of each student
and that adapts to each student’s specific            The challenge for awarding bodies
learning needs, interests and aspirations. The        In considering the future of assessment for
fundamental premises of personalised learning         certification purposes, the challenge facing
have been a part of the writings of educators         awarding bodies is to work out how they can
for decades but have, in recent years, become         take greater advantage of new technologies to
a realisable dream, thanks to the advent of           deliver examinations online and, by so doing,
new digital technologies.                             enhance their capacity to:

Rethinking, aligning and rebalancing                   • a ssess a wider range of valued outcomes;
assessment                                             • create more authentic assessment tasks;
In short, new thinking and digital technologies        • more accurately assess the full range
are transforming assessment and overcoming               of student abilities and speed up the
many current barriers and limitations. Table             process of marking student responses,
ES.3 summarises what we see as the main                  including those to extended response
features of this transformation.                         questions;
                                                       • open up the window of time in which
An integrated, multi-level view of assessment            examinations may be taken and work
Perhaps the most urgent need right now in                towards the longer-term goal of
the field of assessment is an overall conceptual         examinations on demand;
framework and longer-term vision for its place         • use the potential of online assessment
and purpose within the triad of processes that           and developments in psychometric
lie at the heart of schooling.                           methods to more rigorously maintain
                                                         standards and constantly benchmark
Rather than focusing on discrete assessment              them to ensure that these standards are
programmes, we would suggest that it is more             world-class.
productive to view assessment as serving
distinct data needs at three levels:

                                                                                                      9
PREPARING FOR A RENAISSANCE IN ASSESSMENT

The accountability challenges                        1    hink long-term.
                                                         T
Designing an effective accountability system         2      B
                                                             uild partnerships.
involves clarifying who can and should be held       3      C
                                                             reate the infrastructure.
to account for what at each level of the system      4      D
                                                             evelop teacher capacity.
and establishing accountability arrangements         5   Allow variation in implementation.
that are reasonable and effective and that           6    Adopt a delivery approach.
promote a shared trust in the system. This           7      C
                                                             ommunicate consistently.
means being sure that, as far as possible,           8     Apply the change knowledge.
accountabilities are within the power of the
person or organisation being held to account.       In conclusion, we see the changes in thinking
                                                    about assessment as leading to a veritable
In the school educational context, this typically   renaissance – a revival in thinking and practice
means holding systems, schools and teachers         that promises to overcome many of the key
responsible for:                                    limitations of the current paradigm and to put
                                                    assessment more fully in the service of both
 • student growth or progress rather               the curriculum and learning and teaching.
   than purely for absolute levels of               Governments, systems, schools and those
   performance; and                                 within them all have critical roles to play in
 • doing those things that the evidence            bringing this about.
   shows lead to improved outcomes – not
   just for achievement of the outcomes
   themselves, which may be only partly
   attributable to the specific person or
   organisation being held to account.

Equally important in the design of accountability
systems is the need to take into account
capacity-building requirements, particularly
those related to teachers’ assessment literacy.

A FRAMEWORK FOR ACTION

In this chapter, we propose how policy-makers,
schools and school-system leaders and other
key players can prepare for an assessment
renaissance, ensuring that they maximise the
benefits of new developments and changes
in thinking while avoiding the potential
downsides.We present a framework for action
that allows change to be implemented in ways
and in timeframes suited to the starting points,
capacity and readiness of schools and systems.

10
1. SETTING
   THE SCENE
Three core processes lie at the heart of                       order thinking and interpersonal skills vital for
schooling:                                                     living and learning in the twenty-first century.

     1 c urriculum (deciding what students                    In Preparing for a Renaissance in Assessment, we
        should learn);                                         seek to:
     2 learning and teaching;
     3 a ssessment (monitoring student learn­                    • summarise the reasons for and the
        ing).                                                       nature of these changes;
                                                                  • indicate how governments, schools and
When well executed, the three work together                         school-system leaders and other key
symbiotically, and all other activities function to                 players can prepare for these changes
support this triad. This essay focuses primarily                    and ensure they maximise the benefits
on the third process: assessment. It is often the                   and avoid potential downsides; and
piece that sits uncomfortably with the other                      • provide a framework for action to enable
two, and it is the one we believe is currently                      change, which can be implemented
lagging behind in efforts to secure improved                        in ways and timeframes suited to the
learning outcomes for all.                                          starting points, capacity and readiness
                                                                    of schools and systems.
There is now a growing consensus among
leaders in the field that we are on the verge                  We have sought to avoid going into technical
of a radical change in thinking and practice                   arguments and details but instead to provide
regarding assessment in school education.1                     a widely accessible and readable overview
However, the exact form of this change de­                     of the more significant changes without
pends very much on how we anticipate,                          oversimplifying the underlying complexities.
envision, plan for and shape it.
                                                               The field of assessment in school education
If this change is managed skilfully, we believe                is vast, so we have necessarily been selective.
that education will witness an assessment                      Thus, we have opted to review developments
‘renaissance’ – a ‘rebirth’ of the core purposes               affecting K-12, but with an emphasis on the
of assessment – that will lead to a much                       assessment of fifteen- to eighteen-year-olds.
better alignment of all three processes. More                  We consider a number of uses of assessment
specifically, we see assessment changing in                    but emphasise high-stakes uses for the pur­
ways that will help secure a floor of high                     poses of certification, selection, accountability
standards for all, removing current achievement                and improving learning and teaching.
ceilings and supporting a focus on those higher-

 	See, for example, Gordon Commission on the Future of Assessment in Education (2013) and Global Education Leaders
1.

   Program (2014).

                                                                                                                      11
PREPARING FOR A RENAISSANCE IN ASSESSMENT

As we started to write this essay, we realised    they have long been, consisting of classrooms,
that we could not discuss changes in the          halls, libraries, staffrooms and school grounds
field of assessment without relating them to      for recreation and sport. Instruction continues
a much wider set of revolutionary changes         to be delivered by a teacher, who teaches
taking place in education. So, in order to        a class of students of the same age, all
understand the ‘whys’ and ‘hows’ of the coming    progressing through a standard curriculum at
renaissance in assessment, we will begin with     the same pace, with new teachers each year.
a brief overview of the more fundamental          Despite considerable experimentation with
changes happening more broadly in education,      new arrangements and new technology, rows
of which assessment is but one vital part.        of tables and chairs and students working
                                                  with paper, pen and printed texts continue to
THE EDUCATIONAL REVOLUTION                        predominate. The school year and the school
                                                  day reflect the demands of an agrarian society
Change is a constant in the modern world, and     that has long since disappeared, with teachers
we certainly witness it in education (although    and students enjoying long holidays and short
when the dust settles we often remark on          hours that are out of alignment with the
how the fundamentals seem to stay the             working days and hours of their parents and
same). In many areas of educational policy        guardians, who face challenges in organising
and practice, we simply see pendulum swings.      child care. In brief, school education has been
Every now and then, however, radical change       characterised by constant surface-level change
occurs that completely upsets the old ways of     and periodic calls for a thorough overhaul, but
doing things. Such change is revolutionary in     the fundamentals have remained surprisingly
character since it overthrows and repudiates      constant.
established methods and replaces them with
an entirely new order.                            So, not for the first time, we need to
                                                  take stock and ask the question, ‘Are we
One hesitates to use the term ‘revolution’        currently witnessing changes that have more
when talking about fundamental changes in         fundamental and far-reaching consequences
education: after all, no parent welcomes the      and that will lead to a reconceptualisation
notion of their children being caught up in       of school education?’ We have concluded,
anything revolutionary. Furthermore, schools      as have many other commentators, that this
have been among the most stable institutions      time things are different. In particular, we
of society and are not prone to radical change.   believe that two game-changers are at work
                                                  that will shake the very foundations of the
Looking back, we can see that formal ed­          current paradigm of school education. The
ucation’s basic structures and modes of           first is the push of globalisation and new digital
delivery have barely changed over the past        technologies, which are sweeping all before
140 years.That is something one cannot say of     them. As Hannon and colleagues observe,
health care, public transport or policing.        this is an argument that has been ‘exhaustively
                                                  rehearsed, but is no less valid for that’ (2011:
Despite many recent innovations, schools          2). The second is the pull inherent in the
continue to provide the same kinds of             realisation that the current paradigm is no
functions and are recognisably similar to what    longer working as well as it should.

12
SETTING THE SCENE

Globalisation: the key driver of                      Knowledge Society has enormous implications
revolutionary change                                  for the work of schools, for how education is
The key force for change in the modern world          provided and, indeed, for the very existence of
is, and will continue to be, globalisation in all     schools as we currently know them.
its manifestations (economic, environmental,
political, cultural, social and technological). The   Let’s consider the purposes of education. In
big driver for all these changes is technology.       the past, it was possible to talk with some
Digital technologies, in particular, represent        certainty about the kind of education needed
the next, rapidly accelerating phase of human         to prepare young people for life and work, and
evolution. Those of us who operate daily in           with some confidence about the pathways it
the world of Web 2.0 can already envisage             would open up to various careers. In the new
the magnitude of changes that schools must            world, there is much less certainty about the
undergo – and which are already under way in          sorts of jobs that may be needed in the future
many places. But we can barely conceive of what       or the kinds of challenges daily living might
life might be like in the predicted scenarios of      involve.
Web 3.0 and beyond, where unlimited access
to the web will have become a right and an            Whole categories of jobs, which until recently
affordable necessity, artificial intelligence will    employed large numbers of people, are
have surpassed individual human intelligence          disappearing. At airports, staffed check-in
in many areas, and the internet may indeed            counters are being replaced by self-serve kiosks;
have become conscious.2                               the same thing is happening at supermarkets,
                                                      where self-service tills are replacing checkout
Digital technologies and the internet are             staff. Bank tellers and retail sales staff are
transforming almost all aspects of life and           being replaced by internet banking and online
creating what has been called the ‘Knowledge          shopping. Anything that can be automated is
Society’. This is characterised by                    being automated. While particularly true of
                                                      many low-paid, unskilled jobs, this also applies
     • universal and instant access to know­         increasingly to ‘white-collar’ occupations and
       ledge;                                         the professions. At the same time, new jobs
     • rapid obsolescence of knowledge               are being created, but companies are struggling
       and the disappearance of generally             to recruit people with the relevant skills. For
       longer-term jobs dependent upon old            example, as illustrated in Figure 1.1, recent
       knowledge;                                     evidence from Eurostat indicates a widening
     • exponential increase in new knowledge         skills gap in digital jobs in the European
       and the creation of generally shorter-         Union, with demand far outpacing both the
       term new jobs dependent upon new               actual (current) and projected supply of
       knowledge; and                                 graduates with relevant mathematical, science
     • the imperative for ongoing learning to        and engineering backgrounds (European
       update and connect knowledge.                  Commission 2013: 85).

The new world order brought about by                  How should we prepare young people for
globalisation and the emergence of the                such a world? There are those who argue

 	See, for example, Heylighten (2012).
2.

                                                                                                    13
PREPARING FOR A RENAISSANCE IN ASSESSMENT

Figure 1.1 Actual and projected development in digital jobs in the EU: vacancy and graduate
           numbers.

       1,000,000
        900,000
        800,000
        700,000
        600,000
        500,000
        400,000
        300,000
        200,000
        100,000
               0
                        2011            2012              2013           2014            2015

                        Vacancies in the digital sector            New ICT graduates

that knowledge of the fundamentals of the            We agree with these points and don’t believe
disciplines that have long formed the core of        they are in conflict.
traditional school subjects remains vital. At the
same time, there are those who call for:             Discussing knowledge of the core disciplines,
                                                     Daniel Willingham has observed (2006: 1):
 • l
    ess emphasis on memorisation of
   unrelated facts and a greater emphasis on               research literature from cognitive science
   deep learning of big ideas and organising               shows that knowledge does much
   principles (the least obsolescent aspects               more than just help students hone their
   of knowledge);                                          thinking skills: it actually makes learning
 • more explicit and systematic attention to              easier. Knowledge is not only cumulative,
   a set of skills, capabilities, understandings           it grows exponentially. Those with a rich
   and dispositions that run right across the              base of factual knowledge find it easier
   traditional subject-based curricula and                 to learn more – the rich get richer. In
   that facilitate response to change and                  addition, factual knowledge enhances
   the rapid acquisition of new knowledge;                 cognitive processes like problem solving
 • a greater emphasis on ‘doing’ in addition              and reasoning. The richer the knowledge
   to the acquisition of knowledge and on                  base, the more smoothly and effectively
   allowing living, learning and action to                 these cognitive processes – the very ones
   come together in our conceptions of                     that teachers target – operate. So, the
   the educated person.                                    more knowledge students accumulate,
                                                           the smarter they become.

14
SETTING THE SCENE

In other words, what we are really asking for                        report of the Committee on Defining Deeper
is more. Yes, we need to be careful to avoid                         Learning and 21st Century Skills represents
an overloaded curriculum.Yes, we must ensure                         a significant step towards clarifying the
there is space for deeper learning of the                            fundamental definition and research-related
more important content, which does imply                             questions (see Pellegrino et al. 2012).
acquiring a rich base of factual knowledge and,
beyond that, the ability to understand and                           In addition, progress has been made on scoping
apply it. But yes, we also want to ensure, in a                      and sequencing these skills or competencies
more systematic, conscious and explicit way,                         within the context of the overall curriculum.
that, as students learn in specific areas of the                     For example, the online Australian Curriculum
curriculum, they are also acquiring key cross-                       for K-10 students gives prominence to seven
curricular skills, capabilities and dispositions                     general ‘capabilities’:
through direct engagement with a curriculum
that blends living, learning and action. A                             1 literacy;
number of systems have undertaken major                                2 n umeracy;
revisions of curricula to address the need to                          3 information and communication tech-
reduce content coverage in order to promote                               nology capability;
deeper learning, with Singapore one of the                             4 c ritical and creative thinking;
first to take decisive action (Ng 2008).                               5 p ersonal and social capability;
                                                                       6 e thical understanding;
Embedding so-called ‘twenty-first-century                              7 intercultural understanding.3
skills’ or ‘next-generation learning’ into the
curriculum has proved much more challenging.                         Each has been scoped in terms of the key
These learning outcomes are increasingly seen                        outcomes relevant to each capability and
as critical to equip young people with the                           sequenced into six levels spanning years
skills required to be ongoing learners who can                       K-10. Examples are given, with hyperlinks to
navigate an ever-changing world of work and                          specific content areas within mainstream
find fulfilment in their lives. Learning outcomes                    curriculum subjects where these capabilities
include the well-understood basics of literacy                       are particularly relevant and can be developed.
and numeracy but also involve an education
characterised by deep learning and the ability                       However, the task is not one of simply adding
to think, learn, inquire, problem-solve, create,                     a new set of ‘skills’ to the curriculum but of
relate and also to manage oneself and one’s                          continually challenging our concepts of what it
learning.                                                            means to be an educated person. Here, again,
                                                                     it is a matter of more, not less. In addition
Discussion of these higher-order thinking, inter-                    to knowledge of the disciplines and cross-
and intra-personal skills has often taken place                      curricular skills and understandings, schools
without any real agreement on meanings and                           are being expected to provide young people
definitions, and with little research evidence of                    with an appreciation of, and engagement with,
their importance or even whether they can                            the big challenges of the modern world, such as
be taught successfully. The publication of the                       sustainability, peace and conflict, the widening

 	See http://www.australiancurriculum.edu.au/generalcapabilities/overview/general-capabilities-in-the-australian-curriculum
3.

   (accessed 18 November 2014).

                                                                                                                               15
PREPARING FOR A RENAISSANCE IN ASSESSMENT

gap between rich and poor, population                              long been recognised as important, they have
and resources.4 In other words, schools are                        often fallen outside the scope of what has
expected to prepare young people to be                             been mandated, made explicit, assessed or
informed and actively engaged citizens.5                           certificated. As a consequence, it has been all
                                                                   too easy for them to remain at the level of
One example of where this has been taken                           rhetoric rather than at that of deliberate policy.

                                                                   ‘
seriously is Hong Kong’s new credential for
students at the end of Year 12, the Diploma                             New models of learning and teaching
of Secondary Education, which requires
all students to study, in addition to Chinese
                                                                        are evolving that make traditional
                                                                    classroom, teacher and textbook modes
                                                                                                             ‘
language, English language, mathematics and                         of formal learning obsolete
between two and four other subjects of their
choosing, a subject called ‘Liberal Studies’. The                  Globalisation and the new technologies have
aim is to ensure that all students develop an                      fundamental implications, not only for what
understanding of the major issues confronting                      students need to know and be able to do but
society in the twenty-first century and are                        also for how it will be taught. Thanks to high-
equipped with the critical thinking skills they                    speed internet access, the low cost of devices
need to make informed, critical judgements                         such as smartphones and tablet personal
about these issues.                                                computers, social media and the evolution
                                                                   of the semantic web, users can find, share
Beyond skills or competencies and new                              and combine information more easily. New
understandings, there are calls for schools                        models of learning and teaching are evolving
to pay more attention to developing the                            that make traditional classroom, teacher and
character traits and dispositions in young                         textbook modes of formal learning obsolete.
people that will support them in confronting
the unprecedented changes taking place in                          Some form of ‘blended learning’, in which a
the world around them, such as resilience,                         part of what students learn is through online
adaptability, entrepreneurialism, sensitivity                      delivery of content and instruction with
to cultural and personal differences and the                       elements of personalisation for when, where
disposition to think and act ethically. Cultivating                and at what pace, is increasingly becoming
such outcomes is quite a different matter to                       the norm, although the form it takes varies
imparting skills and understandings, because it                    enormously, as does the quality.
means engaging students in situations where
these qualities matter and can be experienced,                     But deeper, technology-enabled transform­
reflected upon and nurtured.                                       ations are on the horizon. Big publishing
                                                                   and information technology companies, in
Whatever name we give to the disparate                             conjunction with universities and foundations,
set of learning outcomes that constitute                           are embarking on the design of new, fully
next-generation learning, it is clear that they                    integrated online learning systems that use
are central to education in the twenty-first                       detailed learning progressions and continuous
century. While many of these outcomes have                         monitoring of progress and responses to

4.
  	A comprehensive framework for considering fifteen global challenges of the early twenty-first century has been developed by
    the Millennium Project. See http://www.millennium-project.org/millennium/challeng.html (accessed 15 November 2014).
5.
  	Regarding the importance of education for citizenship, see in particular, Feith (2011).

16
SETTING THE SCENE

deliver finely calibrated instruction that reflects                 been reached in the delivery of learning
students’ learning styles, needs and aspirations.                   outcomes and in closing achievement gaps.
A key motivation behind the development of                          Investment in school education is no longer
these more ‘personalised’ learning systems is                       yielding the returns it once did, when the focus
the expectation that they will make learning                        was on access rather than outcomes.
more engaging and more efficient. It is hoped,
too, that they will accelerate progress for                         In the USA, which has extensive longitudinal
students who have fallen behind. They have                          data on performance, NAEP (National
significant implications for the role of teachers,                  Assessment of Educational Progress) survey
especially their knowledge and skillset.                            results indicate that overall performance has
                                                                    improved very little since the 1970s.6
Glimpses into the future can be had now
in pioneering schools across the world.                             But the USA is not alone. Figure 1.2 shows
Significantly, the new digital technologies are                     annualised changes in performance in reading
not just an option for advanced economies,                          and mathematics across PISA (Programme
they also offer affordable options for countries                    for International Student Assessment) assess­
in the developing world, particularly through                       ments for the top nine countries between the
the use of mobile phones (m-learning) to                            first survey results (either 2000 or 2003) and
reach places where there are no schools,                            the most recent 2012 survey. (The error bars
teachers or libraries.                                              are 95-per-cent confidence intervals around
                                                                    each change score.) In the case of reading,
In summary, the increasing availability of                          only two of the top nine performing countries
powerful and transformative interactive digital                     in the first survey ( Japan and Korea) recorded
technologies is redefining how learning takes                       a statistically significant improvement, and
place in schools and all other settings.They are                    in the case of mathematics, none did. This
key ingredients of the education revolution.                        was despite significant efforts and additional
                                                                    resources directed at improving outcomes in
The performance ceiling                                             each of these countries.
Digital technologies and the new Knowledge
Society that they are creating, of themselves,                      In addition, some of the high-performing
would probably be sufficient to fuel the                            countries (notably Australia, New Zealand
education revolution, but, as we indicated                          and Finland) have experienced a statistically
earlier, there is another game-changer at work,                     significant decline in performance levels rather
namely the ‘pull’ factor inherent in the growing                    than an improvement. In short, patterns
realisation that the current paradigm of school                     of results from longitudinal surveys of
education is no longer working as it should.                        achievement such as NAEP and PISA would
                                                                    suggest that there are limits as to how much
For many advanced nations, there are clear                          more productivity can be squeezed out of
indications from longitudinal surveys of                            school systems operating within the current
achievement that a performance ceiling has                          paradigm.7

 	For a commentary on this phenomenon, see Tucker (2013b).
6.

 	It should be noted, however, that there are those who argue that tests such as PISA, which seek to provide a common
7.

   yardstick across nations, are not sensitive to improvements in teaching and learning. PISA does not assess how well students
   have learned a specific curriculum but rather their ability to apply understandings in reading, mathematics and science to
   everyday problems and situations.

                                                                                                                                  17
PREPARING FOR A RENAISSANCE IN ASSESSMENT

Figure 1.2 Annualised change across PISA assessments of reading and mathematics for top nine
           performing countries. Source OECD (2013b).

                         Readings

           3.00
           2.00                                                                Korea

           1.00
                                   Canada                                                 Japan
           0.00
                                                        Australia
          -1.00                                                                                              Belgium
          -2.00                                  NZ                 Ireland
          -3.00          Finland
          -4.00                                                                                   Sweden

                          Mathematics

           3.00
                                   Korea
           2.00                                         Japan                     Switzerland
           1.00
           0.00
                                                                          Belgium
          -1.00                                                                                            NZ
                        Finland
          -2.00
                                                                 Canada
          -3.00                          Netherlands                                          Australia
          -4.00

Much of the attention given to improving                            of 2012 data indicated that around 15 per cent
learning outcomes has been directed at the                          of the variance in mathematics performance
school level. Analyses of the 2009 PISA data                        could be attributed to differences between
indicate that in the participating countries,                       schools (OECD 2013c: Table IV.1.12a). In
after adjustments for demographic and socio-                        other words, there are substantial differences
economic characteristics, around 20 per cent                        between schools even when their intake
of the variance in reading performance could                        characteristics have been taken into account.
be attributed to differences between schools                        Research into school effectiveness, much of
(OECD 2011:Table IV.2.2a).The same analyses                         which was undertaken in the 1980s and early

 	It should be noted, however, that there are those who argue that tests such as PISA, which seek to provide a common
7.

   yardstick across nations, are not sensitive to improvements in teaching and learning. PISA does not assess how well students
   have learned a specific curriculum but rather their ability to apply understandings in reading, mathematics and science to
   everyday problems and situations.

18
SETTING THE SCENE

                                                                   ‘
1990s, has provided us with a good knowledge
of the more powerful school-level levers for
                                                                                        ‘
                                                                        quality of teaching is the key to
                                                                        unlocking significant improvements
improvement. Strong educational leadership,                         in outcomes
a small number of strategic priorities and
a climate of high expectations of student                          There is now a wide consensus that quality
behaviour and learning are among the factors                       of teaching is the key to unlocking significant
that have delivered remarkable and rapid                           improvements in outcomes. In 2007, Barber
turnarounds.                                                       and Mourshed, in How the World’s Best-
                                                                   Performing School Systems Come Out on Top,
However, estimates of school effects can be                        concluded that three things matter most:
misleading. Analyses that take into account the
fact that students are not only taught within                         1 getting the right people to become
a given school but are also in a particular                              teachers;
class within that school, result in much lower                        2 developing them into effective instruct­
estimates of the variance in outcomes at the                             ors; and
school level but high proportions of variance                         3 ensuring that the system is able to
at the class level. For example, in one such                             deliver the best possible instruction for
study conducted by Hill and Rowe in Australia                            every child.
in the 1990s, it was found that fitting a two-
level model (students within schools) to local                     In response to the call for a greater focus
assessment data resulted in estimates of school                    on teaching quality, many nations have
effects of 17.6 per cent for English and 16.6                      initiated work on clarifying teacher roles and
per cent for mathematics (very similar to the                      expectations, improving the quality of recruits
OECD two-level model outcomes). However,                           into teaching, ensuring that pre-service
three-level modelling (students within classes,                    teacher training includes a solid foundation
within schools) resulted in estimates of 8.2                       of professional practice and systematically
per cent for English and 5.4 per cent for                          building opportunities to reflect on and
mathematics at school level, but 43.7 per cent                     enhance their practice into teachers’ daily
for English and 56.4 per cent for mathematics                      lives. In a few countries, but particularly in
at class level (Hill and Rowe 1996).                               the USA, a key part of the solution is seen
                                                                   as the implementation of systems of teacher
In other words, it matters more which class                        accountability for student learning, with direct
a student is assigned to than which school                         links between individual teachers and their
they attend.This is not an altogether surprising                   students’ test scores.
conclusion when one considers that learning
takes place in classrooms with a specific                          However, a succession of other commentators,
teacher and a class of students with particular                    beginning with Dan Lortie in 1975 and most
backgrounds, but it points to the fact that,                       recently Jal Mehta (2013), have reached a
in order to improve learning, it is important                      more fundamental conclusion.8 They believe
to focus on what is happening in individual                        that, in many nations, improvements to the
classrooms and on the quality of teaching                          quality of teaching can only come through
received by each student.                                          the transformation of teaching from a largely

 	Lortie is quoted in the insightful and scholarly review of the field by Grossman and McDonald (2008).
8.

                                                                                                                 19
PREPARING FOR A RENAISSANCE IN ASSESSMENT

under-qualified and trained, heavily unionised,        by Hauser, Professor Geoff Masters presents
bureaucratically controlled ‘semi-profession’          a dramatic depiction of the extent of the
into a true profession with a distinctive              overlap in performance of more than a
knowledge base, a framework for teaching,              quarter of a million mathematics students in
well defined common terms for describing               different grades in the USA (2013: Fig. 2.3; see
and analysing teaching at a level of specificity       Figure 1.3). Much of the overlap seems to be
and strict control, by the profession itself,          a consequence of the fact that high-achieving
on entry into the profession. Broadly, we              students make steady progress, but low-
agree with this analysis (noting that this             achieving students make very little progress
characterisation of teaching is less applicable        over time.
in many Asian countries) and believe that the
performance ceiling will remain until the full         The phenomenon of wide variations in
professionalisation of teaching, in this sense,        performance of students of the same age is
has become a reality. This is what Michael             observed in almost all studies where vertically
Barber has called ‘informed professionalism’           equated test data (across age grades) are
(2014: slide 3).                                       available. These variations indicate that the
                                                       greatest opportunities for improvement exist
Whatever the precise contribution of                   at the student level, but, so far, few systems have
teacher effects (quality of teaching) or the           been able to significantly narrow achievement
optimum strategies for maximising them, it             gaps within grades.
is unquestionably the case that the greatest
proportion of variance in learning outcomes            We would suggest that this is in no small part
is at student level. Using data from a study           due to the way in which school education

Figure 1.3 Distributions of students’ mathematics achievements (Years 2–7, USA, 2003).
           Source: Masters (2013).
                    Year 2       Year 3       Year 4         Year 5       Year 6        Year 7

      Band 6

      Band 5

      Band 4

      Band 3

      Band 2

      Band 1

20
You can also read