Assessment in Education - Challenges and possibilities - Professor Christian Lundahl's inaugural lecture for the Fritz-Karsen-Chair - Professional ...
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Assessment in Education –
Challenges and possibilities
Professor Christian Lundahl’s inaugural lecture for the
Fritz-Karsen-Chair
The department of education,
Örebro university
The department of education at Örebro university has a central position in teacher
and pre-school teacher training programs and contributes to the scientific basis of
these courses. Our educational research is mainly conducted within two research
environments: SMED (Studies of Meaning-Making in Educational Discourses) and
Education and Democracy. In addition we have didactical research in the
Humanities, Social Sciences, Science and Math, Sports and in Music.
christian.lundahl@oru.se
My main research interests
• The history of education
• Assessment in education
• International and Comparative education
• Curriculum theory and educational policy
• Research based learning in Higher education
Publications and ongoing projects:
https://www.oru.se/english/employee/christian_lundahl
Research bloggs: www.paristopisa.com ; www.skoloverstyrelsen.se
Twitter: @drlundahl
christian.lundahl@oru.se
Assessment in education
• A broad field that includes theory, policy and practice of
assessment in education.
• Assessments can be based on psychometrical
measurements and/or professional judgements.
• Assessments can be used for evaluative purposes.
• When using assessments, validity becomes central. Validity
is about making sound inferences of assessment data – with
an awareness of consequences of the inferences (Messick
1989)
• It is a fast growing field with 125.000 publications in ERIC
(55.000 PeerR)
• Important journals, e.g: Assessment in Education: Principles,
Policy & Practice; Studies in Educational Evaluation;
Educational Assessment;
christian.lundahl@oru.se
Publications on Assessment in
Education, ERIC (N=155.000)
christian.lundahl@oru.se
Assessment AND education
“Formal educational knowledge can be considered to
be realized through three message systems:
curriculum, pedagogy and evaluation. Curriculum
defines what counts as valid knowledge, pedagogy
defines what counts as valid transmission of
knowledge, and evaluation defines what counts as
valid realization of this knowledge on the part of the
taught.” (Bernstein 1971/1980:47)
christian.lundahl@oru.se
Assessments as re/productive
Assessments does not only evaluate curriculum, but
helps reproduce what is seen as important
knowledge and positions. The result from
assessments also has a productive side, they produce
new information that can be used to organise
learning and people in new ways. (Lundahl 2006)
christian.lundahl@oru.se
The purposes of educational
assessments
Newton 2007, 161-162
christian.lundahl@oru.se
Three aggregated roles of
educational assessments
christian.lundahl@oru.se
Tveit 2019, 48
Perspectives on assessment in
education chosen for today’s lecture
1) Governing education – Focus on how large scale
international assessments (ILSA) structure national policy
(and reforms) on education
2) The certification of education – Focus on grading and
examinations from a professional perspective. Which
challenges do teachers meet when they form their
judgements for certification purposes?
3) Assessment for learning – How assessments can
contribute to a sustainable learning of complex skills.
christian.lundahl@oru.se1. Governing education – the role of
ILSAs in Swedish educational policy
christian.lundahl@oru.seOECD PISA is relatively seen less and
less googled, but common in Sweden
Google trends
2020-02-02
christian.lundahl@oru.seThe uses of PISA and of Educational
research in Swedish media 6 month
after the PISA release 2016
christian.lundahl@oru.seThe use of PISA in parliamentary
debates 2000-2017
Parliamentary debates
45
40
35
30
25
20
15
10
5
0
2016/17(q1)
2001/02
2002/02
2003/04
2004/05
2005/06
2006/07
2007/08
2008/09
2009/10
2010/11
2011/12
2012/13
2013/14
2014/15
2015/16
Riksdagsår
christian.lundahl@oru.sePISA used for more than 40
causes in Parliamentary debates
PISA for increased equivalence in education 59 PISA and improved working environment 3
PISA to focus on teachers 45 PISA to develop teacher education 3
PISA in order not to lower taxes, but to invest more in education 26 PISA to discuss higher education 3
PISA for early grading 23 PISA for increased teaching time 3
PISA for increased order and discipline 22 PISA and the importance of leadership 3
PISA for students in need of support 21 PISA to focus on talented students 2
PISA and digital skills 20 PISA for school libraries 2
PISA in support of focus on preschool 20 PISA to strengthen families 1
PISA to set higher requirements or for a ‘knowledge-based’ school 17 PISA to introduce assessment aid 1
PISA and youth unemployment 14 PISA for teacher licence 1
PISA and upper secondary school 11 PISA for smaller class sizes 1
PISA and free choice of school 10 PISA for school-based research 1
PISA and immigration 9 PISA and international competitiveness 1
PISA and increased expectations from society 8 PISA as the basis for new in-depth investigations 1
PISA for and against decentralisation or re-centralisation 8
PISA and profit interest 5 Total number of coding references 356
PISA and child health 4
PISA and teacher-centred education 4
Lundahl & Serder (2020) accepted
PISA and private schools 4
to Nordic Journal of Studies in
Educational Policy
christian.lundahl@oru.seParty political uses of PISA 2000-
2018
christian.lundahl@oru.seSentiment analysis – the Left wing
parties when talking about schools in
relation to PISA
christian.lundahl@oru.seSentiment analysis – the Right wing
parties when talking about schools in
relation to PISA
christian.lundahl@oru.seSentiment analysis - examples
Positive tonality
During the day I was at high school I had the pleasure of meeting so many
wise young people and talking about the school, life and much in
between.
The government has prioritized the school in budget by budget, and we
are now seeing the results in PISA, Pirls and in the grading statistics. MP
(The Green Party) Elisabeth Knutsson
Negative tonality
Swedish schools have become increasingly violent and certain groups -
unfortunately, especially with immigrant background - do not hesitate to
use violence and threats of violence to assert themselves.
However, this is only one of many problems that I think the Swedish
school is struggling with today. There are several international surveys that
clearly show that Swedish school has lost quality. SD (The Nationalist
Party) Richard Jomshof
christian.lundahl@oru.seIs this a valid use of PISA results?
“Validity is an integrative evaluative judgement of
the degree to which empirical evidence and
theoretical rationales support the adequacy and
appropriateness of inferences and actions based on
test scores or other modes of assessment.” (Messick,
1989, p. 13.)
Threats to validity according to trad. test-theory:
Construct under-representation (important stuff are
not measured)
Construct-irrelevant variance (wrong things are
measured, i.e. reading skills in a math test)
christian.lundahl@oru.sePisa and Timss are fairly valid
measures of Swedish curriculum
outcome
Two recent PhD-theses shows that both Pisa and
Timss are fairly valid to Swedish curriculum, however
both test suffer from some under-representation
when it comes to complex skills such as problem
solving and communication skills. (Sollerman 2019,
Palm Kaplan 2019)
christian.lundahl@oru.seHowever, the political inferences
are seldom valid
• Pisa does for example not say anything about when to best
start formal grading, which Swedish politicians claimed, and
which led to a huge grading reform. When analysing the
statistics we found no correlation at all between grading
systems and Pisa scores. (Lundahl et al 2015, Tveit &
Lundahl 2017)
• In depth analyses of PISA results, can be done by ”thinking
out load”-analysis when the pupils take the test. Recent
studies shows that one major reason to the increased
inequivalence between Swedish schools is that the level of
subject specific language the children are of good command
of differ between schools (Nygård Larsson & Jakobsson
2019). However, providing teachers with better subject
didactic skills is not so politically attractive…
christian.lundahl@oru.se2. The certification of education –
challenges to a fair grading
christian.lundahl@oru.seExamination cultures
• The psychometric paradigm – the measure of mind
(stems from Galton 1869)
• The outcomes-based assessment paradigm – the
setting of goals, criteria or outcomes (stems from
Tyler 1949)
• The curriculum-based assessment paradigm –
stems from Chines imperial examinations and from
grammar-school examinations. Teachers assess
their students’ knowledge based on professionl
experience. Assessment is seen as an integrated
part of the learning process.
Baird et al 2018
christian.lundahl@oru.seThe lack of equivalence in teacher
grading – differences between ’test-
grades’ and final grades
christian.lundahl@oru.seThe lack of equivalence in teacher
grading – differences between ’test-
grades’ and final grades in different
subjects
English upper
Final Course grades secondary
higher than test grades
Final Course grades
lower than test grades
Final Course grades Math upper
higher than test grades secondary
Final Course grades
N=1123 Schools lower than test grades
christian.lundahl@oru.seRegional differences
• In two geographically close regions, one region,
Blekinge awarded 45,7 of their students with a
higher final grade than their test grade whereas the
other region, Gotland only awarded 27,4% of their
students with a higher grade.
christian.lundahl@oru.seWhy is it so difficult to give fair
grades?
• The most common explanation to these differences
is that private schools (Friskolor) gives higher
grades in order to attract students.
• Assessment researchers would also like to add, that
it is a matter of complexity in the system and the
competence to deal with it.
• These difficulties does not only Swedish teachers
have, but since we have our national test to
compare with, we might be more aware of the
problem. (See Vogt 2017, Falkenberg 2020 for a
German Swedish comparison)
christian.lundahl@oru.seWhy is it so difficult to give fair
grades? (A teacher perspective)
• Teachers has to interpret curriculum and course-
planes and turn these interpretations into actual
teaching, assignments and tests.
• Teachers then has to interpret every students
achievement, eg. does the student lack of
understanding, or failed the test for other reasons?
• Teachers has to give weight to different individual
performances – should any achievement be valued
higher than others, and should that affect the
grade?
christian.lundahl@oru.seResearch about teachers
assessments
How reliable are teacher evaluations?
Starch & Elliott (1912):
Two different essays from high school were copied
and distributed to 200 schools.
• Teachers asked to assess ”according to the practice
and standards of the school” (p. 449).
• Results (deviation , 0-100): Essay 1: 60-97
Essay 2: 50-97
christian.lundahl@oru.seResearch about teachers
assessments
Brimi (2011):
Brimi copied Starch & Elliott’s study from 1912 but
with 90 teachers specially trained to assess writing
were asked to evaluate the same essay.
• Results (rating, 0-100): 50-93
christian.lundahl@oru.seResearch about teachers
assessments
Eells (1930):
61 teachers re-marked five assignments in history and geography 11
weeks after their first marks.
• Results (teachers that gave the same marks at the second assessment,
%):
Assignment 1. 16
Assignment 2. 90
Assignment 3. 23
Assignment 4. 34
Assignment 5. 28
None of the teachers gave the same marks to all of the assignments!
christian.lundahl@oru.seResearch about teachers
assessments
Eells (1930):
• Result (correlation): 0,25 – 0,51
”It is unnecessary to state that reliability coefficients
as low as these are little better than sheer guesses”
(p. 52).
christian.lundahl@oru.seHow do teachers give grades
• Holistic assessments/connoisseur approach
• Analytical assessments/arithmetical and or
deliberative approach
christian.lundahl@oru.seHolistic assessment
https://www.cultofpedagogy.com/holistic-
analytic-single-point-rubrics/
christian.lundahl@oru.seAnalytical assessment
https://www.cultofpedagogy.com/holistic-
analytic-single-point-rubrics/
christian.lundahl@oru.seHow do teachers give grades
• Teachers tend to grade holistically. (eg Bloxham, Boyd &
Orr, 2011)
• Holistic assessments are less equivalent compared to
analytical assessments. (Jönsson & Svingby 2007)
• However, students tend to adopt to analytical rubrics
leading to a more instrumental learning.
• Therefor so called ”emergent criteria” has been
promoted (the teacher disclose the criteria used first
when giving feedback). (Sadler 2009)
christian.lundahl@oru.seHow do teachers give grades
• A recent experimental study showed that teachers
who were asked use analytic assessment of an
assignment to 75% came to the same conclusion
regarding the grade, whereas teachers who used
holistic assessment agreed to less than 50%.
(Jönsson & Balan 2018)
• But the problem in not just about giving the right
grade, it is also about doing purposeful
assessments that at least not stands in the way for
learning, which grading sometimes do.
christian.lundahl@oru.sePs. Not only teachers have
problems assessing fairly
Nature:
https://www.nature.c
om/news/2011/11041
1/full/news.2011.227.
christian.lundahl@oru.se html3. Assessment for learning – How can
assessments contribute to learning
The term assessment for learning (AFL) was first used
at the annual conference Association for Supervision
and Curriculum Development in England 1992
(James 1992). Three years later it appears in a book
title (Sutton 1995). The first time AFL was contrasted
with assessment of learning, (summative
assessment) was in Gipps & Stobart 1997, where the
titles are the title of chapters 1 and 2. The difference
between the two types of assessment was then
spread through the Assessment Reform Group (ARG)
in a guide for policymakers 1999 (Broadfoot,
Daugherty et al. 1999).
christian.lundahl@oru.seA seminal definition of Formative assessment “Assessment’ refers to all those activities undertaken by teachers, and by the students in assessing themselves, which provide information to be used as feedback to modify the teaching and learning activities in which they are engaged. Such assessment become ‘formative assessment’ when the evidence is actually used to adapt the teaching to meet the needs.” Black & Wiliam, 1998, p. 2 (Inside the black box)
A seminal definition of Formative
assessment
Wiliam & Thompson 2007
christian.lundahl@oru.seA promising idea that maybe didn’t reached its full potential • Initial systematic research reviews pointed at effect sizes between 0.4 and 0.7 on learning (Black & Wiliam 1998). • This lead to huge reforms in US and England and Norway. But the evaluation of these reforms indicated poor, if any, gain. (eg. Smith & Engelsen 2012; Jönsson, Lundahl & Holmgren 2014) • Critics has pointed at weaknesses in the first review (included to many special not generalisable cases). (Dunn & Mulvenon 2009) • However the problem has more likely to do with the practice of formative assessments – it became too teacher- and teaching oriented and not student- and learning oriented. (Lauvås & Jönsson 2019)
Towards assessments as self-
regulated learning
A special issue of Assessment in education 2015
states: Formative assessment – an optimistic but
incomplete vision.
Paul Black (2016) later clarified: à
christian.lundahl@oru.seTowards assessments as self-
regulated learning
Black (2016) summarizes what it was they originally wanted to
highlight on how formative assessments promote learning:
a) Pupils' ability to learn through critical dialogue
b) Pupils' ability to reflect critically on their work and take initiatives
to improve it
c) Pupils' ability to work in groups
d) The students' ability to gain an overview of their learning
progression and development in relation to the goals set
e) Pupils' ability to develop independence and autonomy
(empowerment of learners), so that they can make informed
decisions about their work
f) The ability of students to manage overall goals so that they can
cope with the increasingly complex reality in which we live.
christian.lundahl@oru.seSelf-regulated learning and
developing an evaluative judgment
• Assessments has, maybe more than other aspects of
education, the potential to ”enable people to become
skilled learners with control over their own learning,
appropriately assessing personal need and applying
strategies for progress” (Fazey 1993, from Nulty 2010,
p. 503)
• This can happen if pupils are trained to develop “an
evaluative judgement”, i.e. to distinguish between good
and not so good work; the realisation that quality is a
reference to a standard, formally written or in the
minds of experts, but always contextually bound to
disciplinary notions or professional practice. (Boud et al
2018)
christian.lundahl@oru.sePractices to align assessment with
self-regulated learning and evaluative
judgment
• Involve students in working with standards, goals
and assessment criteria
• Authentic assignments
• Longsighted assessments (portfolio, process-
writing)
• Process oriented and forward-looking feedback
• Peer-assessment
• Self-assessment (Lundahl 2011, 2020)
christian.lundahl@oru.seClearify expectations with rubrics
Undergraduate Research Presentation Rubric
Name_______________________________________________________________
Undergraduate research is becoming more important in higher education as evidence is accumulating that clear, inquiry-based learning, scholarship, and creative
accomplishments can and do foster effective, high levels of student learning. This curricular innovation includes identifying a concrete investigative problem, carrying
out the project, and sharing findings with peers. The following standards describe effective presentations.
5-4 3-2 1-0
Standards Score Weight Total
Exemplary Satisfactory Unacceptable
Score
Has a clear opening statement Has opening statement relevant Has no opening statement or has
Organization
that catches audience’s interest; to topic and gives outline of an irrelevant statement; gives
maintains focus throughout; speech; is mostly organized; listener no focus or outline of the X2
summarizes main points provides adequate “road map” presentation
for the listener
Content Demonstrates substance and Covers topic; uses appropriate Does not give adequate coverage
depth; is comprehensive; shows sources; is objective of topic; lacks sources X2
mastery of material
Quality of conclusion Delivers a conclusion that is well Summarizes presentation’s main Has missing or poor conclusion;
documented and persuasive points; draws conclusions based is not tied to analysis; does not X2
upon these points summarize points that support
the conclusion
Delivery Has natural delivery; modulates Has appropriate pace; has no Is often hard to understand; has
voice; is articulate; projects distracting mannerisms; is easily voice that is too soft or too loud; X 1.5
enthusiasm, interest, and understood; has a pace that is too quick or too
confidence; uses body language slow; demonstrates one or more
effectively distracting mannerisms
Use of media Uses slides effortlessly to Looks at slides to keep on track; Relies heavily on slides and
enhance presentation; has an uses an appropriate number of notes; makes little eye contact; X 1.5
effective presentation without slides uses slides with too much text
media
Response to Questions Demonstrates full knowledge of Shows ease in answering Demonstrates little grasp of
topic; explains and elaborates on questions but does not elaborate information; has undeveloped or X1
all questions unclear answers to questions
Reviewer_________________________________________________ Grand Total _____________________
https://rubrics.kon.org/rubric-documents/Undergraduate-
7/15/2009 – Dorothy Mitstifer, Kappa Omicron Nu
Research-Presentation-Rubric-2006.pdfInvolve students working with standards. Use Student example. Grade Pass (Art studies)
Pass with distinction
Involve students working with standards: purpose of learning Reading comprehension strategies: The fortune-teller predicts and puts hypotheses on the text by looking at headings, pictures, captions and text genres. The reporter asks questions at different levels about the text. These levels are on the line, between the lines and beyond the lines. The artist creates internal images of the reading. With the help of the senses, the reader lives in the text and can see, hear and feel what the text tells about. The detective casts out ambiguities, new words and expressions. The cowboy summarizes the most important in the text.
Internet tasks can create increased authenticity
The physics on
Angry Birds
(av Daniel
Barker)
Copyright Christian LundahlEleverna i NNB10 lyckades bestämma vilken vinkel fågeln skulle skjutas iväg med för
att träffa där de ville. För den som undrar kan vi avslöja att fågelns utgångshastighet
är 22 m/s och att slangbellans höjd är 4,9 m.
http://norrarealsthlm.blogspot.se/2012/04/fysik-ipad-och-angry-birds.html
Copyright Christian LundahlSelf-assessment
Pre-flight checklist
• Have you made it clear to you who you write the text for and why?
• Do you write the most important first?
• Does the text only contain the information the reader needs?
• Have you formulated headlines claims, requests or questions?
• Do you use key words and verbs in headlines?
• Do you use the same words every time you mention a phenomenon?
• Have you printed all the acronyms?
• Do you use many foreign words?
• Did you turn up any words you are unsure of?
• Have you done a spell check?
• Have you proofread the paper and pencil?
• Have you let someone read your text?
Forsberg (2008): Tydliga texter
58Student reflection • What do you think you did well? • What do you think got less well? • Is there anything you are not sure whether it's good or not so good? (true / false, adequate / inadequate) • How many hours have you put into this work? • How has the process gone, how have you worked? • Could you have worked in a different way?
Teacher comments to student reflection • I completely agree with you that you did a good job with ... • You write that you can not ... But the third section shows that you actually master this! • I agree that you do not yet have control over ... yet. This we will continue to work with. • You are not sure if you understood ... Is this something you would like help or would if you work on yourself? • You say you worked 10 hours ... I can’t really see that. How did you work in more detail? (free from Hartberg, Dobson & Gran 2012, p. 117)
To conclude: Towards Sustainable
assessments
• At a structural, as well as at a professional and learning
level assessments too often tend to be assessments of
past activities, not evaluating its own consequences or
addressing future needs.
• There is however a view that assessment can be made
sustainable if they ‘meets the needs of the present
without compromising the ability of students to meet
their own future learning needs’ (Boud, 2000, p. 151).
• That is, assessment activities should not only address
the immediate needs of governing, certification or
feedback on the current situation, but also contribute
in some way to a more prospective learning at all
levels. This requires -->
christian.lundahl@oru.seWhat can we as scholars and
trainers of teachers do?
• We must be better to analyse the actual construction
and the results of ILSAs. Its not just enough to look at
the political consequences if we like to contribute with
a more complex understanding. (cf Messick; Koretz;
Jakobsson; Serder)
• We can be better at understanding the rationalities
behind teacher grading practices. (cf Waldow et al on
teachers and social justices cultures; Jönsson on
teacher judgments)
• We can be better to involve teacher students in our
own assessment practice. After all we want them to
utilise assessments as a tool for complex and
sustainable learning on their future pupils. (cf Boud;
Saunders; Lundahl)
christian.lundahl@oru.seThank you for listening!
Please mail me if you would like a copy of the
presentation or the references used at
christian.lundahl@oru.se
christian.lundahl@oru.seReferences
• Baird, J., Isaacs, T., Opposs, D. & Gray, L. (eds.) (2018). Examination standards: how measures and meanings
differ around the world. London: UCL IOE Press.
• Black, P. J., & Wiliam, D. (1998). Inside the black box: raising standards through classroom assessment (London,
King’s College London School of Education).
• Bernstein, B. (1971/1980): On the Classification and Framing of Educational Knowledge. I Young, M. F. D. (ed):
Knowledge and Control. New Directions for the Sociology of Education. London: Collier-Macmillan. Pp. 47–69.
• Bloxham, S., Boyd, P. & Orr, S. (2011). Mark my words: The role of assessment criteria in UK higher education
grading practices. Studies in Higher Education, 36, 655-670.
• Boud, D., Ajjawi, R., Dawson, P. & Tai, J. (2018). Developing Evaluative Judgement in Higher Education. London
& New York, NY: Routledge.
• Brimi, H. M. (2011) "Reliability of Grading High School Work in English," Practical Assessment, Research, and
Evaluation: Vol. 16 , Article 17.
Available at: https://scholarworks.umass.edu/pare/vol16/iss1/17
• Broadfoot, P. M., Daugherty, R., Gardner, J., Gipps, C. V., Harlen, W., James, M., & Stobart, G. (1999).
Assessment for learning: beyond the black box. Cambridge, UK: University of Cambridge School of Education.
• Dunn, K. E. & Mulvennon, S. W. (2009). A critical review of research on formative assessment: The limited
scientific evidence of the impact of formative assessment in education. Practical Assessment, Research &
Evaluation Vol 14, 7, 1-11.
• Eells, W. C (1930). Reliability of repeated grading of essay type examinations. Journal of Educational
Psychology 21(1):48-52
• Falkenberg, K. (2020). Gerechtigkeitsüberzeugungen bei der Leistungsbeurteilung. Eine Grounded-Theory-Studie
mit Lehrkräften im deutsch-schwedischen Vergleich. Wiesbaden: VS Springer.
christian.lundahl@oru.seReferences
• Forsberg, J. (2008). Tydliga texter: snabba skrivtips och språkråd. Lund: Studentlitteratur.
• Gipps, C. V., & Stobart, G. (1997). Assessment: a teacher’s guide to the issues (3 ed.). London, UK: Hodder and
Stoughton.
• Hartberg, E.W., Dobson, S. & Gran, L. (2012). Feedback i skolen. Oslo: Gyldendal akademisk.
• James, M. (1992). Assessment for learning Paper presented at the Annual Conference of the Association for
Supervision and Curriculum Development (Assembly session on 'Critique of Reforms in Assessment and Testing
in Britain') held at New Orleans, LA.Cambridge, UK: University of Cambridge Institute of Education.
• Jönsson, A. & Balan, A. (2018). Analytic or Holistic: A Study of Agreement Between Different Grading Models.
Practical Assessment, Research & Evaluation, 23(12).
• Jönsson, A., Lundahl, C., & Holmgren, A. (2015). Evaluating a large-scale implementation of Assessment for
Learning in Sweden. Assessment in Education: Principles, Policy & Practice, 22(1), 104-121.
• Jönsson, A. & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and educational consequences.
Educational Research Review, 2, 130-144.
• Lauvås, P. & Jönsson, A. (2019). Ren formativ bedömning. Lund: Studentlitteratur.
• Lundahl, C. (2006): Viljan att veta vad andra vet. Kunskapsbedömning i tidigmodern, modern och senmodern
skola. Arbetsliv i omvandling 2006:8. Akademisk avhandling vid Uppsala universitet. Stockholm:
Arbetslivsinstitutet.
• Lundahl, C. (2011): Bedömning för lärande. Stockholm: Norstedts.
• Lundahl, C. (2018). The Organising Principles of Disciple Assessment in the Swedish School Ordinances 1561–
1724. Alarcón, C. & Lawn, M. (eds.) Student Assessment Cultures in historical perspective. Studia Educationis
Historica. Berlin, Germany: Peter Lang.
christian.lundahl@oru.seReferences
• Lundahl, C., Hutlén, M., Klapp, A. & Mickwitz, L. (2015). Betygens geografi – forskning om betyg och summativa bedömningar i
Sverige och internationellt. Delrapport från skolforsk-projektet. Vetenskapsrådet. Stockholm: Vetenskapsrådet.
• Lundahl, C., Hultén, M., & Tveit, S. (2017). The power of teacher-assigned grades in outcome-based education. Nordic Journal of
Studies in Educational Policy 3(1), 56-66.
• Lundahl, C. & Serder, M. (2020). Is PISA more important to school reforms than educational research? The selective use of
authoritative references in media and in parliamentary debates. Accepted to Nordic Journal of Studies in Educational Policy.
• Messick, S. (1989): Validity. In R. L. Linn (ed.): Educational Measurement. Third edition 1993. Phoenix: The Oryx Press. Pp. 13–103.
• Palm Kaplan, K. (2019). International large-scale assessments and mathematics textbooks in a curriculum reform process: Changes
in lower secondary school algebra in Sweden 1995–2015 (Doctoral dissertation, Acta Universitatis Upsaliensis).
• Nulty, D. D. (2010). Peer and self-assessment in the first year of university. Assessment & Evaluation in Higher Education, 36:5,
493–507.
• Nygård Larsson, P. & Jakobsson, A. (2019). Elevers samtal om en PISA-uppgift – naturvetenskap i olika skolkontexter. Utbildning och
Demokrati, vol 28, 3, 17-40.
• Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18, 119-144.
• Sadler, D. R. (2009). Indeterminacy in the use of preset criteria for assessment and grading. Assessment & Evaluation in Higher
Education, 34, 159-179.
• Smith, K. & Engelsen, K. S. (2012). Developing an assessment for learning (AfL) culture in school: the voice of the principals.
International Journal of Leadership in Education, 1/20. DOI: 10.1080/13603124.2012.699639.
• Sollerman, S. (2019). Kan man räkna med PISA och TIMSS?: Relevansen hos internationella storskaliga mätningar i matematik i en
nationell kontext (Doctoral dissertation, Institutionen för matematikämnets och naturvetenskapsämnenas didaktik, Stockholms
universitet).
• Starch, D. and Elliott, E.C. (1912) Reliability of the Grading of High School Work in English. School Review, 20, 442-457.
• Sutton, R. (1995). Assessment for learning. Salford, UK: RS Publications.
christian.lundahl@oru.seReferences
• Tveit, S. & Lundahl, C. (2017). New Modes of Policy Legitimation in Education: (Mis)using Comparative Data Effectuating
Assessment Reform. European Educational Research Journal. Article first published online: September 20, 2017
• https://doi.org/10.1177/1474904117728846
• Tveit, S. (2019). Policy Legitimation of Educational Assessment Reforms. The Cases of Norway and Sweden. Diss. Oslo university.
• Vogt, B. (2017). Just assessment in school – a context-sensitive comparative study of pupils’ conceptions in Sweden and Germany
(Doktorsavhandling). Linnaeus University, Växjö/Kalmar.
• Wiliam, D., & Thompson, M. (2007). Integrating assessment with instruction: what will it take to make it work? In C. A. Dwyer (Ed.),
The future of assessment: shaping teaching and learning (pp. 53-82). Mahwah, NJ: Lawrence Erlbaum Associates.
christian.lundahl@oru.seYou can also read
