Are 15-year-olds prepared to deal with fake news and misinformation? - #113 - APO
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
#113
Are 15-year-olds prepared
to deal with fake news and
i n F o c u s misinformation?
Programme for International Student AssessmentAre 15-year-olds prepared to deal with fake news and
misinformation?
• An average of 54% of students in OECD countries reported being trained at school on how to
recognise whether information is biased or not. Among OECD countries, more than 70% of students
reported receiving this training in Australia, Canada, Denmark, and the United States. However, less
than 45% of students reported received this training in Israel, Latvia, the Slovak Republic, Slovenia,
and Switzerland.
• Students from advantaged socio-economic backgrounds in all participating countries and economies
in PISA 2018 scored higher in the index of knowledge of reading strategies for assessing the credibility
of sources than students from disadvantaged socio-economic backgrounds.
• Education systems with a higher proportion of students who were taught whether information is
subjective or biased were more likely to distinguish fact from opinion in the PISA reading assessment
even after accounting for country per capita GDP or reading performance.
Digital technologies have changed how people PISA 2018 asked students whether during their
interact with information. PISA data shows that entire school experience they were taught a) how
15-year-olds increasingly read online to fulfil to use keywords when using a search engine such
information needs (e.g. online news versus as , , etc, b) how to decide
newspapers). At the same time, technological whether to trust information from the Internet,
changes in the digitalisation of communication c) how to compare different web pages and
continue to reshape people’s habits (e.g. chats decide what information is more relevant for their
online versus emails). Fifteen-year-olds’ total schoolwork, d) to understand the consequences of
online consumption has risen from 21 hours making information publicly available online, e) how
a week in PISA 2012 to 35 hours per week in to use the short description below the links in the
PISA 2018 – almost the equivalent of an average list of results of a search, f) how to detect whether
adult workweek in OECD countries. The massive information is subjective or biased, and g) how to
information flow that characterises the digital detect phishing or spam emails.
era demands that readers be able to distinguish
between fact and opinion, and learn strategies to The most common digital skill taught at school on
detect biased information and malicious content average across OECD countries is understanding
such as phishing emails or fake news. Academic the consequences of making information publicly
research is inconclusive about the prevalence and available online (Figure 1). The least common
importance of misinformation and fake news1. skill was how to detect phishing or spam emails
Yet,the consequences of being poorly informed have (76% and 41% of students in OECD countries
been largely documented. It can lead to political reported being taught this during their entire school
polarisation, decreased trust in public institutions experience). There are also considerable differences
and undermined democracy. across and within countries. An average of 54% of
students in OECD countries reported being trained
Students’ use of the Internet at school on how to recognise whether information
continues to increase while the is biased or not. Among OECD countries, more
than 70% of students reported receiving this
opportunity to learn digital skills in training in Australia, Canada, Denmark, and the
school is far from universal. United States. However, less than 45% of students
2 © OECD 2021 PISA in Focus 2021/113 (May)reported received this training in Israel, Latvia, OECD countries was 8 percentage points in favour
the Slovak Republic, Slovenia, and Switzerland. of advantaged students. In Belgium, Denmark,
The percentage difference in students who were Germany, Luxembourg, Sweden, the United
taught how to detect biased information between Kingdom and the United States, this difference is
students from socio-economic advantaged around 14 percentage points or higher.
and disadvantaged backgrounds2 across
Figure 1: Frequency of opportunity to learn digital literacy skills at
school
Students reported that during their entire school experience were taught the following, OECD average
% Top country/economy OECD average Bottom country/economy
100
90
80
70
60
50
40
30
20
10
0
How to detect How to use the How to detect How to use How to compare How to decide To understand
phishing or short description whether the keywords when different web whether to trust the
spam emails below the links information is using a search pages and information consequences of
in the list of subjective or engine such as decide what from the making
results of a biased , information is Internet information
search , etc. more relevant publicly available
for your online on
schoolwork ,
,
etc.
Items are ranked in ascending order of the percentage of students within OECD average.
Source: OECD, PISA 2018 Database, Table B.2.6.
PISA 2018 also included several scenario-based across all participating countries and economies
tasks where students were asked to rate how useful in PISA 2018. Among OECD countries, students
different strategies were to solve a particular reading in Chile, Colombia, Hungary, Korea, Mexico, and
situation. One of these scenarios asked students Turkey had the lowest scores in this index (lower
to click on the link of an email from a well-known than -0.20 points).
mobile operator and fill out a form with their data
to win a smartphone, also known as phishing Students from advantaged socio-economic
emails. Approximately 40% of students on average backgrounds in all participating countries and
across OECD countries responded that clicking economies in PISA 2018 scored higher in the
on the link was somewhat appropriate or very index of knowledge of reading strategies for
appropriate. Students in Denmark, Germany, Ireland, assessing the credibility of sources than students
Japan, the Netherlands, and the United Kingdom from disadvantaged socio-economic backgrounds
scored the highest in the index of knowledge of (Figure 2). Students in Germany, Luxembourg,
reading strategies for assessing the credibility Portugal, Switzerland and the United States, in
of sources3 (higher than 0.20 points) across all particular, reported the largest socio-economic gap
participating countries and economies in PISA 2018. (0.65 points or higher) in this index of knowledge
In contrast, students in Baku (Azerbaijan), Indonesia, of strategies for assessing the credibility of sources
Kazakhstan, the Philippines, and Thailand had the across all participating countries and economies in
lowest scores in this index (lower than -0.65 points) PISA 2018. In contrast, Albania, Baku (Azerbaijan),
Kazakhstan, and Macao (China) reported the
© OECD 2021 PISA in Focus 2021/113 (May) 3smallest socio-economic gap (lower than 0.15 indirect result of disparities in socio-economically
points). Among OECD countries, Canada, Estonia, advantaged and disadvantaged students’
Iceland, Ireland, Italy, Korea, Latvia, and Lithuania knowledge of effective reading strategies. Empirical
reported the smallest socio-economic gap (lower studies have shown that classroom interventions
than 0.35 points). Most importantly, PISA 2018 aimed at developing students’ assessment of
data shows that, on average across OECD information reliability are effective in improving
countries, about one-third (32%4) of the difference in students’ critical thinking when comprehending
reading performance between socio-economically multiple documents.
advantaged and disadvantaged students is the
Figure 2: Students’ knowledge of reading strategies for assessing the
credibility of sources, by socio-economic status
All students Socio-economically disadvantaged students¹ Socio-economically advantaged students
United Kingdom Viet Nam
Japan Saudi Arabia
Germany Croatia
Netherlands Malta
Denmark Iceland
Ireland Israel
Finland Slovak Republic
Singapore Turkey
Austria Jordan
Australia Qatar
New Zealand United Arab Emirates
Greece Brunei Darussalam
France Costa Rica
Sweden Uruguay
Estonia Hungary
B-S-J-Z (China) Colombia
Switzerland Korea
Ukraine Serbia
Belgium Chinese Taipei
Latvia Chile
Portugal Brazil
Canada Panama
United States Mexico
Spain Morocco
OECD average Peru
Czech Republic Malaysia
Slovenia Montenegro
Norway Bosnia and Herzegovina
Poland Georgia
Italy Bulgaria
Lithuania Dominican Republic
Argentina Kosovo
Luxembourg Albania
Russia Kazakhstan
Moldova Philippines
Macao (China) Baku (Azerbaijan)
Belarus Thailand
Romania Indonesia
Hong Kong (China)
-1.00
-0.80
-0.60
-0.40
-0.20
0.00
0.20
0.40
0.60
0.80
-1.00
-0.80
-0.60
-0.40
-0.20
0.00
0.20
0.40
0.60
0.80
Mean index
1. The socio-economic profile is measured by the PISA index of economic, social and cultural status (ESCS). A socio-economically disadvantaged
(advantaged) student is a student in the bottom (top) quarter of the ESCS in the relevant country/economy.
Note: All differences between socio-economically advantaged and disadvantaged students are statistically significant.
Countries and economies are ranked in descending order of the mean index of all students’ knowledge of reading strategies for assessing the credibility
of sources.
Source: OECD, PISA 2018 Database, Tables B.5.11 and B.5.12c
4 © OECD 2021 PISA in Focus 2021/113 (May)Education systems with a higher PISA 2018 data show that the opportunity students
had to learn how to detect whether information
proportion of students taught
was subjective or biased in school is strongly
how to detect biased information associated with the estimated percentage correct7
in school were more likely to in the item that focuses on distinguishing facts from
distinguish fact from opinion in the opinions in the PISA reading assessment (R2=0.46)
PISA reading assessment. in OECD countries (Figure 3). This relationship is
still observed even after accounting for per capita
The PISA 2018 reading assessment included one GDP or reading performance8. This association
item-unit (i.e. Rapa Nui Question 3) that tested is weaker among all participating countries and
whether students can distinguish between facts economies in PISA 2018 (R2=0.15)9. In conclusion,
and opinions. This question is a typical Level 5 when the education system provides students with
task5. In this item, students must classify five in-school opportunities to learn how to detect biased
separate statements taken from a review of a information, it is this rather than overall reading
book, Collapse, as either “fact” or “opinion”. Only performance or GDP per capita that is driving a
students who classified all five statements correctly strong association with the estimated percentage
were given full credit; partial credit was given to correct in the item on distinguishing fact from
students who classified four out of five statements opinion. These results do not mean that opinions
correctly (this corresponds to Level 3 proficiency6). are not important for contextualising information,
The most difficult statement in this list is the first especially when the facts in question require some
statement (“In the book, the author describes explanation. Rather, the results imply that being able
several civilisations that collapsed because of to distinguish fact from opinion, assess the credibility
the choices they made and their impact on the of information sources, and learn strategies to detect
environment”). It presents a fact (what the book is biased or false information are necessary skills for
about) but some students, particularly those whose reading in a digital world. Ultimately, the ability to
proficiency is below Level 5, may have misclassified distinguish good from bad information is important
this as “opinion” based on the embedded clause, to preserving democratic values.
which summarises the book author’s theory
(the civilisations “collapsed because of the choices
they made and their impact on the environment”).
© OECD 2021 PISA in Focus 2021/113 (May) 5Figure 3: Reading item of distinguishing facts from opinions and access to
training on how to detect biased information in school
%
80
Above-average item performance and Above-average item performance and
below-average opportunity to learn above-average opportunity to learn
Percentage correct in the capacity to distinguish facts from opinions
United States
Turkey United Kingdom
Netherlands
New Zealand Canada
60 Ireland Australia
Estonia Belgium
Denmark
Israel Germany
(Equated P+, Rapa Nui Question 3)
Sweden OECD countries
Chile Portugal
Japan R² = 0.46
Slovenia Poland Austria Finland
Hungary OECD average: 47%
Switzerland
Spain¹ Italy
40 Latvia Lithuania
Luxembourg Mexico
Norway Greece
France
Slovak Republic Czech Republic
Colombia
Korea
20
OECD average: 54%
Below-average item performance and Below-average item performance and
below-average opportunity to learn above-average opportunity to learn
0
30 40 50 60 70 80 90
%
Percentage of students who were taught how to detect whether the information is subjective or biased
1. In 2018, some regions in Spain conducted their high-stakes exams for tenth-grade students earlier in the year than in the past, which resulted in
the testing period for these exams coinciding with the end of the PISA testing window. Because of this overlap, a number of students were negatively
disposed towards the PISA test and did not try their best to demonstrate their proficiency. Although the data of only a minority of students show clear
signs of lack of engagement (see PISA 2018 Results Volume I, Annex A9), the comparability of PISA 2018 data for Spain with those from earlier PISA
assessments cannot be fully ensured.
Source: OECD, PISA 2018 Database, Table B.2.8.
The bottom line
Preserving democratic values and reinforcing trust in public institutions relies on having well-informed
citizens. Students must develop autonomous and advanced reading skills that include the ability to
navigate ambiguity, and triangulate and validate viewpoints. Students’ training in distinguishing between
fact and opinion, and detecting biased information and malicious content such as phishing emails
greatly varies between countries and students’ socio-economic profiles. Schools can foster proficient
readers in a digital world by closing these gaps and teaching students basic digital literacy.
6 © OECD 2021 PISA in Focus 2021/113 (May)Notes
1. Social platforms representing a big chunk of online media consumption are particularly vulnerable to the spread of online
misinformation and fake news (Pennycook and Rand, 2019). Social media algorithms are designed to channel the flow of
like‑minded people towards each other. This creates “echo chambers” reinforcing our thoughts and opinions rather than
challenging them, fuelling people’s confirmation bias. Moreover, fake news reaches more people than the truth (Vosoughi, Roy
and Aral, 2018).On the other hand, a recent study in the United States shows that TV news dominates online in a ratio of 5:1 and
estimates fake news to be about 1% of overall news consumption (Allen et al., 2020). Another study in the United Kingdom shows
that people interested in politics and those with diverse media diets tend to avoid echo chambers (Dubois and Blank, 2018).
This study claims that a small segment of the population is likely to find themselves in an echo chamber. In fact, these studies
argue that it is more likely to find people choosing not to be informed than people being deceived.
2. The socio-economic profile is measured by the PISA index of economic, social and cultural status (ESCS). A socio-economically
disadvantaged (advantaged) student is a student in the bottom (top) quarter of the ESCS in the relevant country/economy.
3. Students’ responses about the usefulness of different reading strategies were rated by reading experts to create the index of
knowledge of reading strategies for assessing the credibility of sources (see Chapter 16, PISA 2018 Technical Report).
4. This value is calculated as 1 minus the result of dividing the ESCS coefficient after accounting for the indirect effect of knowledge of
reading strategies by the ESCS coefficient for the total effect and then multiplying by 100 (see Figure 5.12, 21st‑Century Readers:
Developing literacy skills in a digital world).
5. Reading proficiency Level 5 is one of the highest and corresponds to students who scored from 625.61 to less than
698.32 score points. For further details on the reading proficiency levels, please see PISA 2018 Report: Volume I.
6. Reading proficiency Level 3 corresponds to students who scored from 480.18 to less than 552.89 score point. For further details
on the reading proficiency levels, please see PISA 2018 Report: Volume I.
7. Rapa Nui Question 3 is a partial credit item where non-credit is scored 0, partial credit is scored 0.5, and full credit is scored 1.
Therefore, the estimated percentage correct for full credit in this item is lower than 47% on average across OECD countries.
This item was estimated to be 39% correct on average across all PISA 2018 participating countries and economies. Rapa Nui
Question 3 is a Level 5 item. This means that students need to have a proficiency level 5 to have a 62% probability of getting full
credit in this item (see Figure I.2.1, PISA 2018 Results: Volume I).
8. The partial correlation after accounting for per capita GDP was 0.66 among OECD countries.The partial correlation after
accounting for reading performance was 0.60 among OECD countries. The partial correlation is calculated using the percentage
of students who reported learning in school how to detect whether information is subjective or biased (Table B.2.6, 21st‑Century
Readers: Developing literacy skills in a digital world) and the percentage correct in the reading assessment items to assess the
capacity to distinguish facts from opinions (Table B.2.6, 21st‑Century Readers: Developing literacy skills in a digital world), after
accounting for per capita GDP (see Table B3.1.4, PISA 2018 Results: Volume I) and average reading performance (Table B.2.1a).
9. Countries that administered the paper-based form had no available data to perform this analysis: Argentina, Jordan, Lebanon,
the Republic of Moldova, the Republic of North Macedonia, Romania, Saudi Arabia, Ukraine and Viet Nam.
© OECD 2021 PISA in Focus 2021/113 (May) 7For more information Contact: Javier Suarez-Alvarez (Javier.SUAREZ-ALVAREZ@oecd.org) This paper is published under the responsibility of the Secretary-General of the OECD. The opinions expressed and the arguments employed herein do not necessarily reflect the official views of OECD member countries. This document, as well as any data and map included herein, are without prejudice to the status of or sovereignty over any territory, to the delimitation of international frontiers and boundaries and to the name of any territory, city or area. The statistical data for Israel are supplied by and under the responsibility of the relevant Israeli authorities. The use of such data by the OECD is without prejudice to the status of the Golan Heights, East Jerusalem and Israeli settlements in the West Bank under the terms of international law. This work is available under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 IGO (CC BY-NC-SA 3.0 IGO). For specific information regarding the scope and terms of the licence as well as possible commercial use of this work or the use of PISA data please consult Terms and Conditions on www.oecd.org.
You can also read
