Coronavirus News and Information on YouTube: A Content Analysis of Popular Search Terms - The ...
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
COVID-19 Series COMPROP Data Memo 2020.3, April 17 2020
Coronavirus News and Information on YouTube:
A Content Analysis of Popular Search Terms
Nahema Marchal, Hubert Au, Philip N. Howard
SUMMARY
Social media is reshaping how people access information about the coronavirus crisis. The online video-sharing website
YouTube has emerged as a major purveyor of health and wellbeing information. In this memo, we examine the nature
and structural properties of a sample of 320 YouTube videos related to the coronavirus outbreak. We find that:
• four-fifths of the channels sharing coronavirus news and information are maintained by professional news
outlets and that the channels of public health agencies are rarely, if ever, returned with search results;
• searches for popular coronavirus-related terms return mostly factual and neutral video results, with low volumes
of conspiratorial or junk science video results;
• highly politicized health news and information receives on average more public engagement in the form of
comments than any other type of videos.
the coronavirus’s origin, transmission, and cure being
INTRODUCTION politicized, and how much of it is factually inaccurate,
The recent outbreak of the coronavirus disease, known misleading, or conspiratorial? How much public
as COVID-19, is an ongoing public health emergency engagement is received by different types of videos?
that first emerged in Wuhan, China in December 2019.
As of April 17th, 2020, nearly 2,113,230 cases of
infections have been recorded in a total of 210 countries DATA AND METHOD
worldwide.[1] Since the outbreak’s onset, misinformation The goal of this study is to provide a snapshot of the
about the virus, its provenance and scale have type of content average users are likely to come across
proliferated online.[2] On February 2nd, the World Health when searching for information about the coronavirus
Organization (WHO) director warned against the public on YouTube. YouTube is one of the largest search
health consequences of an “infodemic”—an abundance engines in the world by search volume, second only to
of potentially inaccurate claims—that would make it Google, and a gateway to the news for a large number
difficult for citizens to find reliable guidance when of its users. Like many other social media platforms,
needed.[3] YouTube is powered by a search algorithm and
recommender system based on the principle of
With over 2 billion monthly active users, the online “collaborative filtering,” designed to help users navigate
video-sharing website YouTube has emerged as a the millions of pieces of content available on its site.[7]
major source of information about science, technology Video creators also actively attempt to optimize their
and health in recent years, especially for young content for search engines.[8] Research suggests that
people.[4] In the UK alone, an estimated 35.6 million YouTube users rarely scroll past the top twenty results
people visit the site each month, making it the most when searching for content and that recommendations
popular online video platform in the country. The are responsible for 70% of time spent on the site. [9],[10]
average UK adult spends around half an hour on In an effort to replicate this user experience, for our
YouTube every day and screen time has been analysis we thus focus on the first twenty video results
increasing during quarantine.[5],[6] returned for a search query, as well as what YouTube
calls the most frequent “related” videos displayed in the
In the context of a major public health crisis such as the sidebar (see Online Supplement for full methods
2019 coronavirus outbreak, it is therefore important to specification).
understand the type and quality of content that users
are likely to encounter when searching for information For this study, we perform a content analysis of the top
about the virus on YouTube. In this memo, we ask: English-language videos associated with four popular
Which sources and channels are most represented in coronavirus-related search terms in the UK:
search results? To what extent is video content around “coronavirus UK”, “coronavirus China”, “coronavirus
1COMPROP Data Memo 2020.3 COVID-19 Series April 17 2020
symptoms” and “coronavirus conspiracy”. We analyze • Professional News. Channels of established
the top twenty videos results provided by YouTube in news organizations, broadcasters, digital and
order of “relevance”, as well as the top sixty related print media outlets.
videos for each search term. Our final sample thus • State-Backed Media. Channels of media
consists of 320 videos, eighty per search term. organizations that are either directly funded by
the state and are editorially controlled by their
Using Google Trends, our team first collected a list of respective governments.
the ten most popular search queries entered into the
YouTube search bar in the United Kingdom related to Each video in our sample was then reviewed in-depth
“coronavirus” since January 2020. For comparison, we and classified into one of the four categories below. A
also compiled a list of auto-complete suggestions for the heuristic approach was taken to content classification
term “coronavirus” in the YouTube GB search bar, using based on the type of evidence marshalled, degree of
a Google Chrome browser in Incognito Mode (see politicization and factual accuracy of the information
Online Supplement for complete list). The term presented:
“coronavirus” was selected over “COVID-19” as it was
by far the most popular search term at the time of data • Factual and Neutral. Factual and high-quality
collection. Four search terms that overlapped between reporting on the coronavirus pandemic. This
both approaches—namely “coronavirus UK”, includes factoids, real-time case trackers and
“coronavirus China”, “coronavirus symptoms”, and news reports from professional news
“coronavirus conspiracy” —were selected for the final organizations and public health agencies.
analysis, as these cover a wide range of topics • Junk and Conspiratorial. Videos relaying
reflecting strong public interest. verifiably false information or conspiracy
theories about the origin, transmission and
Next, we queried the YouTube API’s search function treatment of the coronavirus; trutherism;
with our four search terms using YouTube Data Tools xenophobia and denial of mainstream scientific
(YTDT) on March 20th, 2020. This generated a list of the positions as assessed against WHO public
top fifty video results for each search term, ranked by advisory information.
YouTube in order of “relevance”—the default setting for • Personal and Investigative. Discussion of the
this parameter.[11] This seed list was then used to further coronavirus pandemic from a personal,
crawl the YouTube network and collect all related testimonial or investigative point of view. This
videos associated with these top fifty results. Using this includes videos of patients describing their
technique, we gathered a total of 8,912 videos. symptoms, testimonies, recommendations and
Following previous research in this area, our team debunking efforts from independent vloggers
created four directed network graphs, where each node and health practitioners, as well as undercover
in the graph corresponded to a video and each edge investigations and talk shows.
denoted a connection from a seed video to a related • Political. Videos in which the coronavirus
video.[12] We then calculated corresponding network pandemic, its provenance, spread as well as
statistics to determine which related videos in each the efficacy of various government responses,
network were most frequently suggested in the side bar. is discussed or debated from a political
Top related videos were filtered by highest in-degree— perspective. This includes political comedy
the number of times a video was featured on the related shows, debates, podcasts of a political nature,
list of another video. Our final sample consists of 320 and ideologically motivated fact-checks.
videos: the twenty top video results and sixty top related • Not Applicable (N/A). Videos unrelated to the
videos associated with each of four search terms. coronavirus pandemic.
First, we classified all channels in this sample, All videos and associated channels in our sample were
according to the following grounded typology: manually classified by a team of two coders who
achieved high inter-rater reliability following two rounds
• Government & Public Agencies. Official of rigorous training and test coding (Krippendorf’s a
channels of government agencies and =0.803 for content codes). Any disagreements were
international bodies, such as the US White resolved by the authors. A further thematic analysis was
House or United Nations. conducted to determine the main topics covered in each
• Independent Content Creator. Channels of video (see Online Supplement).
independent content creators, such as
independent vloggers, media commentators,
health professionals and educators. FINDINGS
• Professional Health. Channels of domestic
and intergovernmental public health agencies, Content and channel types
hospitals and professional health websites, We found that the vast majority of channels were
such as the WHO, the NHS, or WebMD. maintained by professional news organizations (80%)
followed by independent content creators (16%), which
2COMPROP Data Memo 2020.3 COVID-19 Series April 17 2020
include independent healthcare providers, educators, Table 1. Distribution of top 20 video results, per search
and analysts. By contrast, professional health sites and query (Percent)
public health agencies such as the NHS and the WHO Search Terms
Type of Content
constituted less than a single percentage point in this % (N)
sample (0.3%). A small but non-negligible proportion of UK China Symptoms Conspiracy
the videos analyzed by our team originated from state- Factual & 80 45 25 20
backed media outlets (2.2%) including China’s CGTN neutral (16) (9) (5) (4)
and GBTimes, as well as Russia Today’s Ruptly. These Junk & 15 5
conspiratorial (3) (1)
comprised 6.25% of top and most recommended videos
Personal & 15 50 40
for the query “coronavirus conspiracy” (see Table 1 in investigative (3) (10) (8)
Online Supplement). Political 5 35 10 30
(1) (7) (2) (6)
Table 1 shows that factual and balanced reporting Non-English 5 15 5
dominates video results when users search for (1) (3) (1)
Total 100 100 100 100
“coronavirus UK” — fully 80% of top twenty results
(20) (20) (20) (20)
returned are factual or neutral. The picture is more
Source: Authors’ calculations based on data collected on
mixed when searching for videos related to coronavirus 20/03/2020.
conspiracy theories, with about one third of the first Note: Categories are mutually exclusive and columns sum to
twenty videos displayed on the results page consisting 100%.
of politicized content or ideologically-motivated
debunking efforts. Interestingly, half of the top video Table 2. Distribution of top 60 most recommended
results shown for “coronavirus symptoms” consist of videos, per search query (Percent)
explainers and professional advice delivered by Type of Search Terms
Content % (N)
independent vloggers and healthcare professional,
UK China Symptoms Conspiracy
testimonial evidence, such as talk shows featuring
Factual & 57 45 47 40
patients describing their symptoms. Videos peddling neutral (34) (27) (28) (24)
junk, factually inaccurate or conspiratorial information Junk & 5
are low across all searches, except for “coronavirus conspiratorial (3)
China,” where they make up 15% of the top video Personal & 17 13 17 22
results displayed. investigative (10) (8) (10) (13)
18 28 15 15
Political
(11) (17) (9) (9)
Trends are broadly similar for related video content (see 2 8 13 7
Table 2). Among the sixty top related videos per search Non-English
(1) (5) (8) (4)
term, only three re-directed to questionable content that 7 5 8 12
N/A
runs against public advisory information from the WHO. (4) (3) (5) (7)
Investigative reporting and informational videos 100 100 100 100
Total
(60) (60) (60) (60)
showcasing personal narratives and testimonial
Source: Authors’ calculations based on data collected on
evidence account for a large proportion of the top 20/03/2020.
related videos in the network—22% for “coronavirus Note: Categories are mutually exclusive and numbers are
conspiracy” and 17% for “coronavirus symptoms” rounded up to the nearest whole number.
respectively. A user searching for coronavirus news and
information related to China, however, is more likely to Table 3. Average engagement with each type of
come across politicized content than through any other video
search query tested here. Comment-
Avg. Avg. number
to-views
Type of number of
Having classified all videos in our sample, we examined Content of views comments*
ratio
the distribution of video content per channel type. Figure (1,000
(million)
:1,000,000)
1 suggests that professional news channels drive most
Factual & 4.68 3,128 0.67
of the factual and journalistic reporting on the neutral (43.8%)
coronavirus pandemic and that these predominantly fall
under the “News & Politics” label. Public health Junk & 0.33 2,915 8.89
agencies and professional healthcare are not conspiratorial
represented in this figure as the only channel our team (2.2%)
coded as such linked to a non-English language video. Personal & 2.46 5,852 2.38
The Online Supplement presents this data in tabular investigative
form. Politicized content is evenly split between (19.1%)
professional news channels, on the one hand, and Political 2.42 8,923 3.68
independent content creators as well as state-backed (21.3%)
media on the other hand, covering a wide variety of Source: Authors’ calculations based on data collected on
channel types, from “Comedy” to “Music”, “Style” and 20/03/2020. *Note: Comments may be disabled on some
even “Travel” (See Figure 2 in Online Supplement). YouTube videos by the channel owner.
3COMPROP Data Memo 2020.3 COVID-19 Series April 17 2020
Figure 1. Distribution of content by channel type
Source: Authors’ calculations based on data collected on 20/03/2020.
Note: Figure omits non-applicable and non-English language content
above, we can approximate what an average user
Public engagement would see on YouTube when using these search terms.
Average view and comment counts are useful, but
People search for health information on YouTube for a
imperfect, indicators of video popularity amongst
number of psychological and social reasons.[13]
YouTube users. Across the board, videos classified by
Anxieties surrounding one’s own health, current health
our team as relaying high-quality factual information
status, and ability to use the internet, for example, all
accumulate more views than all other content
shape search habits.[14] In practice, there is great variety
categories, averaging 4.7 million per video, and about
in search skills, internet access, and people search for
fourteen times as many views as junk and conspiratorial
health related information in different ways. Women,
videos (see Table 3). Political and personal or
young people, individuals with advanced or college
investigative videos had both received an average of
degree, and those living in higher income households
2.4 million views at the time of data collection.
tend to pro-actively seek out health news and
Politicized content is by far the most commented on,
information online.[15]
averaging almost 9 thousand comments per clip,
compared to just over 3 thousand comments on
Finally, it is worth noting that in March 2020 YouTube
average for factual content. Conspiratorial and junk
shifted between demonetizing and monetizing
content, however, has a higher comment to views ratio
coronavirus-related content for creators under its
than any other category—around 9 thousand comments
“sensitive events” policy, which may have caused
per million views.
omissions from our sample.[16],[17] It is also possible that
some of the prominent creators in our sample extended
CONCLUSION the reach of their content through YouTube advertising
It is important to acknowledge the limitations of this before those restrictions were put in place, which would
study. Given YouTube’s API restrictions and the have improved their performance and discoverability in
proprietary nature of its search algorithm and organic search.
recommender system, we are limited in the kind of
claims we are able to make about the Our study sought to determine the types and
representativeness of our sample. Notably, the video informational quality of video content returned by
sample analyzed in this memo does not account for the YouTube for different search queries related to
personalization and localization of YouTube search coronavirus. We find that: 1) four-fifths of the channels
rankings. Nevertheless, following the steps outlined sharing coronavirus news and information are
maintained by professional news outlets and that the
4COMPROP Data Memo 2020.3 COVID-19 Series April 17 2020
channels of public health agencies are rarely, if ever, or junk science video results; 3) highly politicized health
returned with search results; 2) searches for popular news and information receives on average more public
coronavirus-related terms return mostly factual and engagement in the form of comments than any other
neutral video results, with low volumes of conspiratorial type of videos.
REFERENCES
[1] COVID-19 Map’, Johns Hopkins Coronavirus Resource Center, 2020. https://coronavirus.jhu.edu/map.html.
[2] Institute for Strategic Dialogue, ‘COVID-19 Disinformation Briefing No.1’, Institute for Strategic Dialogue. [Online].
Available: https://www.isdglobal.org/isd-publications/covid-19-disinformation-briefing-no-1/.
[3] WHO, ‘Novel Coronavirus (2019-nCoV) situation reports’. Accessed: Apr. 01, 2020. [Online]. Available:
https://www.who.int/emergencies/diseases/novel-coronavirus-2019/situation-reports.
[4] M. Anderson, and J. Jiang, ‘Teens, Social Media & Technology 2018’, Pew Research Center: Internet, Science & Tech,
May 31, 2018. https://www.pewresearch.org/internet/2018/05/31/teens-social-media-technology-2018/.
[5] Ofcom, ‘Media Nations 2019’, Ofcom, 2019. [Online]. Available: https://www.ofcom.org.uk/research-and-data/tv-radio-
and-on-demand/media-nations-2019.
[6] A. Przybylski and P. Etchells, ‘Don’t Freak Out About Quarantine Screen Time’, The New York Times, Apr. 06, 2020.
[7] M. Bendersky, L. Garcia-Pueyo, J. Harmsen, V. Josifovski, and D. Lepikhin, ‘Up next: retrieval methods for large scale
related video suggestion’, in Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery
and data mining, New York, New York, USA, Aug. 2014, pp. 1769–1778, doi: 10.1145/2623330.2623344.
[8] S. Bradshaw and P. N. Howard, ‘Why Does Junk News Spread So Quickly Across Social Media? — Oxford Internet
Institute’, Knight Foundation, Miami, 2018. Accessed: Jun. 12, 2019. [Online]. Available:
https://www.oii.ox.ac.uk/blog/why-does-junk-news-spread-so-quickly-across-social-media/.
[9] V. N. Gudivada, D. Rao, and J. Paris, ‘Understanding Search-Engine Optimization’, Computer, vol. 48, no. 10, pp. 43–
52, Oct. 2015, doi: 10.1109/MC.2015.297.
[10] A. Rodriguez, ‘YouTube’s recommendations drive 70% of what we watch’, Quartz, Jan. 13, 2018.
[11] ‘YouTube Data API Documentation’, Google Developers, 2020. https://developers.google.com/youtube/v3/getting-
started.
[12] X. Cheng, J. Liu, and C. Dale, ‘Understanding the Characteristics of Internet Short Video Sharing: A YouTube-Based
Measurement Study’, IEEE Trans. Multimed., vol. 15, no. 5, pp. 1184–1194, Aug. 2013, doi:
10.1109/TMM.2013.2265531.
[13] A. Mills and N. Todorova, ‘An Integrated Perspective on Factors Influencing Online Health-Information Seeking
Behaviours’, ACIS 2016 Proc., Jan. 2016, [Online]. Available: https://aisel.aisnet.org/acis2016/83.
[14] C. Lagoe and D. Atkin, ‘Health Anxiety in the Digital age: An Exploration of Psychological Determinants of Online Health
Information Seeking’, Comput. Hum. Behav., vol. 52, pp. 484–491, 2015, doi: 10.1016/j.chb.2015.06.003.
[15] M. Duggan, ‘Health Online 2013’, Pew Research Center: Internet, Science & Tech, Jan. 15, 2013.
https://www.pewresearch.org/internet/2013/01/15/health-online-2013/ (accessed Mar. 31, 2020).
[16] S. Perez, ‘YouTube will now allow creators to monetize videos about coronavirus and COVID-19’, TechCrunch, Mar.
11, 2020.
[17] J. Alexander, ‘YouTube is demonetizing videos about coronavirus, and creators are mad’, The Verge, Mar. 04, 2020.
ACKNOWLEGMENTS
The authors gratefully acknowledge the support of the European Research Council for the project ‘Computational
Propaganda: Investigating the Impact of Algorithms and Bots on Political Discourse in Europe’, Proposal 648311, 2015–
2020, Philip N. Howard, Principal Investigator. Project activities were approved by the University of Oxford’s Research
Ethics Committee, CUREC OII C1A 15-044. We are also grateful to the Adessium, Luminate, and Ford Foundation for
supporting this work. Any opinions, findings, conclusions or recommendations expressed in this material are those of
the authors and do not necessarily reflect the views of the University of Oxford or our funders. We are grateful to Kate
Joynes-Burgess, Lisa-Maria Neudert and Dr. Vidya Narayanan for their contributions to this memo.
ABOUT THE PROJECT
The Computational Propaganda Project (COMPROP), which is based at the Oxford Internet Institute, University of
Oxford, involves an interdisciplinary team of social and information scientists researching how political actors manipulate
public opinion over social networks. This work includes analyzing how the interaction of algorithms, automation, politics,
and social media amplifies or represses political content, disinformation, hate speech, and junk news. Data memos
integrate important trends identified during analyses of current events with basic data visualizations, and although they
reflect methodological experience and considered analysis, they have not been peer reviewed. Working papers present
deeper analysis and extended arguments that have been collegially reviewed and engage with public issues.
COMPROP’s articles, book chapters, and books are significant manuscripts that have been through peer review and
formally published.
5You can also read
