Disclaimer: this study is disputed by fact-checkers
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
“Disclaimer: this study is disputed by fact-checkers”
The influence of disclaimers on the perceived credibility of information and
disinformation in social media posts
Loubna Bouzit
Snr: 2045227
Master’s Thesis
Communication and Information Sciences
Specialization Business Communication and Digital Media
School of Humanities and Digital Sciences
Tilburg University, Tilburg
Supervisor: Dr. R. Cozijn
Second Reader: Dr. F. Folkvord
January 2021
Abstract
With the increase of social media, a lot of misleading information is being spread. People do
not know what to believe anymore. The spread of misleading information is going on for
years now, but draws currently more attention with regards to COVID-19 (Statista, 2020).
Social media platforms have recently responded to this spread of disinformation by starting
to use disclaimers (Chowdry, 2017). Disclaimers are short statements in which someone or
an organization rejects or limits his or her liability about a certain matter (Hewitt & Stokes,
1975). In order to find out the potential effects of these disclaimers on social media posts
with information and disinformation, a survey study is conducted with the within-
participants independent variable Content (information/disinformation) and the
between-participants variable Disclaimer (with/without). The results of this study lead to the
conclusion that one can see the difference between the posts of information and
disinformation and that disclaimers probably have effect on the perceived credibility of
posts with disinformation. However, the differences were not large and not significant, so
further research is required to see the real possible effects of disclaimers on posts with
disinformation described in the Conclusion and discussion section. These findings suggest
that there needs to be more focus on which type of disclaimer can be used and how other
stimuli can provide more insight into the effectiveness of a disclaimer.
2Table of Contents
1. Introduction ............................................................................................................................................ 4
2. Theoretical Framework ........................................................................................................................... 7
2.1 Information, misinformation and disinformation ...................................................................................... 7
2.2 Measures of disinformation ........................................................................................................................ 9
2.3 Credibility ................................................................................................................................................... 10
2.4 Factors that influence the perceived credibility ........................................................................................ 11
2.5 Perceived Credibility Measurements......................................................................................................... 13
2.6 Disclaimers ................................................................................................................................................. 14
2.7 Types of disclaimers ................................................................................................................................... 15
3. Method ................................................................................................................................................. 17
3.1 Design......................................................................................................................................................... 17
3.2 Participants ................................................................................................................................................ 18
3.3 Materials .................................................................................................................................................... 19
3.4 Instruments ................................................................................................................................................ 20
3.5 Procedure ................................................................................................................................................... 21
3.6 Data analysis ............................................................................................................................................. 22
4. Results .................................................................................................................................................. 23
4.1 Perceived Source Credibility ...................................................................................................................... 23
4.2 Perceived Message Credibility ................................................................................................................... 25
4.3 Overall Perceived Credibility ..................................................................................................................... 26
4.4. Behavioral questions ................................................................................................................................ 29
5. Conclusion and discussion ..................................................................................................................... 30
References ..................................................................................................................................................... 36
Appendices .................................................................................................................................................... 40
Appendix A ....................................................................................................................................................... 40
Results pre-test for the posts of information and disinformation .................................................................... 40
Appendix B ....................................................................................................................................................... 41
Measurement scale of perceived source credibility and message credibility ................................................... 41
31. Introduction
“Taking a hot bath does prevent COVID-19.”, “The coronavirus is transmitted through the 5G
network” (WHO, 2020). Fake or real? The rise of new media has increased communication
between people worldwide. Nowadays with the increase of social media, a lot of misleading
information is being spread. People do not know what to believe anymore. The spread of
misleading information is going on for years now, but draws currently more attention with
regards to COVID-19 (Statista, 2020). Besides the COVID-19 pandemic, there is also an
“infodemic”. The World Health Organization (WHO) describes infodemic as “an
overabundance of information – some accurate and some not – that is occurring during a
pandemic” (WHO, 2020).
With the popularity of online communication technologies came the common use of
social media platforms. These platforms allow people to share the information they want.
Although a lot of correct and reliable information is being shared, there is also a lot of
misleading information that is deliberately shared to confuse or deceive people, which refers
to the phenomenon of disinformation (Billiet et al., 2018). During the time of COVID-19, it
turned out that 60% of the population use social media as a source of information regarding
the pandemic (Statista, 2020). It turned out that 59% of this group came across
disinformation (Statista, 2020). This is why the authorities want to take protective measures
to reduce or combat disinformation.
The European Commission already implemented several actions to tackle this issue,
i.e. the Code of Practice on Disinformation (European Commission, 2020). In collaboration
with the largest social media platforms such as Facebook and Twitter, among others, the
European Commission is trying to ensure transparency and reliable information through this
code of conduct.
4Facebook has blocked 1.7 billion accounts as a result at the beginning of this year (Statista,
2020). With regards to COVID-19, tens of thousands of accounts were removed. However, a
great deal of misleading information remains available on different platforms. Three-
quarters of the Dutch population thinks that technology- and (social) media industries
should do more to combat disinformation (Mediamonitor, 2018). Social media platforms
have recently responded to this spread of disinformation by starting to use disclaimers
(Chowdry, 2017).
Disclaimers are short statements in which someone or an organization rejects or
limits his or her liability about a certain matter (Hewitt & Stokes, 1975). Disclaimers can be
sorted into different types. When it is about sharing (dis)information and someone is
disclaiming that the shared content is not based on facts and is not an expert, it is called
“hedging” in social context (Hewitt & Stokes, 1975). According to a significant amount of
research on the use of disclaimers in advertising, using disclaimers does not necessarily
directly give the desired effect (Lewis, Pelled, & Tal-Or 2019; Green & Armstrong 2020).
However, a recent study showed that the use of a disclaimer through warning labels on
social media does affect the perceived credibility of disinformation (Mena, 2020).
Research done by Metzger et al. (2003) noted that the perceived credibility of online
(news) messages can be examined on three elements: source credibility, message credibility,
and medium credibility. Flanagin and Metzger (2007) added that credibility of online content
depends on the attributes of a website such as design, depth of content, and the complexity
of a website. It might be interesting for social media platforms to understand the differences
in (dis)information towards the use of a disclaimer and perceived credibility.
Up to now, only little research has been done with regards to the influence of
disclaimers on the perceived credibility of (dis)information on social media. Therefore, this
5study is designed to see whether the level of perceived credibility of online (dis)information
is influenced by the use of a disclaimer. The research question that is formulated to answer
is as follows:
RQ: What is the influence of a disclaimer on the perceived credibility of a social media
post with information and disinformation?
To gain a better understanding, the most important variables, information, disinformation,
level of perceived credibility, and use of disclaimers are further explained.
62. Theoretical Framework
2.1 Information, misinformation and disinformation
There are many definitions of the term information. Madden (2000, p.343) describes
information as “An item of information or intelligence; a fact or circumstance of which one is
told.”. A study by Pinkster and Bruin (2007) explains in detail how information arises and can
result in a competence, which is referred to as an "information ladder". It all starts with
facts, based on events and circumstances. For example, today it is -5 degrees outside. This
event or fact becomes data after registration. If this has meaning, depending on the
information recipient, then this information becomes knowledge. When information
becomes knowledge, depending on your own knowledge and skills, this knowledge can
become a new competence, which is a combination of own skills and attitude. Moreover, a
study by Satija (2013) adds that information adds value to the daily lives of individuals, it
contributes to important actions and changes in life within the social, political, and economic
domain.
Whereas in the past the libraries were visited physically to retrieve information, they
now are visited digitally. Already for some time, we are in the age of digital information
(Pinkster & Bruins, 2007). This has its advantages as well as its disadvantages. Information
can be presented in various ways, for example information in marketing, information in
management etc. This present study discusses online information in the field of
communication on social media. Research by Nisar, Prabhakarb and Strakovaa (2019)
reported the benefits when it comes to information shared on social media. For instance,
there is a high speed of information exchange: content can be shared in just a fraction of a
second. In addition, information can be distributed to a large group of people, depending on
the number of followers someone has, of course, and on the interactivity between people.
7Agarwal and Yiliyasi (2010) also mentioned some benefits of sharing information via social
media. They noted that information shared via social media offers freedom of speech and
expression. People post their everyday lives, memories, opinions, and much more that is
viewed and shared very fast by others. However, the study by Agarwal and Yiliyasi (2010)
also questions the quality of shared content on social media. The open accessibility, the low
barriers to publication, and the user-friendly interactive interfaces pose several issues with
respect to information quality on social media, making obtaining timely, accurate, and
relevant information a challenge. Social media makes it too easy for individuals to share
information online, which introduces the problem of the spread of misinformation and
disinformation. The main difference between the three concepts of information,
misinformation, and disinformation is the issue of truth. If information is defined as true,
then misinformation and disinformation are defined as not true. The difference between
misinformation and disinformation is that in case of misinformation, inaccurate information
is disseminated, even though the user believes this information to be correct, and in case of
disinformation, misleading information is being spread on purpose. The information is
disclosed publicly to influence other people's opinions. In short, the difference is that
misinformation is created by mistake and disinformation is released intentionally (see, e.g.,
Billiet et al., 2018; Stahl, 2006).
In the last few years, there is a common concern about the spread of false
information, since it has a major impact on society (Allcott, Gentzkow, & Yu, 2019). The
spread of disinformation on social media is becoming more and more common. It has
created many misconceptions that have influenced the decision-making processes in many
areas, including health, economics, and politics. For example, in 2016 during the US election,
various news sites and social media platform such as Facebook reported posts stating that
8pope Franciscus supported Trump's presidency (Faris, et al., 2017). This was false
information, deliberately shared for political gain to mislead others into voting for Trump. It
is therefore important to know how disinformation can be recognized by people. The main
difference between legitimate information and disinformation is how the message is
formulated. Whereas legitimate information is mostly objective, disinformation is subjective
(Campelo et al., 2019). This is mostly how people can make a distinction between
information and disinformation. However, in the last years it can be found hard for people to
see the difference, that is why several measures were taken to tackle disinformation by
authorities and social media platforms.
2.2 Measures of disinformation
In response to the spread of online disinformation, social media platforms have
announced several actions to limit the spread of disinformation (Bradshaw, Hoffmann, &
Taylor, 2019). In 2018, the European Commission published a code of conduct in
collaboration with social media platforms such as Facebook, Twitter, and Microsoft to
actively combat disinformation (Hins, 2018). As a result, these platforms created a strategy
to battle disinformation with factcheckers, algorithmic detectors, and artificial intelligence
(Faris, et al., 2017). Due to the foundation of freedom of speech, the government mainly
leaves it to self-regulation of the social media platforms to try to combat disinformation (Til,
2019). Research done by Allcott, Gentzkow and Yu, (2019) showed that these measures did
help and decreased disinformation on Facebook. However, it is still a challenge for these
social media platforms to detect all disinformation. With the emergence of disinformation
related to COVID-19, since the web has become an important source of health information
for users around the world, the spread of online disinformation has increased to necessity of
9conducting more research. Several studies that were recently conducted with regards to
disinformation and COVID-19 have found that a lot of online disinformation was shared by
people and because of ignorance, people tend to believe what is shared online (Cuan-
Baltazar et al., 2020; Brennen, 2020). In short, people cannot distinguish information from
disinformation and no longer know what is true and what is false. This hurts the credibility of
online content, which will be discussed in the next paragraph.
2.3 Credibility
Research by West related to information and communication defines credibility as "the
qualities of an information source which cause what it says to be believable beyond any
proof of its contentions" (1994, p. 159). The accessibility of online information has led to an
increased influence of social media posts, and therefore knowing what makes a social media
post credible is a valued addition to the online communication literature. There are several
studies that define credibility based on a number of factors. It is generally accepted that two
factors determine source credibility: expertise and trustworthiness. Ohanian (1991) defines
expertise as the knowledge and experience of the messenger, and trustworthiness as the
level of confidence someone has in the published message. These two factors are important
elements to determine the level of perceived source credibility.
Research done by Westerman, Spence, and Heide (2014) examined how published
information in social media influences the perception of the credibility of the source by
conducting an experiment with Twitter as a medium. This study showed that the recency of
a social media post affects the perceived credibility of the source. A study by Flanagin and
Metzger (2007), that conducted a content analysis stated that credibility of web-based
content depends on the attributes of a website such as design features, depth of content,
10and the complexity of a website. Moreover, news websites were assessed with a high level
of credibility and personal websites with a lower level of credibility. On the contrary, when it
comes to sharing an online review about an experience of a specific product, research done
by Park, Lee, and Han (2007) proposed that the perceived credibility is higher and more
valued when the information is obtained from a personal page or website. This shows that
information is more valued when it comes from someone who has experience and
knowledge about a certain matter and, therefore, the information can be trusted. Li and Suh
(2015) have done research on the evaluation of information credibility specific on the social
media platform Facebook, and add that interactivity, medium dependency, and argument
strength are the main determinants of one’s perceived credibility. Looking into these
findings, it appears that the perceived credibility is mainly determined by the person who
wrote the message and also the content of the message, how the message is written.
Therefore, we see again the importance of these two factors that were mentioned before
which are expertise and trustworthiness of the source. In the next paragraph, it will be more
clear about what the important indicators are that influence the perceived credibility.
2.4 Factors that influence the perceived credibility
Miriam Metzger has conducted several studies regarding credibility since 2000. According to
Metzger et al. (2003), the perceived credibility research is started by the interest in the role
it plays in the persuasion process to convince one another. There are various studies that
propose different theories to explain the important factors of the perceived credibility. To
start with, the study by Metzger et al. (2003), noted that the perceived credibility of online
(news) messages can be examined based on three factors: source credibility (which is
already mentioned in the previous paragraph), message credibility and media credibility.
11Perceived Source credibility
Source credibility is, according to Metzger et al. (2003), a concept that stands for the
judgment of the recipient about the credibility of the information sender.
Metzger et al. (2003), refer to the ability of a sender to tell the truth about a subject. The
expertise and trustworthiness are perceived by the receiver by looking at the speaker's
motivation to tell the truth. For example, an expert in the field of sustainability is considered
more credible when he or she speaks about his or her own research into the effects of
plastic use than when he or she makes statements that have nothing to do with the
expertise. That expertise and trustworthiness affect source credibility has been
substantiated by Reichelt, Sievert, and Jacob (2013). They studied the influence of perceived
credibility in the field of Electronic Word of Mouth and mentioned the importance of the
expertise and trustworthiness of information channels, meaning that consumers depend
their buying decision on their perceived source credibility.
Perceived Message credibility
Metzger et al. (2003) noted that message credibility influences the perception of a message
within a news article and determines if the content is more or less credible. The researchers
identified four factors that influence perception: structure, content, language use, and
delivery. The message structure involves the way a message is presented and organized, the
message content involves how well and detailed the message is written. The language
involves the objectivity of shared information, and message delivery involves the style of
how a message is presented. Metzger et al. (2003) note that these factors of message
credibility provide insight into both the credibility of the source and of the message itself
12(Metzger et al., 2003). This shows that there is an overlap between source credibility and
message credibility.
Perceived Medium credibility
According to Metzger et al. (2003), medium credibility is a concept that stands for the
perceived credibility of the medium channels that broadcasters use to present messages. For
example the visibility of the same message via telephones, television, magazines,
newspapers and social media. Research by Paulussen and Harder (2014) showed that the
perceived credibility of social media sources (e.g. Facebook or Twitter) is not high when it
comes to journalism, however, they are used a lot as a reference to confirm news that is
noted in newspapers.
In response to these findings, next paragraph seeks to focus on how the perceived credibility
can be measured, in terms of which measurement tool would be suitable with regards to
information and disinformation on social media platforms.
2.5 Perceived Credibility Measurements
There are various scales that have been used to evaluate the perceived credibility. Research
by Lock and Seele (2017) assessed the perceived credibility within the field of corporate
social responsibility communication. The perceived credibility is determined by four factors:
truth, sincerity, appropriateness, and understanding (Lock & Seele, 2017). When it comes to
assessing blog credibility, Lidy and Rubin (2006) have mentioned other important elements
such as expertise, trustworthiness, information quality, and personal triggers. For the
present study it is important to apply a measurement that fits well within the
communication field of social media. Research by Li and Suh (2015) created a scale specific
13for assessing the perceived credibility on social media platforms, such as Facebook. This
scale consisted of 27 items that were divided into four factors which were medium
credibility, message credibility, expertise, and information credibility. As a potential
measurement to measure the perceived credibility, however, this scale contains items that
are not necessarily all relevant to the present study concerning the credibility of information
and disinformation, and is therefore not used. A more usable scale comes from Kang (2010),
who created a scale with regards to blog credibility that measured the perceived credibility
by factors as the source and message credibility with 14 items. This would be more relevant
for this present study since it involves the perceived credibility of the messenger and the
content of the message, important elements when it comes to information shared via social
media. In response to these findings, the present study seeks to extend the ideas to focus on
how the perceived credibility of online social media posts containing information and
disinformation can be improved with the use of a disclaimer.
2.6 Disclaimers
Hewit and Stokes (1975, p.1) defines the disclaimer as: “an interactional tactic employed by
actors faced with upcoming events or acts which threaten to disrupt emergent meanings or
discredit cathected situational identities”. A disclaimer is a statement that is often included
on a page or in a message for which an organization or a person tries to arouse certain
emotions and limit or exclude their liability (Hewitt & Stokes, 1975). A majority of studies
have pointed out to the use of disclaimers on social media when it comes to the ideal body
image, these studies explain when it comes to women's self-image, insecurities can be
aroused because they don't meet the so-called beauty ideal that is created online
(Selmbegović & Chatard, 2015; Frederick et al., 2016). In additon, when it comes to online
14advertising a lot of research has been done on the effects of the use of disclaimers (Lewis,
Pelled, & Tal-Or, 2019). Although some studies have shown that the use of disclaimers has
little direct effects, other studies found positive effects when it comes to disclaimers on
products within the manufacturing domain. Various studies have shown the added value and
importance of these disclaimers on products where individuals carefully read and observe
the warning label (e.g., creating more awareness of alcohol use by warning people about the
negative health consequences), where the desired effects are achieved (Bollard et al, 2016;
Wakefield, Webster, and White, 2008). Within the field of online information and
disinformation, concerning this present study, the use of a disclaimer would be useful in
terms of informing people, by making them aware and thus prevent possible harm of being
misled. However, a study by Laughery and Stanush (1989) showed that disclaimers are
helpful, but only if they are expressed explitly, which refers to the specifity and
characteristics of the disclaimer (e.g., the details and use of symbols). A study by Brown,
Thomas and Tiggeman (2019) within the field of advertising supported the assumptions of
Laughery and Stanush (1989) concerning the differences in effect of different types of
disclaimer labels, and state that using the wrong disclaimer could lead to opposite effects.
Therefore, it is important to use disclaimers correctly in different cases in order to have
desired effects. There are multiple types of disclaimers where each can be used for a specific
situation, explained in the next paragraph.
2.7 Types of disclaimers
Hewit and Stokes (1975) have identified various types of disclaimers that belong within the
social context. To start with, there is “hedging”, a statement that information is shared with
no expertise and should not be taken seriously (e.g., “I am not very experienced, but..”).
15Another type of disclaimer is described as “credentialing”. With this disclaimer, one tries to
defend oneself before making an insult (e.g., “Please do not take it personally, but..”).
Cognitive disclaimers anticipate doubts that can be operated with regard to the speaker's
ability to control the facts of the situation in which he finds himself (e.g., “I know this sounds
crazy, but..”). Lastly, “appeals for the suspension of judgment”, when someone has an
expectation of how others will react and therefore appeal for the suspension of possible
reactions (e.g., “Let me explain it, before you..”). Every disclaimer is therefore used in
advance in a certain situation to prevent an expected response. A study by Brown, Thomas,
and Tiggeman (2019) used four different warning labels for their research into social
comparison on body satisfaction for advertisements. First, the disclaimer label stated
directly that the picture was edited. Second, the consequence label stated that the picture
can bring out sad emotions. Third, there was the information label that stated that the girl
on the picture has underweight, and last was a graphic label depicting a paintbrush. Each
disclaimer showed a different effect in terms of lowering the body dissatisfaction, but it was
noted that showing a warning label had better effects than no use of a warning label at all.
In short, these disclaimers have shown different effects and need to be used accordingly for
better results, where the disclaimer label probably could have the most potential for this
present study. Since this is also the type of disclaimer that is already been used by social
media platforms such as Facebook, it would be relevant to see the effectiveness of this type
of disclaimer.
With regard to this present study of information and disinformation on social media, little or
no information can be found regarding the effectiveness of the use of disclaimers. Given that
social media platforms such as Facebook and Twitter have recently introduced the use of
16disclaimers to verify information and debunk disinformation, it is therefore a logical
consequence to measure the effectiveness of them on the level of perceived credibility.
With the current literature review research, the following hypotheses have been
formulated:
H1: Information on social media is found more credible than disinformation.
H2: Content on social media is less credible with a disclaimer than without a disclaimer.
H3: The influence of a disclaimer is stronger for content with disinformation than with
information.
3. Method
The purpose of this study is to investigate whether there is difference in the perceived
credibility of information and disinformation on social media and if this is influenced by a
disclaimer. To explore this research question, social media posts were presented to
participants in an online survey in which the perceived credibility of the posts were
measured. This study has been approved by the Research Ethics and Data Management
Committee of the Tilburg School of Humanities and Digital Sciences and has been given the
following reference number: REDC 2020.192.
3.1 Design
A 2x2 mixed factorial design is conducted with the within-participants independent variable
Content (information/disinformation) and the between-participants variable Disclaimer
(with/without). Each participant has seen two social media posts, one with information and
one with disinformation. These posts were presented with or without disclaimer. This
17resulted in four lists of posts: one list with disclaimers for both posts, one list without
disclaimer for both posts, and two lists with a disclaimer for only one of the two posts. The
participants have been randomly assigned to one of the four lists. In this way, the order of
presentation of posts and conditions were balanced in the design.
3.2 Participants
Before the data could be analyzed, the results of the survey were first categorized in Excel
and then exported into SPSS. First, there were in total 132 responses recorded, since a list of
responses were not completed, the data of 30 participants were removed from the dataset,
resulting in 102 respondents. So, the experiment has been conducted by 102 participants
(30 men and 72 women). The average age for men was 23.4 years (SD=.38) and for women
an average of 22.8 years (SD=.28). In terms of the level of education attained, 52.5% (N=53)
of the sample completed their Bachelor’s degree, 25.5% (N=26) of the sample completed
their high school, 19.6% (N=20) completed their Master’s degree, and 4% (N=3) completed
their Associate degree. The participants were recruited by convenience sampling, and
subsequently assigned to two of the four experimental conditions. Therefore, each post was
evaluated by around 50 participants and a total of 202 cases were recorded for the
conditions. This study had no limitations in terms of entry characteristics. Therefore, those
who were interested were free to fill out the online survey and participants could choose to
drop out of the survey at any time.
183.3 Materials
Online social media posts from the platform Facebook were created and used as stimuli
varied in content and the use of disclaimers. The content of information and disinformation
of these posts were created based on how the literature says information and
disinformation is recognized within the textual context, meaning that information is
perceived objectively and disinformation subjectively as is mentioned before (Campelo et al.
2019). The social media posts were adjusted to fit the fictional situation of a social media
post concerning COVID-19 information, with regards to the harm of many conspiracy
theories concerning this issue. The social media post containing information was thus about
COVID-19 and was objective formulated, very specific and no use of language that suggested
disinformation. The social media post containing disinformation was subjectively
formulated, a different style of expression where one can assume that it concerns
disinformation. There were only two post presented with regards to COVID-19. Each
participant has been asked to assess the perceived credibility of the two kinds of social
media posts.
Due to the fact that it is difficult in practice to make a distinction between
information and disinformation, a pre-test was carried out for this study before the
experiment. This was done by a short survey, which was presented to 10 participants, highly
educated, in the age group of 18-25. This survey presented three social media posts with
information and three social media posts with disinformation and was assessed with the
question: “On a scale of 1-10, how likely do you think the post shown below is
Informative/Disinformative?” (with 1 representing ‘not at all likely’; 10 ‘extreme likely’).
Based on the results of this short survey in terms of the highest rates, the content material
19for the experiment that is shown in Figure 1 was selected for the social media posts with
information and disinformation. See Appendix A for the results of the pre-test.
Figure 1 The four conditions of the materials: Two posts with information (A and B) and two
posts with disinformation (C and D), without disclaimer (A and C) and with disclaimer (B and
D).
3.4 Instruments
With the use of a 7-point Likert scale by Kang (2010) that measured blog credibility (with 1
representing ‘strongly disagree’; 7 ‘strongly agree’) in the survey, participants were asked to
rate to what extent they found the online social media post, containing information or
disinformation, credible. The survey consisted of two parts of the variable perceived
credibility. As is mentioned before in the literature review part of this study, perceived
20credibility was measured by two elements: source credibility and message credibility (Kang,
2010). An example of the questions concerning the “perceived source credibility” is: ‘To
what extent do you find the source of this message is transparent?’. An example of the
questions concerning the “perceived message credibility” is: ‘To what extent do you think
the content of this message is accurate?’
Both questions were measured via a 7-point Likert scale (1 = ‘Strongly disagree; 7 = Strongly
agree). See Appendix B for the items measuring perceived credibility by Kang (2010). To get
an overview about participants’ profile, demographic questions were asked regarding
gender, age, level of education (e.g. ‘What is your gender?’ and ‘What is your highest level of
education attained?’). In addition, questions regarding social media behavior have been
asked (e.g. ‘How many hours a week do you spend time on social media?)’. To see how
familiar participants are with disinformation and disclaimers and if there is any influence into
the results, these questions have also been asked (e.g. ‘Do you pay attention to disclaimer?’
and ‘Are you aware of the spread of disinformation?’). Adding these questions, helped to
create a better overall picture and conclusion for this research.
3.5 Procedure
The participants were recruited by convenience sampling, by sharing a link containing the
online survey on the researcher’s social media accounts (Facebook, WhatsApp and LinkedIn).
The experiment is conducted in an online setting via an online survey so participants were
not limited by time and space, since that suits the best for this study. After the participants
clicked on the link of the survey, the survey opened with an introductory text in which the
participant could give his or her participation permission. In order to keep the participant
unbiased, the introductory text did not state directly what the real purpose of this study is.
The survey started with demographic questions regarding gender, age, level of education to
21get an overview of the profile of participants. Furthermore, the survey then continues to the
next page with outlining a fictional situation. Therefore, to be able to conduct an experiment
that is as realistic as possible, the fictional situation described a scenario where the
participants were asked to read a case in which they need to imagine being someone who is
scrolling down on their Facebook feed, a regular day, checking any updates and reading
news and then they will come across two social media posts that piques their interest.
The first online social media post was then presented, after which the level of
perceived credibility was measured. The same procedure was then repeated for the second
post. When the second part of the survey was completed, questions regarding social media
use and familiarity with (dis)information and disclaimers were asked. This was asked at the
end of the survey to keep the participant unbiased. Finally, at the end of the survey, the
participants got a debriefing with the aim of the study and were thanked for their
contribution and asked to share the survey with their network. Filling out the survey took 5
minutes on average.
3.6 Data analysis
Reliability analyses were conducted for the perceived source credibility, the perceived
message credibility and the overall perceived credibility. After the items that consisted
negative wordings were recoded, the perceived source credibility (α = .80), the perceived
message credibility (α = .85), and the overall perceived credibility (α = .90) had a good
reliability. The three resulting perceived credibility variables that were computed by
averaging the scores of their corresponding questions were submitted to the statistical
analyses that are described in the results section.
224. Results
To test whether the post with information was considered more credible as the post with
disinformation and to see if a disclaimer has an influence, a factorial ANOVA was performed
with Content and Disclaimer as between participants variables. This test was conducted on
the level of perceived source credibility, the perceived message credibility, and the overall
perceived credibility.
4.1 Perceived Source Credibility
The credibility scores were not normally distributed since the z-scores of skewness (z-
score = -1.45) and kurtosis (z-score = .89) of content with information, and the z-scores of
skewness (z-score = 2.51) and kurtosis (z-score = .42) of content with disinformation were
not acceptable. The same applies to the z-scores of skewness (z-score = 0.12) and kurtosis (z-
score= -2.28) of content with disclaimer, and the z-scores of skewness (z-score = 2.09) and
kurtosis (z-score = -1.33) of content without a disclaimer, that were not acceptable.
Also, the assumption of homogeneity of variances was not met. Levene’s test was
significant: F (3, 200) = 3.83, p< 0.05). The Factorial ANOVA is fairly robust against the
violations of these assumptions, but the outcomes may not be completely reliable. See Table
1 for the mean scores of the four conditions.
Table 1. The mean perceived source credibility scores as a function of Content
(information/disinformation) and Disclaimer (with/without).
Social media post Mean Standard error
Information with disclaimer 4.4 .139
Information without disclaimer 4.4 .137
23Disinformation with disclaimer 2.8 .141 Disinformation without disclaimer 3.1 .135 There was a main effect of Content on the level of perceived source credibility: F(1, 200) = 104.14; p
4.2 Perceived Message Credibility The perceived message credibility scores were normally distributed since the z-scores of skewness (z-score =-.34 ) and kurtosis (z-score = -1.28) of content with information, and the z-scores of skewness (z-score = .31) and kurtosis (z-score = -1.87) of content with disinformation were acceptable. The same applied to the z-scores of skewness (z-score = - .70) and kurtosis (z-score= -1.51) of content with disclaimer, and the z-scores of skewness (z- score = .54) and kurtosis (z-score = -1.56) of content without a disclaimer, that were both acceptable. Because the assumption of homogeneity of variances was met, the Levene’s test of equality of error variances was not significant (F(3, 200) = 1.22, p = .303), these results could be interpreted accurately. See Table 2 for the mean scores of the four conditions. Table 2. The mean perceived message credibility scores as a function of Content (information/disinformation) and Disclaimer (with/without). Social media post Mean Standard error Information with disclaimer 4.7 .121 Information without disclaimer 4.9 .119 Disinformation with disclaimer 3.3 .122 Disinformation without disclaimer 3.6 .118 There was a main effect of Content on the level of perceived source credibility: F(1, 200) = 123.92; p
The results did not support H2 which stated that a post with a disclaimer has a lower
level of perceived credibility than a post without a disclaimer. There was no effect of
Disclaimer, however the results were almost significant and therefore could be interpreted
as a trend: F(1, 200) = 3.45; p= .065; η² = 0.017. The participants rated the perceived
message credibility of a social media post with a disclaimer on average with a 4.0 (SE=.09),
and a social media post without a disclaimer for information on average with a 4.3 (SE=.08).
This means that, even though the difference was close to significance, the second hypothesis
is rejected.
There was no interaction between Content and Disclaimer: F(1, 200) = 0.10; p= .748;
η² = 0.001. Participants rated the perceived message credibility of a social media post with
information and with a disclaimer lower than for the content of information without a
disclaimer. For social media posts with disinformation, the same applied to the level of
perceived message credibility. Participants rated the perceived message credibility of a
social media post with disinformation and with a disclaimer lower than for disinformation
without a disclaimer. However, when you look to the exact mean differences of social media
post with disinformation between with disclaimer and without disclaimer there can be seen
that the pattern also applies for the perceived message credibility.
4.3 Overall Perceived Credibility
The overall perceived credibility scores were normally distributed since the z-scores of
skewness (z-score =-.40 ) and kurtosis (z-score = -1.29) of content with information, and the
z-scores of skewness (z-score = 1.05) and kurtosis (z-score = -1.53) of content with
disinformation were acceptable. The same applied to the z-scores of skewness (z-score = -
.09) and kurtosis (z-score= -1.93) of content with disclaimer, and the z-scores of skewness (z-
26score = 1.50) and kurtosis (z-score = -1.40) of content without a disclaimer, that were both acceptable. Because the assumption om homogeneity of variances was met, the Levene’s test of equality of error variances was not significant (F(3, 200) = 1.04, p= .376), these results could be interpreted accurately. See Table 3 for the mean scores of all four conditions. Table 3. The mean overall perceived credibility scores as a function of Content (information/disinformation) and Disclaimer (with/without). Social media post Mean Standard error Information with disclaimer 4.6 .120 Information without disclaimer 4.7 .118 Disinformation with disclaimer 3.2 .121 Disinformation without disclaimer 3.4 .116 There was a main effect of Content on the level of the overall perceived credibility: F(1, 200) = 131.42; p
without a disclaimer for information on average with a 4.1 (SE=.09). This means that the
second hypothesis is rejected.
Figure 2. The mean overall perceived credibility scores as a function of Content
(information/disinformation) and Disclaimer (with/without).
There was no interaction between Content and Disclaimer: F(1, 200) = 0.41; p= .525; η² =
0.002. Participants rated the overall perceived credibility of a social media post with
information and with a disclaimer lower than for information without a disclaimer. For social
media posts with disinformation with a disclaimer, the same applied to the level of the
overall perceived credibility. Participants rated the perceived credibility of a social media
post with disinformation and a disclaimer lower than for disinformation without a
disclaimer. However, the mean difference for disinformation with a disclaimer seemed to be
bigger than for information with a disclaimer, which was in line with the expectation, as can
be seen in Figure 1.
284.4. Behavioral questions
At the end of the experiment participants were asked to answer some questions about their
social media behavior and their knowledge of information, disinformation and disclaimers
on social media. See Table 4 for all the scores concerning these behavioral/knowledge
questions.
Table 4. The percentage scores of the answers to the knowledge/behavioral questions.
Social media, disinformation and disclaimers Yes No
Social media as information source 76.5% 23.5%
Disinformation awareness 99% 1%
Disclaimer awareness 81.4% 18.6%
Disclaimer influence 68.6% 31.4%
Information post seen before 74.5% 25.5%
Disinformation post seen before 46.1% 53.9%
The first question here was: “How many hours a week do you spend time on social media?”,
31.4% (N=32), spent 15-20 hours a week on social media, 26.5% (N=27) spent 10-15 hours a
week, 17.6% (N=18) spent more than 20 hours a week on social media and 24.5% (N=25)
spent less than 10 hours a week on social media. With regards to the question if participants
use social media as an information source, the majority of the participants answered with
yes. The question about participants’ awareness of the spread of disinformation was
answered confirmative by almost all participants. Most participants were also well aware of
29disclaimers since the majority of the participants answered with yes. 68.8 % indicated that
disclaimers probably have some effect on how they perceive the credibility of a message and
31.4% answered that disclaimers do not have any effect. The last question was about if they
had seen the social media posts before. With respect to the social media post with
information, 74.5% indicated that they had read the content before and 25.5% answered
that they have not read it before. Regarding the social media post with disinformation,
46.1% indicated that they had read the content before and 53.9% of the sample answered
that they had not read the content before.
5. Conclusion and discussion
This study sought to develop an understanding of whether the use of a disclaimer in an
online social media post influences the level of perceived credibility, and whether it affects
informative and disinformative posts differentially. The conclusion and discussion are based
on the results of the overall perceived credibility. Three hypotheses were tested to answer
the following research question: “What is the influence of a disclaimer on the perceived
credibility of a social media post with information and disinformation?”
Content
The results supported the first hypothesis that online social media posts with the content of
information are likely to have a higher perceived credibility than social media posts with
disinformation. This means that people can see the difference between information that is
found credible and post of disinformation that is deliberately shared to deceive, which is in
line with the study by Li and Suh (2015) and with the results of the pre-test that was
30conducted before. At first, one would think that this has to do with the way that the content
is presented, referring to argument strength and information quality as was discussed before
in the introduction (Li & Suh, 2015). However, both conditions were presented in the same
style and with the same source, so it is questionable that these characteristics had an
influence on the results. It is remarkable that the majority of the participants (74.6%) was
familiar with the social media post with an information content. Research within the context
of E-commerce by Gefen (2000) shows that when something is familiar, it also feels
trustable. Therefore, this shows that the familiarity also possibly had an influence on the
perceived credibility of the social media post with the informative content. This was not the
case for the post with disinformation. Since only half of the participants had seen the social
media post before, a similar conjecture cannot be made.
The use of a disclaimer
The results, furthermore, showed a trend in the influence of a disclaimer on the overall
perceived credibility. The overall perceived credibility of content with the use of a disclaimer
seemed to show some difference compared to content with no use of a disclaimer
concerning H2. So when reading posts with information or disinformation with a disclaimer,
people do seem to have a lower perceived credibility of the content. Looking closer at the
mean scores, it seems that this difference was larger with the disinformative post than with
the informative post, which is in line with the prediction. So, we might conclude that the use
of a disclaimer does seem to have a stronger effect on the perceived credibility of
disinformation than of information, which is in line with the third hypothesis. However, the
data show no strong support. There are several possible reasons why the data show no
strong support. A possible reason that the results do not show a significant difference is
31because of the selected type of disclaimer that was used for the experiment conducted in
this study. Research by Kirchner and Reuter (2020) showed that warning labels can have
effect on the perceived accuracy, especially when social media platforms are transparent
about why the content is disputed. So, using a disclaimer with more detailed information
about why the content is disputed could have given better insight into the effectiveness of a
disclaimer, which is also in line with the study discussed before by Laughery and Stanush
(1989) that explains that explicit warnings with more details are important to see any effect.
There are multiple disclaimers being used already by social media platforms that could have
shown significant results (e.g., “disputed by third-party factcheckers, click to see more
information.”). Facebook uses different types of labels to warn readers about false
information, where some labels are more detailed and extensive about why the content is
disputed with links to credible sources, and some are short with only little information about
that the content is false, similar to the one used for the experiment of this study (Facebook,
2019). The study by Brown, Thomas, and Tiggemann (2019) has shown that the use of
various types of warning labels can show different effects. So, it would be interesting to do a
follow-up study into the use of the other types of disclaimers when it comes to the content
of disinformation. A suggestion is using multiple types of disclaimers that would provide
more substantiating information as to why the information is labeled as false and linking to
reliable and confirmatory articles as evidence. This might help the reader to understand why
the content is false and therefore might have more influence on the perceived credibility. So
being more transparent about why the content of disinformation is disputed would answer
readers’ questions. In addition, it can be good that the results show not strong significance
to the used types of disclaimers in this study, as there is little additional information stated
on the disclaimer why the content is labeled as false. The results shows that the reader does
32not blindly trust something without some substantive evidence and is critically looking for
facts. Moreover, multiple studies stated that critical thinking is an essential skill in identifying
and recognizing disinformation (Machete & Turpin, 2020). Therefore, even though the
results of the conducted tests were not significant, it does show outcomes in the right
direction and the possible effects of a disclaimer.
It can be cautiously concluded, therefore, that disclaimers do help in decreasing the
perceived credibility of social media posts, especially when they contain disinformation, and
that informative posts are not affected as much by disclaimers as disinformative posts.
Limitations and implications
It is somewhat clear that using disclaimers is becoming an important measure for social
media platforms to prevent people from being deceived. However, there are several
limitations and implications that should be taken into account.
The first limitation has to do with the generalizability of this study. The sample of this
study does not represent the Dutch population in terms of age and gender, according to the
to the Central Bureau of Statistics (CBS) (2020) the average age is 31.7. The participants were
mainly in the age category of 18-25. Furthermore, gender was not equally divided since the
majority of the sample (71%) were women. The majority of the participants attained a
Bachelor and Master study, according to CBS (2020) that is also the majority of students in
the Netherlands. It is a legitimate question then whether the results obtained with this
sample can be generalized to the population in general. Disinformation seems to affect less
educated people more severely than higher educated people (Seo et al., 2020). Therefore,
also a replication of this study with a sample from that population would be a very good
idea.
33The second limitation of this study has to do with the stimuli used for the
experiment, the selected social media posts, and the social media platform, i.e., Facebook.
The social media posts were evaluated in a pre-test before the experiment was conducted in
order to identify the informative and disinformative posts. This made a clear identification of
the informative and disinformative content, concerning the validity of this study. However,
the majority of the participants had already seen the disinformative post and could have had
an influence on the responses. Moreover, multiple social media platforms could have been
used, for example Instagram and Twitter since they belong to one of the most popular social
media platforms. The use of one specific social media platform could have had an influence
on the perceived credibility in terms of whether the participants often use Facebook as an
information source or not. There was a question about the use of social media in general,
but not specific about Facebook. A suggestion is to take this into account for future research,
since there is also a possibility that this had an influence on the evaluation of the perceived
credibility of the social media posts.
The perceived credibility was evaluated on the criteria of source and message, as those were
important elements when assessing the perceived credibility of content (Metzger et al.,
2003). To make suggestions for future study, the stimuli can be improved where the source
and message of the content can be described in more detail. The source can be manipulated
by adding more details where readers mostly are looking at, such as number of followers
and the expertise and experience of the source. It would be interesting to see how people
would evaluate the perceived credibility of the posts of disinformation if the source looks
trustworthy and the message is described in more details, with a disclaimer. There are many
posts on social media with disinformation that consists of longer text with argumentation
34You can also read
