Comparing Fake Review Tools on Amazon.com Masterarbeit - unipub
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Marija Basic
Comparing Fake Review Tools on Amazon.com
Masterarbeit
zur Erlangung des akademischen Grades eines Master of Science
der Studienrichtung Betriebswirtschaft an der Karl-Franzens-Universität Graz
Ao.Univ.-Prof. Mag. Dipl.-Ing. Dr. rer.soc.oec. Christian Schlögl
Institute for Information Science and Information Systems
at Karl Franzens University Graz
Graz, April 2020
Table of Contents
1. Introduction 1
1.1 Social influence on opinions and their impact 4
1.2 Ethics, incentives and organizations 5
1.3 Problem statement 7
1.4 Organization of the thesis 8
2. Spam detection 9
2.1 Different types of reviews 9
2.2 Different types of reviewers 11
2.3 About Amazon.com 12
2.3.1 The review system and Amazon dataset 13
2.3.2 Review rating, product review and feedback 14
2.4 Data sources for fake review detection 16
3. Empirical part 18
3.1 State of the art 18
3.2 Objective and research questions 21
3.3 Methodology 22
3.4. Results 25
3.4.1 Selection of the tools 25
3.4.2 Description of the selected tools 26
3.4.2.1 ReviewMeta 26
3.4.2.1.1 Usability of ReviewMeta 26
3.4.2.1.2 Scope, depth and form of outcomes of the reports 30
3.4.2.1.3 Authority of ReviewMeta 45
3.4.2.1.4 Curency of ReviewMeta 46
3.4.2.1.5 Export and share function of ReviewMeta 46
3.4.2.2 FakeSpot 47
3.4.2.2.1 Usability of FakeSpot 47
3.4.2.2.2 Scope, depth and form of outcomes of the reports 51
3.4.2.2.3 Authority of FakeSpot 55
3.4.2.2.4 Currency of FakeSpot 56
3.4.2.2.5 Export and share function of FakeSpot 56
3.4.2.3 TheReviewIndex 56
i
3.4.2.3.1 Usability of TheReviewIndex 57
3.4.2.3.2 Scope, depth and forms of the outcomes of the TheReviewIndex reports 59
3.4.2.3.3 Authority of TheReviewIndex 64
3.4.2.3.4 Currency of TheReviewIndex 65
3.4.2.3.5 Export and share function 65
3.4.3 Comparison of the tools based on criteria 66
3.4.4 Comparison of the outcomes of the tools 71
4. Critical reflection 74
References 76
ii
Figures
Figure 1 Key statistical indicators ......................................................................................................1
Figure 2 The structure of posted reviews by buyers and non-buyers ..................................................2
Figure 3 Increase in suspicious reviews…………………………………………………………..….3
Figure 4 Types of Spams ..................................................................................................................10
Figure 5 An incentivized review .......................................................................................................13
Figure 6 Review rating system on Amazon ......................................................................................15
Figure 7 A review feedback ..............................................................................................................15
Figure 8 Popularity of the topic in academic publications ................................................................19
Figure 9 Research model of the work ................................................................................................22
Figure 10 ReviewMeta extension .....................................................................................................27
Figure 11 The URL search bar on RevieMeta website ......................................................................28
Figure 12 The FAQ browser window on ReviewMeta website ........................................................28
Figure 13 The contact us browser windows on ReviewMeta ............................................................29
Figure 14 Country-based Amazon websites supported by the tool ...................................................30
Figure 15 The final scorecard of the ReviewMeta tool ....................................................................31
Figure 16 Review adjustment function of the report .........................................................................32
Figure 17 An overview of all outcomes in the report .......................................................................33
Figure 18 The most vs trusted reviews in the report ..........................................................................34
Figure 19 The presentation of the deleted reviews in the report ........................................................35
Figure 20 The presentation of suspicious reviewers .........................................................................36
Figure 21 The reviewer ease of the product and the product in its category .....................................37
Figure 22 The presentation of word count comparison .....................................................................38
Figure 23 The rating trend of reviewers ...........................................................................................39
Figure 24 The presentation of reviewer participation ........................................................................40
Figure 25 The presentation of the unverified purchases ....................................................................40
Figure 26 Examples of unverified reviews in unverified purchase report .........................................41
Figure 27 Phrase repetition test .........................................................................................................42
Figure 28 Examples of the posted reviews with repetitive phrases ...................................................42
Figure 29 The percentage of the overlapping reviews .......................................................................43
Figure 30 Percentage of reviews about other products from the same brand ...................................44
Figure 31 The presentation of incentivized reviews .........................................................................45
Figure 32 The currency of the report .................................................................................................46
Figure 33 Browser extension of the FakeSpot tool ............................................................................47
Figure 34 FakeSpot grade on Amazon product page .........................................................................48
Figure 35 FakeSpot grade in a Google search result .........................................................................48
Figure 36 Search bar of the FakeSpot tool.........................................................................................49
Figure 37 The FAQ window .............................................................................................................50
Figure 38 Contact us browser window ..............................................................................................50
Figure 39 Final scorecard of the FakeSpot report ..............................................................................52
Figure 40 The company review report as a part of the final scorecard ..............................................52
Figure 41 The analysis overview section in the report ......................................................................53
Figure 42 The presentation of the review analytic section of the report............................................54
Figure 43 The presentation of the review insights section of the report ............................................55
Figure 44 The information about the currency of the report..............................................................56
Figure 45 The browser extension of the TheReviewIndex tool .........................................................57
Figure 46 The URL search bar of the tool .........................................................................................58
Figure 47 The scorecard overview of the tool ...................................................................................60
Figure 48 The presentation of one of the product performances summarized by the tool ................60
iii
Figure 49 The presentation of warn outcome of the spam test in the report......................................61
Figure 50 The presentation of the passed outcomes of the spam test in the report ...........................62
Figure 51 The specifications section in the report .............................................................................62
Figure 52 Recommendations offered by the tool in the report ..........................................................63
Figure 53 A new browser window of the recommendation section ..................................................63
Figure 54 The web store product link as a part of the report .............................................................64
Figure 55 Last update of a report...………………………………………………………………... 65
iv
Tables
Table 1 Comparison of the tools based on proposed criteria ........................................................66
Table 2 The final outcomes of the compared tools for the Samsung computer ............................71
v
1 Introduction
1. Introduction
Our beliefs and perceptions are widely influenced by opinions of others. These opinions,
consisting of different types of plenty of valuable information, are one of the primary
factors in making our choices and decisions. We rely on others not to make wrong decisions
under uncertainty.1 Generally, how others see and value the things and the world around
themselves largely shapes human behaviour and activities.
Over the years, technology has revolutionized the way how we communicate and created
tools and resources, putting each person’s most useful information at their fingertips.
Modern technologies such as customer review sites, online shops, blogs, social networking
sites, and online communities have allowed individuals to share and read opinions with
other individuals.2 New technologies have influenced the use, scope and effectiveness of
social media in our lives. In favour of this, the new 2018 Global Digital Report reveals that
more than 4 billion people use the internet and more than 3 billion people around the world
use social media (see Figure 1).3 People are prone to be active in online networks in order
to receive opinions and information and are likely to trust others. This is distinctive not
only for consumers as individuals when they are making a purchase decision but also for
firms and organizations when they are making business decisions.
Figure 1 Key statistical indicators (Source: Screenshot from wearesocial.com)
The number of online shops, products and product reviews rises with the growth of users.
1
See Sherif (1935), pp. 47.
2
See Constantinides et. al (2008), pp. 233.
3
See Kemp (2018), n. p.
1
1 Introduction
Thus, a 2017 review report by statista.com reveals that the increased number of product
reviews in online shops has become a major factor in making decisions to buy.4 For
instance, if one wants to buy a product, one will look for relevant information from a large
number of review websites. If most opinions or reviews are positive, one is likely to buy it.
Also, if the opinions of online participants are mostly negative, one will almost certainly
not buy it. Accordingly, if we make our decisions based on fake positive or negative
reviews, then we will make a wrong decision.
There are different reasons for such dishonest actions of users. Recent findings from 2018
made by reviemonitoring.com reveal that almost 18% of reviewers commented on a
product they did not buy in order to receive a gift by friends/family, or they are fans of the
brand. Also, there are individuals who were paid to do it.5 The structure of buyers and non-
buyers is presented below in Figure 2.
Figure 2 The structure of posted reviews by buyers and non-buyers (Source: Screenshot from
reviewmonitoring.com)
Similar to findings from reviewmonitoring.com, 68% of the customers left a review after
a local business asked them to do that. Manufacturers and sellers are today more likely to
trick the system misusing fair business practices by creating incentives for individuals to
write positive deceptive reviews when they are faced with increased competition or when
they have a less established reputation. Positive fake reviews are related to economic
4 See Clement (2018), n. p.
5 See Fake Reviews- Is It Really An Issue (2008), n p.
2
1 Introduction
benefits and may result in significant financial gains, whereas negative fake reviews could
be very harmful to potential buyers and manufacturers which can cause further strong
incentives for writing fake reviews.
The confidence in online reviews is misplaced. According to the Local Consumer Review
Survey 2017, 74% of the consumers have read a fake online review.6 In addition to this
problem, an online tool for detection fake reviews, ReviewMeta.com, reported that the
number of suspicious reviews posted especially in June, July and August 2017 (see Figure
3) have been largely increased, as the average review weight (blue line) drooped off
considerably. The average review weight indicates the measure of trustworthy reviews,
whereas the red line indicates the average rating of suspicious reviews.
Figure 3 Increase in suspicious reviews (Source: Screenshot from forbes.com)
The problem is widespread. According to statistics and many practical examples, fake
reviews will get worse and also more complicated. Both sides, producers and sellers, are
6
See Murphy (2018), n. p.
3
1 Introduction
highly concerned with user reviews. Marketers analyse them intending to find product
problems and provide effective solutions. A manufacturer`s reputation in a business world
is widely affected by user reviews.
Detecting spam reviews will become more and more demanding. That is why it becomes
an absolute necessity to identify and prevent such spamming activities. Therefore, it is
necessary to think about how they can be detected and what are the reliable methods,
approaches, applications or tools available for their identification.
1.1. Social influence on opinions and their impact
Social factors have high relevance in everyday life. In literature, there are numerous studies
and theories that are concerned with the social influence analysing the impact of different
social mechanisms on a change in an individual’s behaviour and how other people’s
opinions affect one’s decisions.
According to psychologists, there are two types of social influence: normative and
informational. Our decisions, opinions and therefore reviews work in both ways as norms
and information (social proof).
Normative social influence is defined in social psychology as “...the influence of other
people that leads us to conform in order to be liked and accepted by them”7. For instance,
if a product gathers more reviews, individuals will be more likely to actually buy it. Thus,
reviews also serve as valuable information. Informational social influence is where a person
conforms to gain knowledge because they believe that someone else is right.8
Psychologist Herbert C. Kelman recognizes three different modes of social influence that
impact customers’ purchase decision, and therefore, there are three different modes of
behaviour of individuals: 9
1. People may be influenced by someone following someone else’s decision or
suggestion despite their own private opinion because of the need to be liked and
not to be different what is also seen in “Asch’s line experiment”10. According to
Kelman, this mode of social influence is called compliance.
7
See Aronson et. al (2005), pp. 253.
8
See Deutsch et. al (1955), pp. 629.
9
See Kelman (1958), pp. 53.
10
See Asch (1955), pp. 5.
41 Introduction
2. The second mode of social influence or internalization occurs when people agree
with a decision of people or groups and adopt it because of the need not to make
wrong decisions. This is particularly seen in “Sherif’s autokinetic experiment”11.
3. The last mode of social influence is identification where a person adopts the
behaviour of a widely respected and admired person, in order to achieve a friendly
relationship with another individual or group.
According to some evidence, people may imitate behaviour and accept opinions of others.
Imitating others’ behaviour can increase people’s efficiency and what was good for one
person will still probably be good for another person.12
All these studies have shown that individuals’ opinions are influenced by opinions of others
not only in real-world but also in online environments. As a consequence, opinions help
people to decide which products or services they could buy and have slowly become an
integral part of the business on the internet.
The digitalization and existence of social media platforms have not largely changed social
behaviours since Kelman’s research. Moreover, the dramatic development of technology
over the past decades has changed only the sources of social influence. They will continue
to have an impact.
Simply, company products cannot compete without other opinions and reviews. The social
media environment is used for advertising, creation of an efficient marketing strategy,
which in turn could help them gain more customers. On the other hand, both positive and
negative deceptive reviews may have a further impact on the revenue, brand awareness,
company reputation and profitability of organizations selling products. These deceptive
reviews by consumers may result in wrong decisions by managers. If managers heavily
rely their tactics and business decisions on them rather than on expert opinions, then they
could make wrong decisions with adverse economic consequences.
1.2. Ethics, incentives and organizations
According to previous findings, reviews have become increasingly popular over the past
decade, and now this phenomenon exists for nearly every product and service. There is a
11
See Sherif (1935b), pp. 49.
12
See McLeod (2011), n. p.
51 Introduction
large number of consumer reviews sites such as Amazon, Angie’s List, Choice,
TripAdvisor and Yelp which bring consumers and businesses benefits.
Their profitability increases with the growth of their popularity or visits too. Higher
profitability caused by the growth of the number of visitors raises the possibility of the
appearance of non-ethical behaviour from organizations and visitors. Sometimes the line
between a good business idea and an unethical or even illegal business practice is not
perfectly clear. In a real market, firms are likely to use illegal means for marketing
purposes, especially when they are faced with an increased competition or established a
bad reputation.
Here are some examples of different forms of unethical behaviour conducted by individuals
and companies:
• Firms are likely to offer customers a product without any payment or to sell them
at a discount in exchange for a review (incentivized reviews).13
• A group of reviewers who collaborate to post fake reviews have been found as
groups with damaging intentions and may be regarded as illegitimate groups.14
• A large number of reviews posted in a very short period may be defined as fake.15
• According to Morning Advertiser, a criminal court in Italy ruled that writing
reviews using a false identity is criminal conduct under Italian law.16
• Online technologies such as automated services (referred to as “bots”) that are
made to create fake identities using a fake person`s name or email were reported
by the European Commission in 2018.17
• Buyers may receive e-mails appearing to come from Amazon, that are actually
false e-mails, sometimes called ‘spoof e-mails’ or ‘phishing e-mails’. The
purpose is to log into an account and to change some parts of their account.18
They can be used for posting fake reviews.
• BBC discovered that Samsung paid students to write negative reviews about
phones made by Taiwan’s HTC.19
13
See McCabe (2018), n. p.
14
See Saradha et.al. (2018), pp. 113.
15
See Mukherjee et.al. (2012a), pp. 1.
16
See Robinson (2018), n. p.
17
See European Commission (2018), pp. 5.
18
See Evans (2018), n. p.
19
See Samsung probed in Taiwan over ‘fake web reviews’ (2003), n. p.
61 Introduction
• Federal Trade Commission in 2010 took action against a public relations business
because of asking their own employees to post reviews as consumers for its
clients.20
• A Dallas business owner accused Yelp of hiding good reviews of his coffee shop
after he refused to pay them for advertising.21
As can be seen from these examples and evidence, fake reviews concern all participants in
a business chain, consumers and sellers. It can cause an erosion of seller credibility and
have an incremental negative impact on the business. To deal with this issue, fake online
reviews should be paid much more attention, as the number of online shoppers has
dramatically increased, and their action may have unethical intentions.
1.3. Problem statement
The challenge of this work is to find the most adequate application or tool for the detection
of fake reviews with the best performance because sometimes it is not possible to recognize
truthful opinions and reviews by merely reading them. There are some of the following
reasons for that:
• Reviews may be written by experts, retailers or any other organization that cannot
be easily distinguished from genuine, honest reviews;
• Organizations may incentivize users to post genuine looking reviews.
The objective of this thesis is to explore online tools available for the detection of fake
reviews. Each tool should be appropriate to identify fake reviews and current habits and
intentions of reviewers. Specifically, I investigated how much these tools may differ from
one another. Also, the work intends to make a comparison using a previously defined list
of criteria and finally to discuss the outcome in more detail. An important aspect of this
research is to analyse their credibility in their detection. This allowed me to infer which
tool is the most reliable, how various tools may increase their performance and what are
the strengths and weaknesses of each of them.
20
See Valant (2015), pp. 6.
21
See Paul (2018), n. p.
71 Introduction
1.4 Organization of the thesis
Chapter 1 gives fundamental explanations of some of the elements and social factors that
influence opinions in the decision-making process. The effect of social influence on
reviews is explained using existing theories. Furthermore, this Chapter highlights different
forms of unethical behaviour of individuals and companies or even possible illegal
activities.
Various types of reviews, spammers, spamming, review ratings, review feedback and
features are considered in Chapter 2. It also presents the online web store amazon.com
focusing more on the main characteristics of Amazon’s dataset that will be used for spam
detection.
Chapter 3 highlights state of the art and presents the research questions and goals of this
master thesis. All steps of the proposed framework for the research are introduced in this
chapter. The steps of the proposed framework are followed by the detailed description of
the tools, their comparison according to well- defined criteria. Finally, the results of the
analysis will be shown.
Finally, Chapter 4 is concerned with conclusions about each analysed tool, their strengths
and weaknesses. This should help not only individual consumers but also businesses and
organizations to choose the most appropriate tool for their purposes. Furthermore, this
chapter suggests directions for possible future work on fake review detection.
82 Theoretical part
2. Spam detection
This chapter should help us better understand reviews and show the differences between
truthful and untruthful reviews. The focus is on definitions and characteristics of reviews,
fake reviews, review ratings and review feedbacks. To inspect the reviews a dataset from
Amazon.com will be used, and their characteristics will be discussed.
2.1. Different types of reviews
In literature, the opinion spam problem was first determined by Jindal and Liu. Basically,
according to Jindal and Lui, there are three types of review spam22:
Type 1 (Untruthful or fake reviews):
This type of reviews are untruthful positive reviews of products that have been posted by
customers or users, with a hidden intention to promote the product or untruthful negative
reviews written with the intention of harming their reputation. These reviews are meant to
mislead readers in their opinions. Usually, they are written with an intention to have an
economic impact.
Type 2 (Reviews on brands only):
These reviews refer to brands, businesses or sellers of a product and do not comment on a
product’s features. For this reason, they are considered as biased. For instance, a reviewer
may write for a specific Samsung product: “I do not like Samsung, and I will never
purchase any of Samsung’s products”.
Type 3 (Non-reviews):
This type encompasses all reviews written for marketing purposes and may not refer to the
product. They can be roughly categorized into two subtypes:
• Advertisement,
• irrelevant text or reviews having random texts, answers, etc.
Most studies in the literature have shown that types 2 and 3 of spam reviews can be very
easily detected by merely reading them. Even a reader with little or no knowledge may
recognize them easily while reading. However, type 1 is tough to detect because those
22See Jindal et. al. (2008a), n. p.
92 Theoretical part
reviews are very similar to truthful reviews. For this reason, the identification of type 1 is
a challenging task as it is hard to distinguish between fake and real reviews by reading
them. This type of reviews could be posted by experts who are paid to write believable
reviews. Neither positive nor negative fake reviews are based on reality. At best, they are
based on a shared misunderstanding; at worst they are outright lies. Both
positive and negative opinions have been shown to influence others in the decision-making
process directly.
Different classifications of fake reviews were the subject of research of many authors. For
instance, there is a classification of opinion spam into deceptive and disruptive opinion
spam.23 Deceptive opinion spam refers to type 1, whereas disruptive opinion spam refers
to types 2 and 3.
Figure 4 Types of Spam (Source: Screenshot)24
Figure 4 presents six different forms of fake reviews. Positive fake reviews for a good
quality product (Type 1) or negative fake reviews for a bad quality product (Type 6) are
not very harmful, because they may be partly true. But there are reviews that demonstrate
the conflict of the review level and product quality and may have hidden motives such as
damaging the reputation of the product or brand with a final result of creating losses or
profits (Type 2, 4) or to promote the product (Type 3 and 5). The primary focus is the
detection of these types of damaging fake reviews using free based web tools.
Also, in literature, there are many researchers that define fake reviews as a form of
“deception”. Buller and Burgoon define deception as “a message knowingly transmitted
by a sender to foster a false belief or conclusion by the receiver”.25 The traditional meaning
of deception, which refers to a false pretence of facts of feelings, has become an integrated
part of the online environment. For instance, a deceptive reviewer writes in a certain way,
23 See Ott, M. et.al. (2011a), pp. 309.
24 See Lim, E. et.al. (2010a), pp. 1.
25 See Buller, D. B., (1996), pp. 239.
102 Theoretical part
using special signals in the text or often using words in order to conceal the truth.26 Many
researchers developed a set of cues for their identification.
Instead of fake reviews, in the literature, a “fraud review” can be used as a term for a fake
one with the same meaning. Fraud reviews occur when online vendors, individuals or
reviewers write “consumer” reviews while presenting themselves as real customers.27 The
Concise Oxford Dictionary defines fraud as “criminal deception; the use of false
representations to gain an unjust advantage”.
2.2. Different types of reviewers
A fake reviewer or spammer may be anyone who writes fake reviews, any type of business
or organization or even a competitor with an intention to provide a kind of reward to those
who post fake reviews for the competitor`s products. Basically, they may act individually
or in a group, without any indication of their secret behaviour at first sight. This is a
collective act, where the members of a group cooperate blindingly.
In the research work of Mukherjee, fake reviewers can be divided into two groups: an
individual spammer and a group of spammers.28 They differ according to the following
characteristics:
• An individual spammer is considered to be an individual, who works exclusively
alone, and therefore has its own user account.
• A group of spammers is a group of individuals who act together, and they can be
• categorized further into two sub-groups:
- A group of individuals who act in a team with a purpose to build up or to demote
the reputation of a target product.
- A single person may be behind different user IDs that help him avoid his
detection and gain rewards by writing many fake reviews.
The distinct behavioural features of fake reviewers were also the subject of the research of
many authors:29
• Spammers are likely to write more reviews for a single product.
• Spammers are likely to write more reviews also for a group of products that
26
See Zhou, L. et. al. (2008), pp. 1.
27
See Hu, N. et. al. (2011), pp. 614.
28
See Mukherjee et.al. (2012b), pp. 1.
29
See Lim, E. et.al. (2010b), pp. 3
112 Theoretical part
belong to a brand.
• The rating score of a group of fake reviewers differs from the rating score of
honest reviewers.
• Spammers tend to rate the product in the early phase after a product’s release.
Taking into account features such as time, frequency and content, the following possible
attributes of spammers were discovered:30
• They act during the same time of the day.
• They post the same text over and over.
• They use different email IDs.
• They are prone to sharing their contacts when they post.
2.3. About Amazon.com
The analysis of this thesis is based on reviews from amazon.com. Amazon has a consumer
review platform where users may review all types of products such as books, music, videos,
electronic cards, auctioned items and many other products offered by sellers.
The company officially launched its websites in Seattle by Jeff Bezos in 1994 and today is
one of the most successful e-commerce companies in the world. The main goal of Amazon
is to sell different products through the same means, from the same storages or through
subsidiaries through a common web site for browsing, searching, and buying. The
simplicity, easiness, and low cost of the shopping experience is the driving motive of this
company.
In October 2018, Amazon offered more than 3 billion products across 11 marketplaces
worldwide.31 According to the statista.com report from October 2018, Amazon received
approximately 2.4 billion32 unique visits and contains more than 7 million reviews.
Amazon counts more than 76 million user accounts and 1.3 million active sellers.
One of the concerns of Amazon is “incentivized reviews”.33 Incentivized reviews are
product reviews posted by buyers that received compensation, such as a discount or free
item, for writing a review (see Figure 5).
30
See Husna, H., et. Al. (2008), pp.1
31
See Scrape Hero (2018), n. p.
32
See Statista. (2019), n. p.
33
See Amazon.com. (n. Y.-b), n. p.
122 Theoretical part
Figure 5 An incentivized review (Source: Screenshot from landingcube.com)
Amazon offers a few ways for sellers to get an initial set of reviews for newly launched
products such as “Early Reviewer Program” and “Vine Program”.
Through “Early Reviewer Program”, Amazon provides direct incentives (e.g. small
discount) for verified buyers if they share their real and honest experience about products.34
Actually, vendors ask reviewers through Amazon for their real feedback, but without any
direct contact between them.
The “Amazon Vine Program” is exclusively an invitation program for honest reviews.35
Amazon invites the members of Vine program to post reviews about a new product. The
members of Vine program are the most trusted reviewers or top-ranked reviewers. Their
opinions may help those who are not prone to comment, to get information about products.
2.3.1. The review system and Amazon dataset
One of the key reasons for the company’s success is that Amazon has review systems,
which may help both customers and organizations. Basically, a review system is a part of
the reputation of every online company because it may increase the trust in internet
interactions.36
The first of Amazon’s Leadership Principles is “Customer Obsession”37 and to be
customer-focused. Indeed, customer reviews have been a crucial part of Amazon for over
20 years. The 5-star rating system allows users to rank products (1 is the worst, 5 is the
best), write or read comments. In this way, ranked products with stars demonstrate the level
of popularity or quality, which is often useful to other buyers when they cannot touch or
test products before purchasing.
34
SeeAmazon.com Help: What is the Early Reviewer Program? (n. Y.), n. p.
35
See Amazon.de- Amazon Vine- Club der Produkttester Programm (n. Y.), n. p.
36
See Resnick, P. (2000), pp. 45.
37
See Amazon.com. (n. Y.-a). n. p.
132 Theoretical part
Amazon presents a compelling setting in which fake reviews can be investigated. Amazon
counts more than 7 million reviews. In particular, the reasons for using the Amazon dataset
are the following:
• Amazon is the top starting point for the product search by consumers, according
to the recent finding of ReviewMonitoring.com38
• the dataset is large and transparent,
• it has a rating system,
• it is easy to access the data, and
• it covers a very wide range of products.
Each of Amazon’s review dataset consists of the following features:
• reviewer’s name (a real name or a nickname),
• product name,
• rating score (from one to five stars),
• review title,
• review body,
• number of feedbacks,
• number of helpful feedbacks, and
• the date of the review.
2.3.2. Review rating, product review and feedback
Each review contains three parts: product review and feedback. The main distinctions
between online reviews and feedback are the kind of data being collected and how it is
processed.
Product reviews refer to consumer evaluations and judgments about a particular product in
the form of 5-star scale (with 1 being the worst, 5 being the best). A low rating indicates a
very poor opinion about a product, whereas a high rating indicates a positive opinion. Those
evaluations contain quantitative and qualitative information on the product’s attributes.
Overall, the final rating of the product is calculated as the average of all the ratings (see
Figure 6).
38
See ReviewMonitoring.com. (n. Y.), n. p.
142 Theoretical part
Figure 6 Review rating system on Amazon (Screenshot from amazon.com)
Review feedback presents truthful or untruthful information about prior online experience
of an individual posted at web sites. Other users may use that information to adjust and
improve future decisions. Moreover, these feedback results provide businesses with deep
insights into consumers’ experience, which marketers can analyse and use to adjust or
improve their services and products.
A review product differs from review feedback. According to Amazon, review feedback is
considered to be a review for the seller or the purchasing experience that emanates after
purchase, whereas a product review refers exclusively to an item purchased (see Figure
7).39 One typical example for feedback appointed to a seller may be the following: “The
phone case was delivered in time that I expected, and the product was exactly as I saw it
pictured. Thank you!”
Figure 7 A review feedback (Source: Screenshot from amazon.com)
In order to be legitimate and not manipulated, Amazon has defined guidelines for feedbacks
39 See Amazon.com (n. Y-d), n. p.
152 Theoretical part
strictly. If feedback contains “promotional content, obscene or abusive language, personal
information”40, the product reviews may be removed at Amazon’s discretion.
2.4. Data sources for fake review detection
There are three important data sources which can be used for fake review detection:41
• Review content refers basically to the content of a review. On this level, one
learns essentials about the content (e.g. that texts are very similar for different
products, or that a critique of a book was simply copied), about the
professionalism (e.g. text length) or the words used for lies.
• Meta-data informs users about the given stars (for example, a reviewer gave 5-
stars for the brand and 1 star for the competitor’s brand), the user ID (a reviewer
has more user IDs), the time of posting (the average rating of a product has been
changed suddenly or a considerable number of reviews appear suddenly), IP and
MAC addresses, location of the computer, and the number of clicks.
• Product information reports information about the product, e.g., sales volume,
the rank of a product or a product’s characteristics. For instance, a product is
selling well but has many negative reviews.
Taking into account these three data sources, Liu, in his book, suggests the following
indicators for detection:42
• Review content:
- Headline of review
- Text
• Meta-data
- Review URL
- Rating
- Number of reviews
- Number of ratings with five stars
- Number of ratings with a star
- Reviewer
40
See Amazon.com (n. Y-e), n. p.
41
See Liu, B. (2012-b), pp. 126–127.
42
See Liu, B. (2012-c), pp. 126–127.
162 Theoretical part
- User ID
- Verified purchase
- Review was helpful
- Date
• Product information
- Product description
- Sales rank
- Sales volume.
Considering all these elements of reviews, the intentions and behaviour of spammers can
be possibly discovered. For their detection, there are free web tools that report fake reviews,
reviewers and groups of spammers on the basis of the features described above. They may
explain untruthful behavioural patterns based on this data. In this work, several fake review
tools were analysed and compared to see how reliable they are.
173 Empirical part
3. Empirical part
This Chapter is divided into three parts. The first section is devoted to the state-of-the-art
with an aim to present an overview of the studies in this field of analysis. For that purpose,
I searched publications in Thomson Reuter’s online database Web of Science. This is
followed by the description of the research object and the research questions. The second
part’s purpose is to explain the different steps that were carried out in order to conduct
the tasks and analysis of this thesis. More precisely, conducting the tasks requires steps
such as well-defined criteria for the selection of web tools that will be used for fake review
detection, description of the web tools, data gathering and finally, testing the tools.
3.1. State of the art
This section gives an overview of the state of the art and a literature review of the most
important research papers on fake review detection so far. For that purpose, I conducted
the following steps: First, I conducted a search query on Thomson Reuters Web of
Science. Second, as there has been much research on this topic in recent years, after
examining titles, abstracts, and knowledge domains and used synonyms, I narrowed the
search results. In the third step, the significant research methodologies on fake reviews
are presented. In order to conduct a meaningful search in Web of Science, it was necessary
to analyse frequently used synonyms. In the course of the literature search, besides the
“fake review”, the following synonyms were used: fake reviews, deceptive reviews,
bogus reviews, fraudulent reviews, and spam. In order to find the methods, which were
used for detection or identification of fake reviews, I used the advanced search function
of the Web of Science. The original search term “TS = (Fake OR Spam OR Deceptive
OR Bogus OR Fraudulent) AND (Review*)” was extended by “AND TS = (Detect* OR
Identify*) AND TS = (Method* OR Tool* OR Application* OR Criteria OR
Requirement*)”. Thus, the search result includes tools and software, which can be used
for the identification of fake reviews. Figure 8 reveals the trend in articles on this topic
from 2000 to 2019 showing especially significant growth of publications in 2017 with
significant slowdown in 2018 and 2019.
183 Empirical part
Figure 8 Popularity of the topic in academic publications (n=270) (Source: Screenshot from webofscience.com)
After examining titles, abstracts, knowledge domains and topics that thematically related
to the subject of the thesis considering the comparison of methods, tools and software,
almost 270 articles were related to the fake review meaning. Additionally, the references
of these articles have been looked at and some of the sources have been used for the
literature review as well.
The subject of fake review, review spam or opinion spam has been studied by many
authors. Various methods for the identification of spam reviews have been proposed by
different authors. However, a very small number of authors have based fake review
detections on tools. For instance, Ott, Choi, Cardie & Hancock have developed an open
tool reviewsceptic.com43. In their study, they claim that the tool may detect a fake review
with 90% accuracy by using psychological and linguistic criteria.44
Methods dedicated to fake review detection can be grouped into three categories:
• Review Centric:
Most works in this field propose a set of features that will give information about
43
See Review Sceptic (2013), n. p.
44
See Ott, M. et.al. (2011-b), pp. 316
193 Empirical part
the reviews. These features refer to characteristics, sentiment and content extracted
from the review. Thus, various combinations of clues engineered from review
content have been studied by Jindal et al.45, Lie et al.46, Ott et al.47, Fei et al.48,
Shojaee et al.49, Li et al.50 , Ong et al.51, Kim et al.52
• Reviewer Centric:
In terms of reviewer centric, many authors focus their studies on reviewer features.
These features give information about the behaviour of reviewers and reviewers’
profile features. For instance, differences between rating and honest reviewer
meanings have been shown by a group of the authors Savage, Zhang, Yu, Chou,
and Wang. 53
• Time Centric:
Time centric works use time-related clues to develop time-based patterns such as a
spam attack in bursts for one product. Many authors suggest well-defined criteria
as indicators for detecting review spammers in review bursts (e.g. frequency of a
review). For instance, Fei, Mukherjee, Liu, Castellanos and Ghosh have
implemented an algorithm which detects a massive number of reviews within a
short time.54
After examining many articles and studies, it can be concluded, that research in this field
is pretty much dedicated to the detection of fake reviews by proposing different methods
for their analysing. However, a very small number of authors have based their study on
tools.
In terms of a comparison of web tools for fake reviews detection, there is only a negligible
number of authors. For instance, Ana Pieper, in her paper, has suggested an approach,
ReviewAlarm, and compared it with an already developed automated tool for identifying
non-hotel fake reviews, Reviewsceptic. This new approach additionally considers
behaviour based and time-based clues. Finally, based on the previously defined dataset,
45
See Jindal et. al. (2008-b), n. p.
46
See Li, F. et. al. (2011), pp.2488–2493.
47
See Ott, M. et. al. (2011-b), pp. 310
48
See Fei, G. et.al. (2013a), pp. 176.
49
See Shojaee, S., et. al., (2013), pp. 55.
50
See L, and J., et. al. (2014), pp. 1566.
51
See Ong, T., et. al. (2014), pp. 69–78.
52
See Kim, S., et. al. (2015), pp. 2–10.
53
See Savage, D., et. al. (2015), pp. 8650–8657.
54
See Fei, G. et.al. (2013b), pp. 176.
203 Empirical part
the performances of Reviewsceptic and ReviewAlarm were tested.55 Additionally,
Crawford, Khoshgoftaar, Prusa, Richter and Najada have compared the results of the
techniques and methods used for fake review detection based on determined criteria but
without considering any web tools.56
Despite the vast interest in the subject and the wide popularity of some methods, there are
no studies which compared existing fake review detection tools. However, there are
plenty of authors that compare different social media tools according to advanced defined
criteria. For instance, a group of authors presented a systematic approach for the
evaluation criteria of testing tools. Their model relies on the ISO/IEC 9126 quality model
with the addition of other criteria such as functionality, reliability, usability, efficiency
and many others.57 Similarly, several Business Intelligence tools were compared,
whereby a set of features were defined in order to choose the best one.58
Since there is no study on this topic, there is a strong need for comparing fake review
detection tools. This comparison may contribute to a better understanding of the potential
limitations, advantages, and disadvantages of popular free web tools.
3.2. Objective and research questions
According to many findings, consumers are increasingly relying on web tools in their
decision-making process. Free web tools can empower consumers and bring transparency
to the reviews. Therefore, the main objective of this thesis is to explore, test and analyse
the current Web platforms for fake review detection.
The objectives of this thesis are:
• To investigate which tools can be used for spam detection;
• To build an extensive list of criteria that can be used to compare free web tools;
• To test the reliability of web tools in detecting fake reviews considering a list
of criteria for that;
• To propose recommendations on how to increase the performance of these
tools.
55
See Pieper, A. T. (2016), pp. 2.
56
See Crawford, M., et. al. (2015), n. p.
57
See Illes, A., et. al. (2005), n. p.
58
See Brando, A., et. al. (2016), pp. 6–9.
213 Empirical part
Therefore, this study aims to answer the following research question:
1. Which open-access tools for spam detection are currently available?
2. How strongly do the tools differ from each other?
3. What are the strengths and weaknesses of free web tools?
3.3. Methodology
This Chapter provides a detailed description of the procedures and methodology set in
this thesis. Following a broad definition of the problem and its associated objects, the
proposed methodology aims to provide an effective framework for comparing online fake
review tools. More precisely, the analysis of each employed web tool is conducted in
order to distinguish respective contributions and performances of each tool.
For this purpose, the research model consists of the following four steps: selection,
description, comparison and analysis (see Figure 9).
Comparison of
Selection of tools Description of tools
tools
Figure 9 Research model of the work (Source: self-presentation)
1. Selection of the tools
The first step of the proposed framework involves the selection of free online tools for
fake review detection according to previously well- defined criteria. All tools that fully
utilize information from the online web platform Amazon.com are included in analysis.
Generally, detection should be presented by providing a report of product reviews. This
means that each tool can be regarded as a filtering system where a product review is an
input, and the result is a report indicates whether a review is truthful or spam. The
selection of the tools is based on the following criteria:
• The tool can be used by everyone (sellers, buyers, users) and not only for sellers;
223 Empirical part
• It is a free source and grants users permission without any costs to use and study
detection results;
• It should allow further investigation in the form of a produced report and does
not deliver the result whether a review is truthful or untruthful.
2. Description of selected web Tools:
Each selected tool has its own individual features and characteristics. Therefore, the
presentation of each tool is followed by pointing out useful characteristics about usability,
scope and depth as well as giving some hints on the functionality of each tool. More
precisely, conducting the description requires the analysis of each tool according to a
previously defined set of criteria. For this purpose, different criteria from different articles
have been looked at and on this basis of different criteria, tools are compared.
The selected tools will be described and compared according to the following set of
criteria:
Criteria 1 Usability
• Accessibility59
- How quickly can users use the tool?
- Is registration required?
- How detailed is the registration?
• Supported browsers and devices
- Works across different browsers and operating systems?
- How many browsers?
- Works on devices (mobile phones or tablets) that users prefer to use?
- In order to use the tool, does anything have to be downloaded and installed?
• Ease of navigation/search60
- Can users find the fake review detection search bar quickly?
• User support61
- Is there any help function for users?
o Availability of support and knowledgebase (specific help section, FAQ) for
users?
59
See Hassan, S., et. al. (2001a), pp 4.
60
See Hassan, S., et. al. (2001b), pp 4.
61
See Hassan, S., et. al. (2001-c), pp 4.
233 Empirical part
- Does the tool offer any options to report possible errors?
o Is there any contact address for technical concerns available?
• Language support62
- Language of the reports and user interface
o Are the results of the tool supported in different languages?
o Which languages are supported?
o Is the used language mentioned on the website?
- Language of the reviews
o Can the tool analyse reviews from different country-based Amazon sites?
o Which Amazon sites are supported?
Criteria 2: Scope, depth and forms of the outcomes of the reports
• Scope
- How many different tests does the tool run?
• Presentation of the results
- Is the report well-structured?
- How does each tool summarize the results (through letter grade A, B, C, D, E, F
or through PASS, WARN or FAIL)?
- Is it easy to understand the results?
o Does the tool use simple sentences for the presentation of the results?
o Does the report include multimedia elements (image, voice and video data)
for the purpose of results presentation?
• Depth
- Is the report supported by evidence such as statistics? Is each test supported by
evidence?
- Does the report of the tool provide explanations for users to better understand the
discovered outcomes?
Criteria 3: Developer of the tool
• Availability of author information
- Is it clear who the developer is?
62
See Nantel, J., et.al. (2008), pp. 112–122.
243 Empirical part
Criteria 4: Currency
• When was the tool developed?
• When was the tool updated last time? Is there information provided on what was
changed?
• Is it possible to conduct an analysis with current data?
Criteria 5: Export and share function
• Is it possible to export the report?
- Which formats can reports be exported?
3. Comparison of selected web Tools:
The next step of the proposed framework is dedicated to the comparison of the selected
tools. More precisely, the described tools will be compared according to the criteria.
Indeed, the performance of each tool reflects the degree of the tool quality. For instance,
the higher the degree of usability, the higher is the quality of the tool. The goal of the
proposed method for comparing different tools should be useful to pinpoint the strengths
and weaknesses of the tools in terms of their effectiveness.
3.4 Results
In this Chapter, each step of the proposed framework will be performed, described and
presented. For that purpose, the product reviews will be used from Amazon.com. All
product reviews will be further divided among five product categories, i.e., from those
with minor reviewing activity to those with substantial reviewing activity. Based on these,
evaluation conclusions will be drawn about tool performance and how it varies for
products with different volumes of reviewing activity.
3.4.1 Selection of the tools
In the selection process, all free online tools that meet all the condition stated above in
Section 3.3 are taken into account. They are:
• ReviewMeta.com
253 Empirical part
• FakeSpot.com
• TheReviewIndex.com
3.4.2 Description of the selected tools
3.4.2.1 ReviewMeta
Tommy Noonan has developed a fake review website with the full name
“ReviewMeta.com - Amazon Review Checker” in May 2016. Inspired by the success of
his previous website SupplementReview.com for the review of supplement products, he
developed the ReviewMeta platform to analyse and filter the product reviews transparent
for consumers before they make a purchasing decision on Amazon.com.63The Web
address of the platform is reviewmeta.com. The visitors may analyse the product reviews
and ratings from Amazon.com, Amazon.co.uk, and Amazon.ca and Bodybuilding.64
3.4.2.1.1 Usability of ReviewMeta
Accessibility
The tool is designed to work for all people whatever their hardware, software, location,
or ability. There are no fees charged for its use. To access the tool, registration is not
required. It is suggested only for a full report in order to be sent to a visitor’s email address
for free. Also, for the purpose of using the forum section, the registration of the user is
required.
Analysing the extent to which the tool is compatible with different browsers, it was found
that the content and design of the website work on the following web browsers:
- Internet Explorer 11,
- Google Chrome 72,
- FireFox 65,
- Opera 58,
- Safari 5.
For a full analysis, this service works by copying any Amazon product URL into the
63 See ReviewMeta.com Press & Media - ReviewMeta Blog. (2018a), n. p.
64 See ReviewMeta.com Press & Media - ReviewMeta Blog. (2018b), n. p.
263 Empirical part
search bar on ReviewMeta.com or by using the ReviewMeta browser extension (Chrome,
Firefox, Safari or Explorer) in the toolbar. The tool is designed to accommodate also
portable devices (mobile sites) for Android and IOS. To use an extension, users must
only download it, and the extension is added to an offered browser. This allows users to
read an adjusted rating of the product directly during the visit of the Amazon website (see
Figure 10). The adjusted rating is colour based on the authenticity of reviews. Red colour
indicates unnatural behaviour of reviews (fake reviews), whereas green coloured adjusted
ratings refer to the reviews which passed.
Yellow coloured adjusted rating suggests neutral behaviour of reviews. To see a detailed
report, users should click the ReviewMeta extension in the toolbar.
Figure 10 ReviewMeta extension (Source: Screenshot from reviewmeta.com)
Ease of navigation/search
Navigating around ReviewMeta is not difficult as there are two search bars available.
They are very distinctive, and it is easy to find them. One is located at the top of the page
and the other at the centre of the home page. This enables users an intuitive, simple and
easy to use system (see Figure 11). If users analyse product reviews with the browser
extension, they can quickly scan an adjusted rating in a horizontal movement across the
upper part of the ReviewMeta page.
273 Empirical part
Figure 11 The URL search bar on RevieMeta website (Source: Screenshot on reviewmeta.com)
Interactivity and user support
Although there is no particular “help section” provided on this website, the
ReviewMetatool provides several help links or options available for user support. For
instance, how visitors should use the tool is described in the “How it works” section,
located at the top of the website. Additionally, a detailed knowledge base is also available
in its “FAQ” section which helps users to find their own answers, explanations and
concerns related to the platform and results of the report (see Figure 12). Also, on the
home page of the tool, it is clearly described how to use the service.
Figure 12 The FAQ browser window on ReviewMeta website (Source: Screenshot from reviewmeta.com)
28You can also read
