Sentiment Analysis in TripAdvisor
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
AFFECTIVE COMPUTING AND SENTIMENT ANALYSIS
Editor: Erik Cambria, Nanyang Technological University, Singapore, cambria@ntu.edu.sg
Sentiment Analysis
in TripAdvisor
Ana Valdivia, M. Victoria Luzón, and Francisco Herrera, University of Granada
T
he number of Web 2.0 websites has recently product, and so on is generally positive, negative,
experienced significant growth. These web- or neutral. The interest in sentiment analysis has
increased significantly over the last few years due
sites emerged as an evolution of Web 1.0, or static, to the large amount of stored text in Web 2.0 ap-
websites. In Web 2.0, users aren’t only content plications and the importance of online customer
consumers, they can also generate content and col- opinions. As a result, more than 1 million research
laborate with other users. In this way, users take papers contain the term “sentiment analysis,” and
an active role and create a virtual community. various start-ups have been created to analyze
Web 2.0 websites include blogs (Blogger or Word- sentiments in social media companies.
press), media content (Prezi, YouTube, Flickr), wi- Multiple studies on TripAdvisor exist, but there
kis (Wikipedia, WikiSpace), collaboration (Drop- is no complete analysis from the sentiment analy-
box, Google Docs), and social networks (Twitter, sis viewpoint. This article proposes TripAdvisor
Facebook, Google+).1 The burgeoning informa- as a source of data for sentiment analysis tasks.
tion explosion offered through Web 2.0 suggests We develop an analysis for studying the match-
that customers often check other users’ opin- ing between users’ sentiments and automatic sen-
ions in forums, blogs, and social networks be- timent-detection algorithms. Finally, we discuss
fore buying a product or contracting a service. some of the challenges regarding sentiment analy-
TripAdvisor emerged in 2004 as a Web 2.0 appli- sis and TripAdvisor, and conclude with some final
cation for the tourism domain. This user-generated remarks.
content website offers a plethora of reviews de-
tailing travelers’ experiences with hotels, res- TripAdvisor and Sentiment Analysis
taurants, and tourist spots. TripAdvisor has According to Wikipedia, TripAdvisor is an Amer-
since been ranked as the most popular site for ican travel website company providing reviews
trip planning, with millions of tourists visit- from travelers about their experiences in hotels,
ing the site when arranging their holidays (see restaurants, and monuments. Stephen Kaufer
Figure 1). and Langley Steinert, along with others, founded
Sentiment analysis is a natural language pro- TripAdvisor in February 2000 as a site listing in-
cessing tool that is useful for monitoring Web 2.0 formation from guidebooks, newspapers, and mag-
applications, as it can reveal public opinion about azines. InterActiveCorp purchased the site in 2004,
numerous issues without requiring satisfaction and one year later, spun off its business travel
enquiries. 2,3 According to the Oxford diction- group, Expedia. After that, the website turned to
ary, sentiment analysis is the process of compu- user-generated content. It has since become the
tationally identifying and categorizing opinions largest travel community, reaching 390 million
expressed in a piece of text to determine whether unique visitors each month and listing 465 million
the writer’s attitude toward a particular topic, reviews and opinions about more than 7 million
72 1541-1672/17/$33.00 © 2017 IEEE IEEE INTELLIGENT SYSTEMS
Published by the IEEE Computer Society
accommodations, restaurants, and attractions in 49 markets worldwide. Figure 2 shows the Google search rate for TripAdvisor, illustrating its popu- larity around the world. Because it has so much data, Trip Advisor has become extremely popu- lar with both tourists and managers. Tourists can read the accumulated opinions of millions of everyday tour- ists. They can also check the popular- ity index, which is computed using an algorithm that accounts for user Figure 1. Granada (Paseo de los Tristes). TripAdvisor is the most popular site for reviews and other published sources planning a trip. such as guidebooks and newspaper articles. This index runs from num- ber 1 to the overall total number of restaurants, hotels, or other attrac- tions within the city. Travelers can find the most interesting visitor at- traction or most popular restaurant. Linked to this is the bubble rating (user rating), a 1–5 scale where one bubble represents a terrible experi- ence and five bubbles an excellent experience. All reviewers are asked to use this scale to summarize their feedback. Together with this rating, users include their opinions, which can cover the performance of a res- taurant, hotel, or tourist spot. There- fore, reading and analyzing reviews Figure 2. Map of the popularity of TripAdvisor searches in Google over the last can help develop a business. five years. The World Travel & Tourism Coun- cil report shows that tourism gener- ates 9.8 percent of the wider gross data. Sentiment classification, the detect sentiments from documents, domestic product and supports 248 best-known sentiment analysis task, sentences, or words. A large variety million jobs.4 These numbers suggest aims to detect sentiments within a of SAMs address the different catego- that the tourism industry is the most document, a sentence, or an aspect. ries of texts (blogs, reviews, tweets, important economic driver of many This task can be divided into three and so on). However, the analysis of economies. Therefore, it’s important steps: polarity detection (label the feelings is not a perfect science, espe- to understand the main drivers of the sentiment of the text as positive, neg- cially when applied to the unstruc- tourist flow as well as tourists’opinions ative, or neutral), aspect selection/ tured texts that predominate in social about a city’s restaurants, hotels, and extraction (obtain the features for networks.9 Human language is com- tourist attractions. structuring the text), and classifica- plex, so teaching a machine to detect TripAdvisor has enough standing tion (apply machine learning or lexi- different grammatical nuances, cul- to be used as a text source, 5 storing con approaches to classify the text). tural variations, jargon, and misspell- numerous reviews of tourist busi- Sentiment analysis methods (SAMs), ings in messages on the network is a nesses around the world. Sentiment which are trained for sentiment po- difficult process, and it is even more analysis extracts insights from this larity detection,6–8 can automatically difficult to automatically understand JULY/AUGUST 2017 www.computer.org/intelligent 73
Alhambra Mezquita Córdoba Sagrada Familia Alhambra Mezquita Córdoba Sagrada Familia
100%
of total SentiStrength
60%
Percent
90% 40%
20%
80% 0%
Percent of total Bing
70% 40%
Percent of total user rating
60% 20%
0%
50% 80%
Percent of total Syuzhet
60%
40%
40%
20%
30%
0%
20%
Percent of total
40%
CoreNLP
10%
20%
0% 0%
Negative
Neutral
Negative
Neutral
Negative
Neutral
Negative
Neutral
Negative
Neutral
Negative
Neutral
Positive
Positive
Positive
Positive
Positive
Positive
Figure 3. Distribution of sentiments between TripAdvisor users (bubble ratings) and four sentiment analysis methods (SAMs):
SentiStrength, Bing, Syuzhet, and CoreNLP. Red indicates negative sentiments, orange neutral, and green positive.
how the context can affect the mes- social networks, such as TripAdvisor for Figure 3 shows the results. We ob-
sage’s tone. Because humans can ap- analysis.12 Similarly, other research- serve that the distributions of polari-
ply a contextual understanding, we ers developed a tool to analyze tour- ties are different from user ratings. The
can intuitively interpret the intention- ists’ opinions of restaurants as well as user ratings bar plot shows that us-
ality of any writing. Computers, how- hotels from a region in Chile.13 ers tend to rate their visits to the three
ever, have difficulty understanding the monuments positively, with more than
context in which a phrase is expressed A Study for Calibrating 90 percent of ratings having four or
and detecting whether a person is be- User’s Polarity five bubbles. SentiStrength and Syuzhet
ing sarcastic or not. We scrape TripAdvisor webpages methods reach a similar distribution
A few sentiment analysis studies on three well-known monuments in to the user rating. However, Bing and
set TripAdvisor as a data source. For Spain: Alhambra, Mezquita Cór- CoreNLP detect more negativity in the
example, researchers have collected doba, and Sagrada Familia. We con- TripAdvisor opinions. In all cases, the
reviews about the TripAdvisor app in sider user ratings of one and two number of neutral polarities is higher
Google Play Store for extracting app bubbles as negative, three as neutral, than the neutral ratings (three bubbles).
features to help developers.10 Others and four and five as a positive sen- Next, we studied the distribution of
analyzed TripAdvisor’s hotel reviews timent. We then apply four SAMs bubble ratings over the negative SAM
for classifying good and bad customer (SentiStrength,14 Bing,15 Syuzhet, and polarities. We thus analyzed the behav-
opinions.11 Still others propose a system CoreNLP6) and extract the overall ior of user feedback against the SAM
to summarize comments from travel polarity on each opinion. evaluations. Figure 4 presents 12 bar
74 www.computer.org/intelligent IEEE INTELLIGENT SYSTEMS
Alhambra Mezquita Córdoba Saqrada Familia
100%
SentiStrength
57.84%
50% 40.22% 42.86%
32.24%
22.22% 23.72%
19.41% 16.26%
14.29% 10.56% 11.89%
9.94% 6.43% 6.60% 9.90%
0%
100% 91.18% 92.86%
78.26%
60.37% 57.14% 55.19%
Bing
50% 45.35% 47.71%
38.89% 38.64%
33.27% 33.95% 31.64%
27.83% 26.56%
0%
100%
Syuzhet
50% 45.10%
36.96% 35.71%
26.06% 28.57% 27.32%
20.49% 16.43%
14.14% 10.52% 16.67% 13.32% 12.15%
10.94% 9.98%
0%
100% 94.12% 92.39%
85.71%
76.60% 71.43%
Core NLP
58.33% 57.38% 59.03%
54.16%
48.02%
50%
35.68% 32.44% 35.96%
25.41% 29.26%
0%
* ** *** **** ***** * ** *** **** ***** * ** *** **** *****
Figure 4. Distribution of SAMs’ negative polarity by user rating in TripAdvisor.
plots (four SAMs for each of three we suggest not setting the user rate as TripAdvisor-based opinions. Although
monuments) containing the shares of a label sentiment for the whole review some related topics have been exten-
all negative SAM polarity over the bub- and analyzing the opinions in depth. sively studied in the literature, their
ble evaluation (percent over the origi- This study clearly shows the need adaptation to the context of TripAdvi-
nal user ratings). Analyzing this data, to analyze opinions beyond user rat- sor opinions requires revisiting them.
we observe that Senti Strength and ings. As a practical methodology, we Aspect-based sentiment analysis (
Syuzhet detect at best 57.48 and 45.10 propose following three steps: handle ABSA) is an important sentiment anal-
percent of negative reviews. However, the negative opinions identified with ysis task.16 An aspect refers to an at-
they misclassify on average 20 percent SAMs via learning models, get a good tribute of the entity, for example, hotel
of positive user reviews (three and four clusterization according to consensus room cleanliness, the staff at a tour-
bubbles). On the other hand, Bing and degrees among SAMs, and discover ist spot, or the service at a restaurant.
CoreNLP methods detect as negative relationships among common aspects ABSA aims to identify the sentiment
more negative user ratings, but misclas- to characterize the cause behind the toward an aspect and extract fine-
sify over 30 percent of positive reviews. negative comments. grained information about specific
Bing and CoreNLP tend to highlight TripAdvisor-based opinions (hotels,
the negative opinions. Challenges monuments, restaurants, and so on).
In general, we observe that users tend Several challenges arise when Trip Recent relevant studies are based on
to write negative sentences on positive Advisor uses sentiment analy- deep learning,17 which should be ana-
user ratings, and vice versa. Therefore, sis, due to the specific content in lyzed in the TripAdvisor context.
JULY/AUGUST 2017 www.computer.org/intelligent 75
ABSA is helpful to business man- applications in the tourist domain, Sentiment Analysis Methods,” EPJ Data
agers because it allows for the ex- sentiment analysis has great potential Science, vol. 5, no. 1, 2016, pp. 1–29.
traction of transparent customer to directly influence quality improve- 7. J. Serrano-Guerrero et al., “Sentiment
opinions. Discovery knowledge tech- ment in tourism. Analysis: A Review and Comparative
niques such as subgroup discovery18 Because of inconsistencies between Analysis of Web Services,” Information
can be applied to discover relation- user ratings and SAM evaluations, Science, vol. 311, Aug. 2015, pp. 18–38.
ships among common aspects and get with users often writing negative sen- 8. E. Cambria and A. Hussain, Sentic
aspect associations for both positive tences in positive opinions and vice Computing: A Common-Sense-Based
and negative opinions. versa, we need new approaches to fix Framework for Concept-Level Senti-
The detection of irony and sarcasm the positive, negative, and neutrality ment Analysis, Springer, 2015.
is a complex sentiment analysis task. via consensus among SAMs, as well 9. F.H. Khan, S. Bashir, and U. Qamar,
The detection of ironic expressions as design models to discover rela- “TOM: Twitter Opinion Mining
in TripAdvisor reviews is an open tionships among common aspects to Framework Using Hybrid Classification
problem that could help to extract characterize the reasons behind nega- Scheme,” Decision Support Systems,
more valuable information about the tive comments. vol. 57, Jan. 2014, pp. 245–257.
study’s subject.19 Spam is another 10. E. Guzman and W. Maalej, “How Do
sentiment analysis-related concern. Users Like This Feature? AFine Grained
Some authors have developed studies Acknowledgments Sentiment Analysis of App Reviews,”
to measure the credibility of TripAd- This work has been supported by FEDER Proc. IEEE 22nd Int’l Conf. Require-
visor with satisfactory results. 20 (Fondo Europeo de DEsarrollo Regional) ments Eng., 2014, pp. 153–162.
and the Spanish National Research Project
A novel approach is the extraction 11. D. Gräbner et al., “Classification of
TIN2014-57251-P.
of aspects/features from opinions Customer Reviews based on Sentiment
to raise the issue as a bag of feature Analysis,” Information and Comm.
vectors, considering the problem as References Technologies in Tourism, Springer,
multi-instance learning. 21 This might 1. P. Andersen, What Is Web 2.0? Ideas, 2012, pp. 460–470.
provide a robust approach from the Technologies and Implications for Edu- 12. S. Palakvangsa-Na-Ayudhya et al.,
classification viewpoint. cation, tech. report, JISC Technology & “Nebular: A Sentiment Classification
Standards Watch, 2007; www.ictliteracy System for the Tourism Business,” Proc.
S
.info/rf.pdf/Web2.0_research.pdf. 8th Int’l Joint Conf. Computer Science
entiment analysis is an incipi- 2. E. Cambria, “Affective Computing and Software Eng., 2011, pp. 293–298.
ent research field. It is difficult to and Sentiment Analysis,” IEEE Intel- 13. E. Marrese-Taylor, J.D. Velásquez, and
determine how it will evolve in the ligent Systems, vol. 31, no. 2, 2016, F. Bravo-Marquez, “A Novel Determin-
future, although there is a general pp. 102–107. istic Approach for Aspect-Based Opinion
belief that this analysis needs to go 3. B. Liu, Sentiment Analysis: Mining Mining in Tourism Products Reviews,”
beyond a simple classification of Opinions, Sentiments, and Emotions, Expert Systems with Applications, vol.
texts on a positive and negative one- Cambridge Univ. Press, 2015. 41, Dec. 2014, pp. 7764–7775.
dimensional scale. Over the last few 4. World Travel and Tourism Council, 14. M. Thelwall, “Heart and Soul: Senti-
years, the list of sentiment analysis- Travel & Tourism Economic Impart ment Strength Detection in the Social
related challenges has grown (subjec- 2016 World, 2016; www.wttc.org/- Web with SentiStrength,” Cyberemo-
tivity classification, opinion summa- /media/files/reports/economic%20 tions: Collective Emotions in Cyber-
rization, opinion retrieval, and so on). impact%20research/regions%202016 space, Springer, 2017, pp. 119–134.
Through Web platforms such as /world2016.pdf. 15. M. Hu and B. Liu, “Mining and
TripAdvisor, tourists can openly de- 5. P. O’Connor, “User-Generated Summarizing Customer Reviews,”
scribe their experiences and thus af- Content and Travel: A Case Study on Proc. 10th ACM SIGKDD Int’l Conf.
fect a business’s viability. Therefore, TripAdvisor.com,” Information and Knowledge Discovery and Data Min-
the implementation of sentiment Comm. Technologies in Tourism, ing, 2004, pp. 168–177.
analysis techniques to mine sources P. O’Connor, W. Höpken, and U. Gret- 16. K. Shouten and F. Frasincar, “Survey on
of opinion is crucial to understand- zel, eds., Springer, 2008, pp. 47–48. Aspect-Level Sentiment Analysis,” IEEE
ing the faults and assets of a tourist 6. F.N. Ribeiro et al., “SentiBench: ABench- Trans. Knowledge and Data Eng., vol.
service. Given the large number of mark Comparison of State-of-the-Practice 28, March 2016, pp. 813–830.
76 www.computer.org/intelligent IEEE INTELLIGENT SYSTEMS
17. S. Poria, E. Cambria, and A. Gelbukh, Influence on Recommendation at University of Granada. Her research
“Aspect Extraction for Opinion Min- Adoption and Word of Mouth,” Tour- interests include sentiment analysis, arti-
ing with a Deep Convolutional Neural ism Management, vol. 51, Dec. 2015, ficial intelligence, and computer graphics.
Network,” Knowledge-Based Systems, pp. 174–185. Luzón has a PhD in industrial engineering
vol. 108, Sep. 2016, pp. 42–49. 21. F. Herrera et al., Multiple Instance from the University of Vigo. Contact her at
18. M. Atzmueller, “Subgroup Discovery,” Learning: Foundations and Algo- luzon@ugr.es.
Wiley Interdisciplinary Reviews: Data rithms, Springer, 2016.
Mining and Knowledge Discovery, Francisco Herrera is a full professor in
vol. 5, no. 1, 2015, pp. 35-49. Ana Valdivia is a research fellow in the the Computer Science and Artificial Intel-
19. S. Poria et al., “A Deeper Look Computer Science and Artificial Intelligence ligence Department at the University of
into Sarcastic Tweets Using Deep Department at the University of Granada. Granada. and head of the Soft Comput-
Convolutional Neural Networks,” Her research interests include big data, so- ing and Intelligent Information Systems
Proc. 26th Int’l Conf. Computational cial media analysis, and artificial intelli- research group. His research interests in-
Linguistics (COLING16), 2016, gence. Valdivia has an MsC in data science clude data science, data preprocessing,
pp. 1601–1612. from the University of Granada. Contact big data, sentiment analysis, and compu-
20. R. Filieri, S. Alguezaui, and F. McLeay, her at avaldivia@ugr.es. tational intelligence. Herrera has a PhD
“Why Do Travelers Trust TripAdvi- in mathematics from the University of
sor? Antecedents of Trust Towards M. Victoria Luzón is an associate profes- Granada. Contact him at herrera@decsai
Consumer-Generated Media and Its sor in the Software Engineering Department .ugr.es.
JULY/AUGUST 2016
IEEE SOFTWARE
July/August 2016
SOFTWARE JANUARY/FEBRUARY 2016 IEEE Software offers
QUALITY pioneering ideas,
IEEE SOFTWARE
WWW.COMPUTER.ORG/SOFTWARE
SOFTWARE QUALITY
expert analyses, and
January/February 2016
thoughtful insights for
ARCHITECTURAL DESIGN PRINCIPLES // 15
software professionals
WHY LARGE IT PROJECTS FAIL // 117
Volume 33 Number 4
who need to keep up
THE FUTURE OF SOFTWARE ENGINEERING
MARCH/APRIL 2016
WWW.COMPUTER.ORG/SOFTWARE
with rapid technology
IEEE SOFTWARE
WWW.COMPUTER.ORG/SOFTWARE
TINY PROGRAMMING TOOLS // 24
REQUIREMENTS
& SOCIAL RESPONSIBILITY // 109
change. It’s the authority
March/April 2016
CODE CLARITY // 22
on translating software
theory into practice.
SOFTWARE
Volume 33 Number 1
ON A COMET // 81
BIG DATA
www.computer.org/
software/subscribe
Volume 33 Number 2
JULY/AUGUST 2017 www.computer.org/intelligent 77You can also read
