Using Data Science on TripAdvisor Reviews (Part 1)

This will be the first of a series of posts that show the knowledge that may be extracted from TripAdvisor (or similar websites) using Information Extraction, Data Mining and Text Analytics.

This will be the first of a series of posts that show the knowledge that may be extracted from TripAdvisor (or similar websites) using Information Extraction, Data Mining and Text Analytics. For me, it has always been great to find applications that combine several analytical techniques to extract useful knowledge.

TripAdvisor contains a wealth of information on experiences of travelers from hotels and restaurants all over the world and I decided to try analyzing this information to extract actionable knowledge from thousands of reviews.

As a case study I analyzed 18000 reviews from hotels in one of the Top Travel Destinations in the world, Santorini:

TripAdvisor reviews contain an overall score, scores on Cleanliness, Value for Money, Sleep Quality and Service. However this is very basic knowledge because text from reviews contains information on the details of the experience and the feelings that this experience has generated such as:

–How great the welcome drinks upon arrival felt.
-The breathtaking view of the Caldera

but also

-The fact that there was dripping water from the faucet which wouldn’t stop all night
-The pillows that were “hard like a sandpaper”
-The “tiny” bathroom
-The unexpected charges for coffee and tea

We start with a Key Question: Which Topics are discussed in Highscore reviews and which topics are found in Negative Reviews? (We consider a Positive Review a 5 stars rating, anything less is Negative). Let’s take a look at the following Bar chart which shows the Frequencies(%) of Topics being discussed in each Category:

Notice how often the topic “Service” has been discussed in Positive reviews but it is almost completely lacking from Negative Reviews. “Bathroom” on the other hand is found almost entirely in Negative Reviews. HIGHPOS is a Topic which includes words and phrases that are extremely positive such as awesome, breathtaking, once in a lifetime, fantastic.

So far we can hypothesize that mentions for Bathrooms are commonly found in Negative Reviews.

Next, we use a Text classifier to identify which words (not topics) are found to positive reviews (shown in green) and which on negative reviews :

team	0.468
spotless	0.411
champagne	0.398
moment	0.386
thank	0.355
delicious	0.344
hospitality	0.295
decoration	0.292
patio	0.286
memorable	0.282
wine	0.266
welcoming	0.249
shower	-0.249
bad	-0.266
overall	-0.276
paid	-0.282
said	-0.284
told	-0.295
bathroom	-0.306
sleep	-0.333
basic	-0.454

Note that we cannot be sure about the context for which these words refer to but we can understand in most cases after a bit of searching and co-occurrence analysis: A spotless place, complimentary champagne, delicious Food, a Nice Breakfast and true Greek Hospitality are the way to success. On the other hand Basic facilities, problems with having a restful Sleep, a Shower which doesn’t work as it should and unpleasant smells from the Bathroom, all these issues could lead to a non-favorable review.

Information Extraction enables us to find specific states, situations and behaviors hidden inside each Review. These could be:

– That some Hotel owners charged their visitors for Security Keys (and this was not received positively)
– That Stray Cats are not always welcome
– Which elements create a Perfect Romantic Sunset or the most Memorable Wedding Ceremony
– Which experiences generate intense positive feelings.

And the list can go on:

– The Bathroom was rather small

– Lack of privacy

– Noise from Slamming doors and Airplanes

– Quality of Bed / Pillows / Mattress

– Hospitality
– Reception of new Guests / Friendliness

Next, with Information Extraction we can generate a Table that maps each Review along with the Topics and incidents/events that have been mentioned in each Review:

Each ‘F’ means that the relevant Topic has not been found within a Review while a ‘T’ means that this Topic was discussed. We can analyze this table to find which Topics / Incidences and different elements of the Customer Experiences (Room Service, Beds, Welcome Drinks, Breathtaking view, etc) are important in creating a memorable experience and thus a good rating and Top position on TripAdvisor.

The following example shows how this information is extracted. A Predictive Model scores which Topics are discussed in Reviews with Highly Positive words (but the same can be done for High score reviews):

Oia is an area in Santorini where one can experience a truly amazing sunset. Caldera is also another area. Notice that View is the single most important element (but it should be pointed out that several other topics have been removed from the analysis shown above). Smell(s) obviously is not part of a unique experience but the graph simply says that this element should *not* be a part of the experience! (the same applies for Uncomfortable, Small_Bathroom, Noise and several other Topics)

In the next post: More examples of how this knowledge can effectively be used to give a competitive edge in the Hotel and Tourism Business.