When you pick a restaurant to eat at or order a product on Amazon, how often do you read the reviews first?
If you're anything like most people, chances are you're reading reviews every single time. Online reviews are incredibly influential in a customer's decision to buy.
Most companies try to keep an eye on their reviews, but they're usually limited to the structured part - the star rating or the recommendation score - because the text explanations are hard to analyze and quantify. Those high-level metrics are helpful, but only in a limited sense. The meat of the feedback is in the open-ended text customers are submitting.
As a business, finding effective and scalable ways to analyze unstructured text-like in reviews - makes it possible for you to understand your customers more fully. Sentiment analysis is a prime example of this. When done right, sentiment analysis unlocks a whole world of color and context about how your customers view your business.
What is sentiment analysis?
Sentiment analysis is a text analytics technique in which an algorithm reads text and identifies the feeling and emotions behind it.
In its simplest form, it classifies text as positive, negative, or neutral:
- "The food was great!" would be positive.
- "The service was awful" is negative.
- "The service was okay and the food was fine" is neutral.
More advanced sentiment analysis involves trying to detect specific emotions, such as happiness, sadness, anger, or surprise. When combined with other techniques like text classification, sentiment analysis can really enrich the data you're working with, providing more actionable insights for your business and helping you more accurately measure customer experience.
Sentiment analysis has a ton of different potential applications, including social media and brand monitoring, understanding customer reviews, and analyzing customer support tickets. In short, it may help any place where you get unfiltered, open customer feedback (more on this below).
Sentiment analysis shows you which topics are driving negative and positive feedback from your customers, surfacing where you need to improve and where you're already successful. And it does all of that automatically, without you needing to read through thousands of comments and categorizing them manually.
Sound almost too good to be true?
The pitfalls of customer sentiment analysis (or when not to use it)
Sentiment analysis is powerful, but it's not 100% perfect.
If you've done a lot of written communication before, you'll know that identifying emotions in text is not as easy as it sounds (even when you're a person, who theoretically should know the nuances of language). Just think about the last time you struggled to interpret what someone said in a text message.
Let's take this sentence as an example: "Great, my flight got canceled!"
A computer might interpret this sentence as positive. If the person saying it was really dreading that flight, that interpretation might even be correct. But it's much more likely that this comment was meant sarcastically and the sentiment is actually negative. This is the type of nuance that an algorithm can't easily pick up on.
There are a few scenarios where sentiment analysis data might give you wrong information, such as sarcasm, the use of jargon, and double negatives.
There are other situations where sentiment analysis might not give you much new information. Say you analyze responses to a positive question like, 'What are your favorite features of our product?" You'll also end up with mostly positive sentiment data.
That doesn't mean that there's no value to sentiment analysis. It just means that some applications are more suitable for it then others, and that there's a right way to approach working with sentiment analysis data.
Here's how to do it right.
The right way to measure customer experience with sentiment analysis data
To measure customer experience using sentiment analysis in a way that's actually helpful for your business, there are four steps you should follow:
Choose an appropriate use case.
Decide how you'll use the data.
Ask open-ended questions.
Invest time in quality control.
1. Choose an appropriate use case for sentiment analysis data
You don't use a screwdriver when you need a hammer. Likewise, only use sentiment analysis where it makes sense. It's a waste of time to use sentiment analysis if you know up front it's going to provide little to no helpful information.
Here are some circumstances where sentiment analysis is usually not appropriate:
If you're dealing with very short and sparse responses. It's typically hard for an algorithm to identify sentiment accurately if responses are only a few words long.
If you're working in a narrow or highly technical domain with a distinct vocabulary. For example, if you're selling software for auditing compliance with GDPR and your data is filled with technical and legal jargon, a general-purpose sentiment model will have a hard time processing feedback from your customers.
If the context of the question is already clearly positive or negative. As mentioned above, analyzing the sentiment to questions like "What didn't you like about your experience with us?" won't give you new insights.
Sentiment analysis works best when you're working with truly open-ended data. When you can't identify the sentiment based on the context, sentiment gives you insight into how your customers feel (and how strongly they feel those emotions).
A good example is when customers submit requests to your support team. In this situation, emotion detection can pick up on things like when customers are angry or outraged, enabling you to prioritize or route their tickets differently. Insights like these can have a meaningful impact on business metrics, such as reducing customer churn.
2. Decide how you'll use the sentiment analysis data
When you're measuring customer experience, sentiment data can be complementary or primary.
Sentiment as a complementary metric
The ideal use of sentiment is as complementary data. It should add more context to existing metrics. It's supplemental, giving color to the things you're already looking at.
For example, if you run an NPS survey, NPS will be the primary metric you're trying to influence. Sentiment analysis should be used to enrich the insights you get from your data, providing an additional dimension for understanding your customers.
Using sentiment in this way means you won't be misled by a wrongly identified sentiment. For example, a highly satisfied customer might say 'I can't get service this good anywhere else!' A response worded like this could be labeled as negative by an algorithm, although humans can tell at a glance that it's positive feedback.
This kind of mistake isn't that important if you're using sentiment to enrich other data, because it exists simply to provide context. You're not going to start making business decisions based purely on a sentiment score'at least, without doing significant analysis of the text first, where you'll catch mistakes like these.
Sentiment as a primary metric
Because sentiment data can't be 100% accurate, it's hard to use it as a primary metric.
There are some situations where making sentiment a primary metric might be appropriate. For instance, take social media analysis, where you're often working with limited data to begin with. You don't really have a top-level primary metric (like NPS), so in this case, sentiment might be noisy but can still be helpful.
You should decide from the beginning what level of change in sentiment is considered meaningful for your business. Typically, small variations in sentiment are likely to be insignificant. If you're only analyzing a small volume of data, it's also likely that you may see significant swings in sentiment over time.
3. Ask open-ended questions
Open-ended questions are where you'll get the most value out of sentiment analysis.
When you're designing a survey, you generally want a good mix of close-ended and open-ended questions. Close-ended questions are often tied to specific metrics, like NPS or CSAT on a given area. These can give great quantitative data, but the real gold mine in your customer feedback comes from open-ended questions.
The best survey questions should:
Encourage your customers to describe their experience factually.
Avoid biasing the customer towards a specific aspect of their experience.
Lead to an open dialogue where possible.
This is why the standard NPS survey is so effective. Combining the recommendation score (a close-ended question) with the very open-ended 'Why?' leads to the best insights.
4. Invest time in quality control of sentiment analysis data
The last step to using sentiment analysis data properly is to take the time to validate your results. Even the best sentiment models will make mistakes, and it's good to make sure that these mistakes won't impact your decision-making.
The best way to do this is to check a sample of the data regularly. A good stress test on your sentiment analysis tool is to look at the extremes. In the case of NPS, this might be:
Promoters with negative sentiments
Detractors with positive sentiments
Since this is the opposite of what you'd expect (Promoters are generally positive, Detractors are negative), you'll probably uncover opportunities for your sentiment analysis model to improve.
You might find your sentiment model is confused by linguistic quirks like double negatives. Or that your most satisfied customers still have lots of things they don't like and are prepared to tell you about them in great detail. You might also find that sentiment is more accurate for some segments than it is for others, such as if you have a population of customers for whom English is not their first language (and your survey is in English).
Finding flaws doesn't invalidate the model as a whole, but it does mean you should be careful when you're interpreting and reporting on the results. This might mean not taking small variations in sentiment too seriously and only paying attention to larger changes.
Combining sentiment analysis data with an analytics toolkit
Sentiment analysis is most valuable when it's one of many tools that you're using to analyze customer data and generate insights. It's a key way to drill down and measure the core aspects of your customer experience.
Investing in feedback analytics lets you combine multiple tools at the same time, so you can use sentiment analysis data properly and build a complete understanding of your customers.
At Kapiche, we've developed a unique method that enables you to pull actionable insights from large data sets in no time. If you want to level up your Voice of Customer program or start exploring sentiment analysis, book a demo with us today!
