Twitter, Politics, and Sentiment Analysis

I had an interesting conversation with a journalist recently that made me think a bit more on the use of social data for decision support, the accuracy of socially generated big data and the ability to apply current technologies to turn that data into useful information. The topic was the potential to use socialytics in the current US Presidential race, specifically the Twitter stream, but the concepts apply much more broadly. The 2008 presidential race was the first time that US candidates used social networks and social media in a campaign, and that practice has expanded quite a bit in the current campaign efforts. Twitter, in particular, has gotten a lot of attention as both a medium for creating constituent engagement and also as a way for candidates to get a real time look at public opinion on any given issue or candidate. In some ways, Twitter is like a real time version of an exit poll with a massive increase in sample size.

There seems to be some doubt around the value of analyzing the Twitter data stream and over what specifically that analysis has the capability to show. I’m a little surprised by this, since the technology, text analytics, and the validity of applying analysis to massive data sets, is advanced and widely accepted. It seems to me that there are several use cases for Twitter in the context of a political campaign, including real time feedback on public sentiment on an issue or candidate, a measure of sentiment around a specific debate and even a predictor of outcomes. The questions seem to be: 1. does the measure of sentiment on Twitter accurately reflect the sentiment of the broader population when politics is the subject, 2. is Twitter used to discuss political issues or is it simply a vehicle to spread opinions, 3. can Twitter be used to accurately predict the outcome of an election?

There are several studies that have examined Twitter’s effectiveness as a tool in a political campaign and often the questions raised above were a basis for conducting the research. In a 2010, for example, a study conducted by the Technical University of Munich and presented at the AAAI conference, looked at Twitter as a predictive tool for election outcomes. The study used the tweets leading up to the federal election of the national parliament (Sept 27th, 2009) and used a form of the three questions I raised above to see if there was a correlation between the sentiment analysis of the tweet stream and the election outcomes. The study concluded:

1. Twitter is used as a platform for political deliberation
2. The sentiment profiles of candidates and parties reflected in the tweet stream accurately represented the actual sentiment of the population at large.
3. The number of tweets reflected the election results and was comparable to traditional election polls.

In another study, researchers at Carnegie Mellon University analyzed the tweet stream from 2008-2009 and compared them to surveys on political issues and consumer confidence and found a similar correlation to the German study. There are a few other studies with similar conclusions, but I think you get the point, Twitter does reflect the broader political sentiment, it is used to discuss politics, not just broadcast opinion and it can be used to predict election outcomes…at least according to the body of research so far.

So from a political campaign perspective how can candidates use Twitter, and can those uses be applied to business? Here are a few use cases that I think fit both politics and business use:

• Real time sentiment analysis around public issues. For politicians this is a powerful tool that reaches far beyond traditional polls and surveys, which have a lag (sometime significant lag) in response time. This means that a politician could measure sentiment around any public statement or issue and adjust messaging in real time. Leaving the ethical judgment aside for now, this would mean that a candidate could tweak messages in fine detail all through a campaign (I’ll just chalk this up to communication clarity, not to pandering). For a business, this could be used for brand sentiment, particularly around a crisis where clarity of message is business critical. By monitoring sentiment a company could adjust PR / communications to more effectively defuse a situation or adjust marketing / communications to provide a clearer message.
• Twitter is not only a broadcast channel but it can also foster discussion around an issue. For a candidate this could be used to clarify public opinion on an issue, collect feedback and even crowdsource some potential solutions. For businesses Twitter can serve as one channel for collecting customer feedback, ideas and even support.
• Twitter can, with reasonable accuracy, predict the outcomes of elections. I do think that this is a little less useful than the real time use a sentiment analysis tool, but still, it has proven reasonably accurate. I also don’t want to infer that normal statistical principals don’t apply, they do, this is just one more method to find useful information. There will be inaccuracies and bias, just like any other survey, sample, etc. That said, predictive capabilities could be applied to any public issue that has fairly widespread appeal. Twitter’s accuracy, at least in part, is related to the shear volume of information that can be pushed through an analysis engine. This is tied to one of the key factors around the value of big data, large data sets with a broad representation.

There are issues with any type of analysis of course, but from what we’ve seen so far, there are opportunities to apply sentiment analysis to social channels like Twitter to gain insight. It’s not the only data source, but it is one interesting point in making / supporting decisions. What do you think, how could analysis of large social data sets be used to support business?