This research identifies and quantifies discussion themes in Twitter data in order to investigate what indicators can help understand people’s perceptions and concerns around food, fuel, finance and housing in the US and Indonesia.
Crimson Hexagon (www.crimsonhexagon.com)
The purpose of this research project is to determine which indicators might be present in social media data that could shed light on how populations cope with global crises, such as commodity price volatility or the continuing global economic crisis.
In this investigation, the analysis was limited to publicly available data from Twitter for July 2010 through October 2011 in Javanese/Bahasa Indonesia and English. The topics of focus included the affordability/availability of food, fuel, housing and loans.
By classifying a populations’ tweets into several categories associated with relevant topics, it was possible to perform quantitative analysis to better understand populations’ concerns:
The number of tweets discussing the price of rice in Indonesia over the last year follows a similar function as the official inflation statistics for the food basket.
This research has confirmed that Twitter data can be useful for understanding the immediate worries, fears and concerns of populations, but at the same time, the research suggested that it is a poor source of data for gauging people’s long term aspirations. There are several remaining challenges, in particular that Twitter has a specific culture and demographic which needs to be better understood. Overall, this exploratory research shows some of the potential of Twitter data for exploring people’s perceptions of crisis-related stress and suggests research lines and methodologies for further investigations.
Conversations around finance in the US, modulated by a baseline weekly pattern of fewer discussions on the weekends, show an increase of conversations from the 15th July to the 15th August motivated by the US debt ceiling debate.
To delve deeper into the Twitter data analyzed in Bahasa and Javanese, download this background annex (PDF), which provides example Tweets and the taxonomies which were developed about food, fuel, finance, and housing. The sample Tweets represent the type of material used to train machine learning algorithms to classify the Tweets into the different categories.