Twitter and Perceptions of Crisis-Related Stress
Description:
This research identifies and quantifies discussion themes in Twitter data in order to investigate what indicators can help understand people’s perceptions and concerns around food, fuel, finance and housing in the US and Indonesia.
Partner:
Crimson Hexagon (www.crimsonhexagon.com)
Overview:
The purpose of this research project is to determine which indicators might be present in social media data that could shed light on how populations cope with global crises, such as commodity price volatility or the continuing global economic crisis.
In this investigation, the analysis was limited to publicly available data from Twitter for July 2010 through October 2011 in Javanese/Bahasa Indonesia and English. The topics of focus included the affordability/availability of food, fuel, housing and loans.
By classifying a populations’ tweets into several categories associated with relevant topics, it was possible to perform quantitative analysis to better understand populations’ concerns:
- detecting anomalies such as spikes or drops in the number of tweets about particular topics (e.g. comments about power outages in Indonesia or student loans in U.S.),
- observing weekly and monthly trends in Twitter conversations (e.g. discussions around debt in U.S.),
- finding patterns in the volume of particular topics over time (e.g. discussions around housing in U.S.),
- comparing the proportions of different sub-topics to understand shifts in trends over time (e.g. the ratio of tweets about formal loans vs. informal loans in Indonesia),
- or relating trends in Twitter conversations with external indicators (e.g. conversations around the price of rice in Indonesia mimicking the official inflation statistics).
The number of tweets discussing the price of rice in Indonesia over the last year follows a similar function as the official inflation statistics for the food basket.
This research has confirmed that Twitter data can be useful for understanding the immediate worries, fears and concerns of populations, but at the same time, the research suggested that it is a poor source of data for gauging people’s long term aspirations. There are several remaining challenges, in particular that Twitter has a specific culture and demographic which needs to be better understood. Overall, this exploratory research shows some of the potential of Twitter data for exploring people’s perceptions of crisis-related stress and suggests research lines and methodologies for further investigations.
Conversations around finance in the US, modulated by a baseline weekly pattern of fewer discussions on the weekends, show an increase of conversations from the 15th July to the 15th August motivated by the US debt ceiling debate.
Learn More about the Project Methodology:
To delve deeper into the Twitter data analyzed in Bahasa and Javanese, download this background annex (PDF), which provides example Tweets and the taxonomies which were developed about food, fuel, finance, and housing. The sample Tweets represent the type of material used to train machine learning algorithms to classify the Tweets into the different categories.