This week, our partners from SAS released a press release summarizing the findings from our collaborative research project, in which we worked together to investigate whether there are useful signals in online sentiment that could give early-warning indicators of rising unemployment, or indicate how people cope with unemployment. Two years worth of social media data (half a million blogs, forums and news sites) from the US and Ireland were analyzed for references to unemployment and how people were coping.
The key findings were summarized in the press release and an accompanying infographic (above):
…SAS compared mood scores and conversation volume with official unemployment statistics to see if upticks in those topics were indicators of spikes in unemployment. The analysis revealed that increased chatter about cutting back on groceries, increasing use of public transportation and downgrading one’s automobile could, indeed, predict an unemployment spike.
After a spike, surges in social media conversations about such topics as canceled vacations, reduced health care spending, and foreclosures or evictions shed light on lagging economic effects. Such information could be invaluable for policymakers trying to mitigate negative effects of increased unemployment.
…In the US, a rise in “hostile” or “depressed” mood occurred four months before the unemployment spike. Increases in “anxious” unemployment chatter in Ireland correlated with an unemployment spike five months later. Increased “confused” chatter preceded the spike by three months, while “confident” chatter decreased significantly two months out. A dashboard displayed results, including trends, moods on unemployment expressed in social media, mood change over time, and leading and lagging indicators of unemployment shocks.
You can learn more about the analysis by visiting the project page on our website, where you can download the methods paper from the research, or by watching this short video presentation of the research findings at a technical briefing hosted at the UN in December 2011:
We embarked on a series of four additional similar “proof of concept” research projects in 2011 with the hope that they may eventually help establish new methodological approaches for analyzing real-time data, identify tools which can provide a faster and clearer understanding of population behavior during periods of stress, and hopefully contribute to the development of new proxy indicators for real-time tracking of development.
These preliminary findings represent just the beginning of what new, real-time, data sources can tell us about human well-being. A great deal more work and investigation, particularly around validation, sample size and causality, still needs to be conducted, as we enter this emerging field of study. We look forward to working closely with data scientists, development experts, and others to bring these methods to a more robust state.
As our colleague I-Sah Hsieh notes in the SAS blog:
Could social media analysis replace the need for official statistics? Not in my opinion. Social media analysis is an excellent complement to official statistics, often bringing more insights. For example, official statistics may tell us that there is a decline in the number of children enrolled in school in a developing region, and the public chatter may tell us why children are being pulled out of schools (to work in the local markets for food, to protect the village from civil unrest, etc.) And, as shown in the project with UN Global Pulse, the analysis can also determine the types of chatter that usually precede or follow an event captured by official statistics.
