This guest blog post was produced by Giselle Lopez and Wayne St. Amand of Crimson Hexagon. Global Pulse worked with Crimson Hexagon on a project exploring Twitter and Perceptions of Crisis Related Stress. It is part of a series detailing the open innovation process undertaken by Global Pulse and our partners on a number of collaborative “proof of concept” research projects exploring the utility of various new, digital data sources to answer traditional development related questions. Check out the first post from the series, interview with a Global Pulse data scientist for background.
In this age of Big Data, it is essential for organizations to understand the networks that develop within and between communities and to contribute to the conversation. The UN Global Pulse initiative has demonstrated a dedication to innovation that will help policymakers and institutions navigate the rapidly changing landscape of communication networks (both on- and off-line) around the world to more effectively address social issues.
Along with the expansion of mobile phone use, individuals have acquired use of other digital technologies and communication platforms to build and connect with networks and communities. The use of social media platforms including Twitter and Facebook has increased at an astounding rate, connecting individuals and communities and allowing for the rapid spread of information. The content these platforms are creating is now considered a form of Big Data. Furthermore, this global expansion of social media use and digital networks has inspired the development of technologies to intelligently analyze the Big Data produced by these interactions. These technologies have created new possibilities for companies and organizations to assess public opinion in new ways and react more effectively and efficiently to conversation trends.
At Crimson Hexagon, we were thrilled to work alongside a leader in international development and social science research to gain insight into the use of social media analytics in development and policymaking. The Crimson Hexagon platform is traditionally used for commercial-based research by global brands, agencies, and media organizations to engage with and better understand broad consumer trends. During this project, we leveraged the platform to analyze public opinions about pressing social and economic concerns that people face on a daily basis. We sought to assist Global Pulse in understanding how “digital smoke signals” can be detected through the world of social media and the factors that should be accounted for to produce an accurate and actionable assessment of emerging issues in a population.
For the project, we looked at Twitter conversation over two years beginning July 2010 in Indonesia and the United States. With the rapid expansion of Twitter use in the Indonesian population over the past two years, Indonesia was an ideal location for research into social issues as they are discussed in social media. Over the course of our collaboration, we developed an understanding of the importance of context in social media analysis at the international level.
When conducting social media research internationally, there is a need to account for the language, location, and cultural context of conversation. Where language is concerned, Crimson Hexagon’s ForSight platform is language-agnostic: as long as the researcher who trains the software understands the language, the software will accurately assess results through its text-parsing technology.
Beyond language, there is a need for context that can help researchers distinguish between relevant and irrelevant conversation and effectively analyze results of the research. A significant proportion of Twitter conversation is spam, advertisements, jokes, news, and disengaged chatter. Because much of this conversation will have little to no relevance to an assessment of social concerns, it is important for analyses to capture the distinction and recognize irrelevant conversation as such. For this reason, smart technologies are essential for organizations such as the Global Pulse to identify trends in public opinion that are valuable for development, policymaking, and crisis response.
Through our collaboration with the UN Global Pulse, we also discovered the importance of maintaining broad objectives when conducting research into public concerns. Because conversation of social issues is inherently dynamic and ambiguous, this type of research is less targeted than research into other types of social media conversation. While knowledge about a population’s most pressing social issues is necessary to frame initial research objectives, certain assumptions about the actual nature of conversation can hinder the development of the research. Through a broad initial approach and with the Global Pulse’s knowledge about the primary issues in the population, we identified four general categories of socio-economic concerns: housing, gas and fuel, personal finance, and food. From these broad areas of focus, we then looked deeper into the data to identify differences in conversation about these topics in Indonesia and the United States. This methodology allowed the UN Global Pulse to identify trends at various stages and to adapt research objectives accordingly.
While analyses of the results often confirmed intuitive patterns and common sense (such as an increase in housing conversation around the first of the month), the nuances, anomalies, and correlations in the results showed that this research can be used to identify unique trends in the online conversation surrounding these kinds of events. When compared alongside outside (non social media based) research data, which reflected a static perspective, the online research results revealed more real-time perceptions of events and shifts over time in the reaction to longer-term changes.
Going forward, we hope to continue to assist in advancing the use of social media analytics in social science research and crisis response. We look forward to further expanding the scope of social science research through future collaborations with the UN Global Pulse team and with the broader academic and nonprofit community.
[Image: Heat map of tweets about food in Bahasa Indonesia/ Javanese from July1, 2010 to October 25, 2011.]
Find a project summary, methods paper, and video presentation of the project at: https://www.unglobalpulse.org/projects/twitter-and-perceptions-crisis-related-stress
