The objective of this ongoing collaborative project is to experiment with new tools that can replicate the standards of traditional household surveys, in real-time on a global scale. In particular, Jana and Global Pulse are testing the potential of deploying mobile surveys, over SMS, in multiple countries. The project will also explore the feasibility of conducting longitudinal surveys over time and the possibility of reaching underrepresented populations that have access to mobile phone technology.
Household surveys have long been the gold standard for collecting data on populations, both for national statistics, impact assessment, and programme monitoring & evaluation. However, household surveys are burdensome and costly, and thus difficult to scale. Mobile-based surveys have already shown, in recent years, their potential to complement traditional data collection at the country-level.
For this project, Global Pulse partnered with Jana to conduct a survey at global scale.
In its network, Jana has over 2 billion mobile subscribers who have opted into a process where they will answer survey questions in exchange for a small amount of airtime. Since the majority of plans in the developing world and emerging markets rely on pay-as-you-go fees, this method has the potential to cover a great deal of the world’s mobile users.
Global Pulse collaborated with Jana to develop a set of questions about well-being and interconnectedness that were translated into 15 languages, and sent over mobile phones directly to citizens more than 30 countries, collecting more than 90,000 answers between August and November 2011.
The short video below (produced by Paul Butler, as a part of Data Without Borders) demonstrates the rapidity of data collection by showing the number of responses collected from July 25 to October 16, 2011.
Most of the questions were multiple-choice, with participants pressing a number on their keypad corresponding to an answer. Some of the questions were free-text, but expected a numerical answer (i.e. “How many days did you work in the last 7 days?”). Finally, a very few of the questions were free text and required a text answer (i.e. “If you had 15 USD, how would spend it?”).
Below is a sample of the survey questions we asked, clustered by theme:
Questions about Well-Being
- How did you feel over the past 7 days? Answer Options: 1 – Great, 2 – Good, 3 – Bad, 4 – Neither good nor bad, 5 – Very Bad
- How many days did you work in the past 7 days? (Free Text)
- Were you sick in the past 7 days? Answer Options: 1 – No, 2 – Yes but did not need treatment, 3 – Yes and received treatment, 4 – Yes and needed medical treatment but did not receive it
- If you had 15 USD, what would you spend it on? (Free Text)
Questions about Interconnectedness:
- How much money did you spend on airtime in the past 7 days? (Free Text)
- In the past 7 days, did you communicate with friends and family using: Answer Options (please indicate all that apply): 1 – Face to Face, 2 – Mobile (voice), 3 – Mobile (SMS), 4 – Landline, 5- Internet (email), 6 – Internet (social networks)
- In the past 7 days, did you communicate with anyone (please indicate all that apply: Answer Options: 1 – Within walking distance, 3 – In your city/village, 4 – In your province/state, 5 – In your country, 6 – Out of your country
- Have you ever purchased goods or transferred money using your mobile phone? 1- Yes, 2- No
- Have you ever purchased goods or services over the Internet? 1- Yes, 2- No
—
A visualization of a partial subset of the answers was included in the “PopTech: World Rebalancing” iPad app earlier this year.
The PopTech app provided an opportunity for Global Pulse to begin visualizing some of the results of the survey. While the answers that were visualized in the app are not a representative sample (in fact, some countries only yielded less than 50 responses), this exercise was an excellent opportunity to start thinking creatively about how to present the results of the larger survey, in an interactive format.
The next step for the Global Pulse/Jana collaboration is to perform a more in-depth analysis of the results to date and eventually conduct similar surveys over time in order to map the evolution of trends in well-being over time, continue to establish demographic baseline data (including information about economic status, gender, age, literacy, etc.) and to see what works, what doesn’t work and why.
