Fantastic post, Robert. Very clear, precise examples like this one provide a concrete puzzle that everyone in the broader community can swarm around. Starting a brainstorming session with precisely this post would yield many concrete next steps I think. Congratulations to the Global Pulse team for your incredible progress!
Digital Smoke Signals
Over the years, governments, development organizations and the private sector have invested untold billions in creating mobile phone-powered programs and services with the potential to help people lift themselves out of poverty and realize their dreams.
Wherever people are using mobile phones to access digital services and participate in development programs, they are leaving trails behind in the data. We call this "data exhaust," and people everywhere are contributing to an ever-widening ocean of it, for free, merely by going about their daily lives. The private sector is now using innovative technologies to analyze data exhaust from commercial services to understand their customers, identify new markets, and make investment decisions. The time has come for policymakers to begin using these new kinds of data, tools and approaches to protect communities from multiple slow-onset crises that threaten to reverse hard-won progress in human development.
Mobile Services as Crisis Monitoring Networks
It's our working hypothesis here at Global Pulse that when people begin to be impacted by slow-onset crises, they change how they use services and participate in programs. We think of these changing patterns of usage as potential "digital smoke signals" that could alert policy makers that vulnerable populations are in trouble. Once we figure out a way to anonymize and aggregate this data to protect both the individuals and organizations involved, the implications are profound. What if we could find a way to transform all of these existing programs and services that people use to better their lives into dual-purpose sensor networks capable of detecting the early impacts of slow-onset crises?
The Invisible Descent into Poverty
When a crisis hits, affected populations cope by changing their collective behavior in response. Thanks to decades of research, this process is a fairly well understood phenomenon. At the risk of vastly oversimplifying a complex and non-linear process, it might be conceived of in three phases:
- Those who can afford to, start out by reducing unnecessary expenses while also seeking opportunities to earn additional income. They often reach out to family, friends and neighbors for help.
- If the situation fails to improve, the next step for households is usually to begin spending whatever savings they have accumulated, taking on debt, and selling off property to raise money. Here is where the harm begins, as they erode their last remaining financial resilience against the impacts of future crises.
- In the end, as options dwindle, the real sacrifices begin: do they send their children off too school too hungry to pay attention in class, or pull them out of school entirely to earn money in the market? The damage done here cuts deep, and those affected may never fully recover.
Real-time tracking of the coping strategies adopted during a crisis is notoriously difficult. Governments may be able to get access to real-time contextual information such as food prices, fuel prices, and rainfall. Yet data on behavioral changes households adopt in response to shocks is gathered through rapid impact assessments that are too expensive to implement for large-scale routine monitoring. Often, we don't find out about the nature of what is happening until real harm has already been done. To catch these changes earlier will require a radically different approach.
Imagine how a slowly unfolding crisis might have looked through the eyes of a typical mobile phone carrier in sub-Saharan Africa offering services for mobile banking, livestock trading, and access to health and agricultural hotlines. Let us suppose that fuel and food prices have been on the rise for some time, and rainfall has been below average in many areas. As affected subscribers begin to cope with these shocks, they begin using their mobile phones in different ways, and in some cases, mobile phones play a critical role in helping them cope. All the while, they leave behind characteristic patterns in data. What could mobile carriers and services providers know about the impacts of crises on their customers?
- As food and fuel prices rose across the country, the carrier might see subscribers in the western part of the country begin to shift from adding their average $10 once per month on their SIM cards to a pattern of only topping off with 50 cents every few days. Perhaps they also see a drop in voice calls and an increase in texting.
- Soon, subscribers begin emptying their "mobile money" savings accounts, with the average dropping ominously week after week.
- A month later, people in the affected areas begin to default on mobile repayments of microloans in large numbers.
- As the months pass, carriers see a significant increase in attempts to sell livestock though a mobile trading network. Many of these attempts are originating in communities where livestock sales are rare at that time of year.
- Soon carriers see that subscribers who purchased their mobile accounts in drought-stricken rural areas have been connecting through cell towers in urban areas and other areas not affected by drought.
- Inevitably, calls to health hotlines from rural areas begin to see increased volumes of calls reporting symptoms consistent with the health impacts of malnutrition and unsafe water sources.
Making sense of digital smoke signals
The scenario above is purely hypothetical. We do not yet know how crisis-related coping strategies play out in the use of mobile services. Yet it's not difficult to imagine how the descent into poverty might be mirrored in "data exhaust" -- the unfolding story of people coping with a crisis that ultimately touches many aspects of their lives.
In isolation, the first few digital smoke signals to appear may reveal nothing more than generalized economic stress. Where there is smoke, there is fire, but understanding quickly whether the school, the factory, the farm, or the clinic is likely on fire requires additional analysis of context. Those smoke signals that appear later might yield clearer indications of impacts in specific sectors such as agriculture and health, but by then harm is already underway. What is needed here is a process for integrating these signals into our existing monitoring systems.
Lessons from Public Health
As it happens, a model for this process has been used for years in the field of public health to monitor for outbreaks of infectious diseases. Detecting the appearance of a combination of changes in collective behavior such as failing to show up at work or school, buying increased quantities of certain drugs, and presenting certain symptoms (e.g., fever, headache) at clinics is a reliable harbinger of the beginning of flu season. When this combination of anomalies appears, it is used to trigger a process of investigation, collection of samples, laboratory analysis, verification, and response.
More recently, Google made headlines with a project called Flu Trends. As it turns out, one of the first changes in collective behavior among people in households affected by flu is an increase in the number of online Google searches that include certain keywords such as "fever." By monitoring for changes in how people use its search services, Google is able to detect the onset of seasonal influenza weeks earlier than is possible using traditional methods of outbreak surveillance.
Turning this into a Science
Global Pulse is looking to extend this approach beyond public health, looking for the smoke signals that could reveal collective changes in behavior related to incipient harm in many areas of human life. Our Pulse Labs -- beginning with Pulse Lab Kampala late this summer -- will allow governments, development organizations, academe and the private sector to come together, experiment with new data, tools and approaches, and develop a formal methodology whereby the earliest smoke signals are used to trigger a process of investigation, verification, and response. We will likely need to analyze data gathered through remote sensing, social network analysis, news media mining, and crowdsourcing of citizen reports, and combine it with traditional indicators already used in crisis monitoring. Only then might we be in a position to understand the underlying causes and risks of future impacts on health, nutrition, livelihoods, or education, and to use this information to improve our capacity to protect vulnerable populations from harm.
For more about UN Global Pulse's research into real-time monitoring, open source technology platform, and network of Pulse Labs, please see our About page.
My career I have been founded on data for decision making, both in the private sector and in the international relief and development sector. The tools to process data are millions (or is it billions) of times more powerful now than at the beginning of my career 50 years ago ... but decisions are no significantly better than long ago, in fact may be much worse in terms of impact on the weak, powerless and vulnerable. This is no accident. There is a deep structural flaw in the way global resources are managed ... aided an abetted by a deeply wrong system of financial/socio-economic metrics.
The piece reminds me of an incident in my career about 25 years ago related to data and the Ethiopian famine. There were all sorts of ideas about early warning ... and eventually there was the US supported FEWS ... but nobody would talk about how the data about imminent famine would get translated into something that would actual stop people dying. We are still in the same modus operandi almost 3 decades later with more and more ways to get data and very little of these data being used to do socially valuable things. The advertising industry and the commercial profit world is trying to mine these data ... for profit benefit ... and equivalent powerful initiatives for social benefit seem to be tiny, under-resourced and maybe also lacking in ambition! Worse, the powerful establishment of rich and powerful are quite likely to work against anything significant that threatens the prevailing gravy train of IT for wealth concentration!
This is a fascinating idea, but we can't make assumptions about what these "smoke signals" might look like.
Many people are spending their money on their mobile/SMS at the cost of their children's education or bed nets: http://www.nytimes.com/2010/05/23/opinion/23kristof.html?_r=1 Maybe these people will be forced to cut back when a drought hits. But will they react early enough to be an "early warning system", like Flu Trends?
Does a family member moving to the city indicate he's left the farm to look for work, or that he's in boarding school?
Food prices aren't a great indicator. But they're reliably reported, untainted by wishful thinking, and react quickly to changes in the market. I think you'll have difficulty finding a better predictor of the market than the market itself.
Great post and some great work being done by Global Pulse.
We've been working on something along this line for well over a year, although some slightly different approaches to data collection, analysis and visualization with our artificial intelligence for analysis.
I agree on the "privacy" side; that will present some significiant challenges.
One aspect we're working on is "weak signal synthesis" and turning a number of seemingly "insignificant" comments and messages into a stronger signal of a looming crisis or situation. Although this is proprietary some form will be released to crisis mappers in 2012.
The other side is language. But that's a whole other set of issues!
There was a NASA research project in the "four corners" area on an illness transmitted by dust and I will try to find it. It has an interesting scenario. There was also a very good climate change scenario group at Los Alamos but it was disbanded.
It's an inspiring vision, but even if one stipulates that all of the methodological and interpretive challenges mentioned in previous comments are solvable, the main (and likely fatal) impediment to its realization is likely to be the unwillingness of communications network operators of any kind -- pots, cellular, Internet, etc. -- to provide you with the necessary data. Consider the kind of (annual time-scale, highly aggregated, overwhelmingly market demographic-focused) data that communications services make available -- typically under legal compulsion -- to institutions like the ITU and the OECD. Note that, with extremely few exceptions, none of the data that has ever made it into the public domain, through those or any other channels, even vaguely resembles the sort of fine-grained, high-frequency transaction-level or behavioral data that would be required to make such an early warning system work. There are numerous reasons why communications services providers almost never provide such data to third parties (and never ever do so without significant monetary or regulatory inducement), the most intractable of which is the potential for such information to be commercially exploited at some point in the future -- e.g., by established and/or aspiring competitors who could use the data to target upward-trending services or downward-trending/under-served communities, by market watchers who could capitalize on the information in financial markets (or use it to target private companies for acquisition), etc., etc. And that's assuming that the network operators in crisis-prone regions even possess the capability to aggregate transaction-level communication data in various ways that would be useful for would-be crisis mappers (or alternately, that crisis mappers with very specialized -- and typically very expensive -- know-how in production network traffic monitoring and instrumentation would be granted direct access to the network operator's critical service delivery infrastructure). Speaking from personal experience as a one-time early-warning researcher and someone who has been involved in implementing similar (albeit commercially-oriented) monitoring capabilities within one of the world's most well-financed and technologically advanced IP networks, I would estimate that the odds of satisfying the minimum basic preconditions (just) for the deployment of such a system to be possible are not good.
If this seems excessively pessimistic, I would simply suggest that any project of this kind should begin with a survey of several "typical" network operators in crisis-prone regions, to determine what data monitoring and aggregation capabilities and infrastructure they currently possess, what kind of data outputs they'd be willing to share, and/or what level of access they'd be willing to extend aspiring crisis mappers.
The display of information gathered by the Pulse processes is often shown on maps. The quality and functionality of those maps could be significatly improved by presentation in GIS based formats which would allow greater comparative and locational specifieity to bring needs, resources and delivery processes together in views with data behind them. This approach would provide order placement, delivery monitoring, and speedy realocation of resources and even of production as specific needs subside. This kind of capability is possible through online GIS such as the CIIIGeo project described in my Masters Project at Tartu University in 2001.
For more details:
as it is unlikely that I will find this blog again soon.
One of the Resources I would like to put ON OFFER through such a delivery system is a mobile, modular, regional system to support the needs of individulas, families and communities. It is a design of Boxes made of MDF Panels and Ribbon to form Beds, Furniture, Wall Structures, etc. to form simple living quarters to be housed in large open floored buildings, on land prepared for camps, on individual 'home sites,' etc.
Where in the Pulse system can such a resource be presented so that Agencies can find the information? Are there better Venus for the presentation of this kind of information?
Please respond to:
following the floods in Benin last year and research work I did post Asian Tsunami a group of broadcasters have been trying to build bridges between telecom and community radio networks to help build a post-disaster communications plan. We've examined what works and what doesn't and which open source tools should be in the kit that stations use. We can see that it will take some test cases to prove to government and mobile operators that sharing data has a benefit and that it will be used to help stations and emergency services get relevant messages out in local languagues. I hope globlalpulse works out a clever, clear communications strategy. It's clearly going to be country specific, but its a good step that people are thinking constructively meeting the hurdles step by step. There are also new tools popping up all the time. Use local broadcast media to help involve the public and celebrate success if going to be important simply to tell the monitors and volunteers what to do tomorrow.
You may know your work was mentioned at the excellent BBC Social Media Summit in London this past Friday. http://www.youtube.com/bbccojovideo#p/c/2AD6DFC3E488F317/3/0cNHq6rc-X8 about 15 minutes in.
I found the global pulse idea linking organizational behavior and trends to shape crisis a wonderfull one. For example, I supposed if an organization (an individual or a houshold) is on low earning, during an income fall, it should start to prioritize a cut on tobacco smoking and spending to solve the income issue and this between the fluctuating trend (the tobacco consumption) and the spending (behavior) would be clearly shown in a data chart.