What is “big data for development”?
Big Data for Development is a movement to foster widespread recognition that Big Data can be a useful public good. We are living in an age of Big Data. The explosion in access to mobile phones and digital services means that people everywhere are creating vast amounts of digital data every day—as they buy and sell goods, transfer money, search for information and share their experiences on social networks. This digital data is known as "Big Data" due to its sheer quantity, diversity and speed. Advances in computing and data science now make it possible to process and analyze Big Data in real-time.
Big Data is both the information that is passively generated as by-products of people’s everyday use of technologies and the information people willingly communicate about themselves on the web. Even when individuals do not have direct access to mobile phones or other technologies, they may still be passively emitting information as they go about their daily lives (e.g, when they make purchases, access services, or when they interact with better-connected members of the community). These digital trails, or “smoke signals” can reveal changes in population behavior trends in our collective preferences, struggles and overall well-being. What makes this opportunity so unique is that the analysis can be done in real-time, to help improve the timeliness of actionable evidence used in decision-making around population resilience. This process offers enormous promise for increasing efficiency in targeting and monitoring of development programmes.
Big Data for Development refers to Global Pulse’s effort to determine how Data can be used to better inform the policy and planning of development programmes. It also refers to Global Pulse’s efforts to galvanize governments, non-profts and corporations—i.e. both public and private sectors—to join in the effort to discover the tools and technologies necessary to apply Big Data and data science to development challenges..
For more information on the power of digital data, watch this short introductory video.
What types of digital data is Global Pulse interested in exploring?
For purposes of discussion, Global Pulse has developed a loose taxonomy of types of new, digital data sources that could be relevant to global development:
How are the data sources accessed?
Most sources of this data are already available in the public domain. The challenge is developing the tools to both capture and responsibly use the raw data. Partnerships are also critical. Since its inception, Global Pulse has been building partnerships to access new sources of data. This includes reaching out to data providers within and outside the UN to access information that has long been used for development (e.g. data from national statistics offices and international agencies), and to unlock privately held data (e.g. mobile phone data, sales data, search data). Global Pulse has positioned itself as a thought-leader on “data philanthropy,” driving discussions with business leaders and the public about the importance of sharing private sector data streams for the global good.
How does Global Pulse deal with the issue of data privacy?
Mining data to take the pulse of populations does not require monitoring individuals. Our work does not identify specific individuals or even groups of individuals. Rather we depend on large data sets of annonymized, aggregated data that can give us a sense of how whole populations or communities are coping with shocks that can result in widespread behavioral changes.
Global Pulse has consistently argued that safe and responsible use of Big Data must ultimately be viewed in a human rights context, as the risk of re-identification through misuse of Big Data could open the door to discrimination.
In consultation with privacy experts, and building on existing models such as recent work in health data privacy, Global Pulse is working to develop a set of privacy protection guidelines for how to apply big data safely and responsibly. These guidelines will be shared with all our partners and applied in our collaborative projects.
What type of projects is Global Pulse working on?
Global Pulse matches interested UN agencies, governments and partners with private sector organizations that have the capabilities required to investigate research questions and develop working proofs of concept. Global Pulse works with partners to design experiments, coordinate research, evaluate results and communicate findings. When R&D efforts yield useful tools and approaches, Global Pulse supports UN System partners in pursuing broad institutional adoption.
Five general “Proof of Concept” projects were completed in 2011, demonstrating the feasibility and utility of real-time digital data (such as social media, online news, mobile phone surveys, and online food prices) to answer questions relevant to decision-makers (such as unemployment trends, coping strategies, food prices, and public perceptions). (see: http://unglobalpulse.org/applyingbigdatatodevelopment)
Research projects conducted at Pulse Labs in 2012-2014 will apply the new methodologies and technologies towards specific programme and policy related questions. Projects will be identified in-country by UN agencies and their local partners, and the public sector.
What are Pulse Labs?
Pulse Labs are physical centers of innovation and R&D that bring together government, the UN and local partners in academia and the private sector to test, refine and scale methods for using digital data streams to support development goals. Crucial to the success of Pulse Labs, is having government and local expertise involved in their creation and management. Global Pulse’s headquarters are in New York. In 2012, Pulse Lab Jakarta was established, and plans are underway to establish Pulse Lab Kampala in early 2013.
Pulse Labs conceive and conduct projects that bring together experts from government, NGOs and private companies in the region to work on applying innovative analytic approaches and data science to thematic social development challenges. For example, approaches for exploration include:
For more information on Pulse Labs go to http://www.unglobalpulse.org/pulse-labs
How does Global Pulse deal with different levels of technological access within and between countries?
The reach of mobile technologies is expanding every day. Even individuals who are not directly connected to mobile and other technologies are passively emitting information as they go about their daily lives. When they make purchases (even at informal markets), access services like healthcare, or interact with community leaders, these activities are recorded.
The availability and types of digital data will indeed differ from country to country. Countries with high mobile phone and internet penetration rates will produce more data that is directly generated by citizens. Countries with large aid communities will produce more programme-related data than less aid-dependent countries. And countries with a vibrant local business environment will offer greater opportunities for private sector involvement.
The availability of data also varies between age groups, economic income brackets, gender and geographic location. These types of biases will be addressed in the way Global Pulse research projects are designed and conducted.
How can I partner with Global Pulse?
Global Pulse partners with governments, corporations, non-governmental organizations. We depend on partnerships for the following resources:
For more information on plans to grow the initiative, see our Strategy and Roadmap section.