About

What is “big data for development”?

Big Data for Development is a movement to foster widespread recognition that Big Data can be a useful public good. We are living in an age of Big Data. The explosion in access to mobile phones and digital services means that people everywhere are creating vast amounts of digital data every day—as they buy and sell goods, transfer money, search for information and share their experiences on social networks. This digital data is known as "Big Data" due to its sheer quantity, diversity and speed. Advances in computing and data science now make it possible to process and analyze Big Data in real-time.

Big Data is both the information that is passively generated as by-products of people’s everyday use of technologies and the information people willingly communicate about themselves on the web. Even when individuals do not have direct access to mobile phones or other technologies, they may still be passively emitting information as they go about their daily lives (e.g, when they make purchases, access services, or when they interact with better-connected members of the community). These digital trails, or “smoke signals” can reveal changes in population behavior trends in our collective preferences, struggles and overall well-being. What makes this opportunity so unique is that the analysis can be done in real-time, to help improve the timeliness of actionable evidence used in decision-making around population resilience. This process offers enormous promise for increasing efficiency in targeting and monitoring of development programmes.

Big Data for Development refers to Global Pulse’s effort to determine how Data can be used to better inform the policy and planning of development programmes. It also refers to Global Pulse’s efforts to galvanize governments, non-profts and corporations—i.e. both public and private sectors—to join in the effort to discover the tools and technologies necessary to apply Big Data and data science to development challenges..

For more information on the power of digital data, watch this short introductory video.

What types of digital data is Global Pulse interested in exploring?

For purposes of discussion, Global Pulse has developed a loose taxonomy of types of new, digital data sources that could be relevant to global development:

• Data Exhaust – passively collected transactional data from people’s use of digital services like mobile phones (call detail records, location data, airtime purchase patterns), making purchases, transferring remittances or mobile money, etc., and/or operational metrics and other real-time data collected by UN agencies, NGOs and other aid organisations to monitor their projects and programmes (e.g. stock levels, school attendance); these digital services create networked sensors of human behaviour;
• Online Information – web content such as news media and social media interactions (e.g. blogs, Twitter), web searches, news articles obituaries, e-commerce, job postings; this approach considers web usage and content as a sensor of human intent, sentiments, perceptions, and want;
• Physical Sensors – satellite or infrared imagery of changing landscapes, traffic patterns, light emissions, urban development and topographic changes, etc; this approach focuses on remote sensing of changes in human activity.
• Citizen Reporting or Crowd-sourced Data – Information actively produced or submitted by citizens through mobile phone-based surveys, hotlines, user-generated maps, etc; While not passively produced, this is a key information source for verification and feedback.

How are the data sources accessed?

Most sources of this data are already available in the public domain. The challenge is developing the tools to both capture and responsibly use the raw data. Partnerships are also critical. Since its inception, Global Pulse has been building partnerships to access new sources of data. This includes reaching out to data providers within and outside the UN to access information that has long been used for development (e.g. data from national statistics offices and international agencies), and to unlock privately held data (e.g. mobile phone data, sales data, search data). Global Pulse has positioned itself as a thought-leader on “data philanthropy,” driving discussions with business leaders and the public about the importance of sharing private sector data streams for the global good.

How does Global Pulse deal with the issue of data privacy?

Mining data to take the pulse of populations does not require monitoring individuals. Our work does not identify specific individuals or even groups of individuals. Rather we depend on large data sets of annonymized, aggregated data that can give us a sense of how whole populations or communities are coping with shocks that can result in widespread behavioral changes.

Global Pulse has consistently argued that safe and responsible use of Big Data must ultimately be viewed in a human rights context, as the risk of re-identification through misuse of Big Data could open the door to discrimination.

In consultation with privacy experts, and building on existing models such as recent work in health data privacy, Global Pulse is working to develop a set of privacy protection guidelines for how to apply big data safely and responsibly. These guidelines will be shared with all our partners and applied in our collaborative projects.

What type of projects is Global Pulse working on?

Global Pulse matches interested UN agencies, governments and partners with private sector organizations that have the capabilities required to investigate research questions and develop working proofs of concept. Global Pulse works with partners to design experiments, coordinate research, evaluate results and communicate findings. When R&D efforts yield useful tools and approaches, Global Pulse supports UN System partners in pursuing broad institutional adoption.

Five general “Proof of Concept” projects were completed in 2011, demonstrating the feasibility and utility of real-time digital data (such as social media, online news, mobile phone surveys, and online food prices) to answer questions relevant to decision-makers (such as unemployment trends, coping strategies, food prices, and public perceptions). (see: http://unglobalpulse.org/applyingbigdatatodevelopment)

Research projects conducted at Pulse Labs in 2012-2014 will apply the new methodologies and technologies towards specific programme and policy related questions. Projects will be identified in-country by UN agencies and their local partners, and the public sector.

What are Pulse Labs?

Pulse Labs are physical centers of innovation and R&D that bring together government, the UN and local partners in academia and the private sector to test, refine and scale methods for using digital data streams to support development goals. Crucial to the success of Pulse Labs, is having government and local expertise involved in their creation and management. Global Pulse’s headquarters are in New York. In 2012, Pulse Lab Jakarta was established, and plans are underway to establish Pulse Lab Kampala in early 2013.

Pulse Labs conceive and conduct projects that bring together experts from government, NGOs and private companies in the region to work on applying innovative analytic approaches and data science to thematic social development challenges. For example, approaches for exploration include:

• Social Media & Twitter Analysis
• Mobile Phone Data Analysis
• Rapid Mobile Surveys
• Geo-Spatial Mapping

For more information on Pulse Labs go to http://www.unglobalpulse.org/pulse-labs

How does Global Pulse deal with different levels of technological access within and between countries?

The reach of mobile technologies is expanding every day. Even individuals who are not directly connected to mobile and other technologies are passively emitting information as they go about their daily lives. When they make purchases (even at informal markets), access services like healthcare, or interact with community leaders, these activities are recorded.

The availability and types of digital data will indeed differ from country to country. Countries with high mobile phone and internet penetration rates will produce more data that is directly generated by citizens. Countries with large aid communities will produce more programme-related data than less aid-dependent countries. And countries with a vibrant local business environment will offer greater opportunities for private sector involvement.

The availability of data also varies between age groups, economic income brackets, gender and geographic location. These types of biases will be addressed in the way Global Pulse research projects are designed and conducted.

How can I partner with Global Pulse?

Global Pulse partners with governments, corporations, non-governmental organizations. We depend on partnerships for the following resources:

• Research Design and Development: We work with United Nations agencies, NGOs and other partners to conceive research questions and to co-design and develop projects that address the relevant development challenges.
• Data: Partners share data sets or provide access to streaming data to support Global Pulse’s groundbreaking public research into areas such as unemployment, food security, urban poverty, migration, public health, climate change, human rights, and conflict prevention.
• Technology: Access to the best tools for data mining, real-time analytics, storage, computing, and data visualization is essential to our data scientists ability to successfully tackle research questions. These types of partnerships also showcase how cutting-edge technologies can help make the world a better place.
• Expertise: Engineers, data scientists or researchers can collaborate with Global Pulse on specific projects to support UN agencies, national governments, and international development organizations.
• Sponsorship: Global Pulse is entirely funded through voluntary contributions from individual UN Member States and philanthropic organizations. We welcome any expression of interest from donors or sponsors who would like to help expand and accelerate the work of Big Data for Development. Corporate sponsorship or grants of data, technology and human expertise to Global Pulse can be received through our fiduciary partner, the UN Foundation, a tax-exempt 501(c)(3) based in Washington, D.C.

For more information on plans to grow the initiative, see our Strategy and Roadmap section.