About

What is “new data”?

New data encompasses “massive passive data” as well as “the open web.” It is both the information that is passively generated as by-products of people’s everyday use of technologies and the information people willingly communicate about themselves on the web. Even when individuals do not have direct access to mobile phones or other technologies, they may still be passively emitting information as they go about their daily lives (e.g, when they make purchases, even at informal markets; when they access basic health care; or when they interact with local community leaders). For more about the promise of new data analysis, see our blog post The New Data Landscape.

How are new sources of data accessed?

Most sources of this data are already available in the public domain. The challenge is developing the tools to both capture and responsibly use the raw data. Partnerships are also critical. Since its inception, Global Pulse has been building partnerships to access new sources of data. This includes reaching out to data providers within and outside the UN to access information that has long been used for development (e.g. data from national statistics offices and international agencies), and to unlock privately held data (e.g. mobile phone data, sales data, search data). Global Pulse has positioned itself as a thought-leader on “data philanthropy,” driving discussions with business leaders and the public about the importance of sharing private sector data streams for the global good.

How is the data analyzed? Who will do the analysis?

The explosion in the volume and diversity of data has been met by advances in methodologies to help make sense of it all. New tools exist to collect, filter, integrate, fuse, analyze, map and visualize information. While analytical techniques such as data mining have long been used in the private sector, they are increasingly being mobilized by governments and the public sector as well. For example, the Billion Prices Project at MIT represents a complementary approach to traditional inflation monitoring. Instead of monitoring price changes across thousands of consumer items in person or via phone, the Billion Prices Project culls daily price fluctuations from the web, tracking around 5 million items sold by more than 300 online retailers in more than 70 countries. In the near-term, Pulse Labs will lead and enable the development of discreet collaborative research projects to test the new analytical methods. In doing so, they will learn what works and what doesn’t, scale up the most effective, proven solutions, and integrate new methods and approaches into the everyday work of decision makers at local, national and international levels.

How does Global Pulse account for different levels of technological access within and between countries?

The reach of mobile technologies is expanding every day, such that even individuals who are not directly connected are passively emitting information as they go about their daily lives—when they make purchases, even at informal markets; when they access basic health care; or interact with local community leaders; etc. The availability and type of digital data will differ from country to country. Clearly, countries with high mobile phone and internet penetration rates generate more data directly from citizens than others. Countries with large aid communities will produce more programme-related data than less aid-dependent countries; and countries with a vibrant local business environment will offer greater opportunities for private sector “data philanthropy.” It will also vary between age groups, economic income brackets, gender and geographic location. When tapping these alternative data sources for “pulse data” such selection biases need to be clearly understood.

How can you ensure that citizen privacy is protected?

The opportunity to use new data to inform decision makers on how populations are impacted by crisis is enormous; however, the challenges in using that data responsibly to protect individual privacy needs to be properly incorporated at the very beginning. A key part of developing this system is ensuring that all personal data will be aggregated and anonymized such that individuals cannot be identified.

What has Global Pulse achieved to date? What are the next steps?

Global Pulse was launched in late 2009 as an innovation initiative within the Office of the Secretary-General. With 2010 as the design phase of the initiative and 2011 is its first year of implementation Global Pulse has drawn together a multi-disciplinary and cross-institutional team of academics, analysts and technologists who are collaborating to build the technological platform; to develop new tools to harness data; and to develop the methodologies for mining and analyzing large quantities of unstructured data. The New York staff is complemented by a small team in Uganda, whose role is to build strategic partnerships and test new technologies, analytical approaches, and the utility of real-time data in the field, together with the Ugandan government and UN agencies.

Some highlights of Global Pulse’s achievements to date include:

• Through ongoing research, the Global Pulse team is developing a framework for how to analyze new “digital smoke signals” against data from existing monitoring systems. We have partnered with top data analysis teams from both the private sector and academia to demonstrate the methods for unlocking new data streams (data mining of on- line sources, remote social impact sensing, information exhaust and crowd-sourcing) and analyzing them for "digital smoke signals" of how populations change their behaviors to cope with slow-onset crises. The global and country-level case studies will showcase how new data and analysis can shrink the knowledge and time gap in our understanding of the impacts of increasing global volatility on populations on the ground.
• Global Pulse has collaborated with Adaptive Path and a team of volunteer designers and coders to build a prototype for HunchWorks (a tool that will allow development experts to create hypotheses about present or future events, link them to evidence and share them through social networks of trusted colleagues) ready for user testing in early 2012.
• “Pulse Lab New York” is operational. It is staffed by a team of thirteen experts with skills ranging from traditional development economics to new data sciences and technology. In addition, the development phase of Pulse Lab Kampala has been received with strong support by the Ugandan government. And Pulse Lab Jakarta, with the support of the Indonesian government and initial funding provided AusAID is in scoping and design phase.
• Thought-leadership on “data philanthropy”: driving discussions with business leaders and the public about the importance of unlocking private sector data streams for the greater global good. 

For more information on plans to grow the initiative, see our Strategy and Roadmap section.