From Pilots to Policy Lessons: New Methods for Information Collection, Analysis and Use

SIPA research team on the road from Gulu to Awach, Uganda, to conduct research at rural health clinics.
Photo courtesy of Mark Weingarten

What is innovative and compelling to one party may seem unsound and suspect to another. Fulfilling the UN Global Pulse vision will require a truly multilateral, cross-sectoral effort, yet what exactly does this constellation of collaborators look like? And, more importantly, how might these potential comrades view the unprecedented approaches Global Pulse is proposing?

While Global Pulse’s 2010-11 Road Map focuses largely on the initiative’s technical to-do’s, much of the effort in realizing Global Pulse will lie in securing buy-in from country governments, local communities, and diverse partners large and small. Though technology is at the heart of the initiative, skillful, devolved navigation of cultural sensitivities, political barriers, and operational challenges will be critical to its success. Without partners that fully understand and are committed to the sub-global modules of Global Pulse, the larger ecosystem is destined to fail. Technology, no matter how cutting-edge and sophisticated, is but a fraction of the battle.

To this end, UNICEF’s Innovation Unit and Columbia University’s School of International and Public Affairs (SIPA) undertook a semester-long (December 2009 to May 2010) research project, “New Methods for Data Collection, Analysis, and Use”, to explore the non-technological challenges Global Pulse will face as it works to bring its vision to global scale.

Although Global Pulse will be responsive to widely varied regional parameters, indicators, and priorities, in order to focus the research, SIPA was asked to analyze two current UNICEF projects, one in Uganda and the other in Iraq. Though both involved real-time data collection, they were entirely distinct in aim, focus, set-up, and stage of development.

UNICEF Uganda is developing a mobile-based health vulnerability monitoring programme that builds upon and provides real-time support to existing surveillance systems. UNICEF Iraq is exploring how to better understand, and thus serve, the needs of the country’s most vulnerable children by providing them a direct (mobile) connection to reach out to those that can help them. Beyond differences in project nature and scope, Uganda and Iraq, as countries, vary widely in character, technical infrastructure, level of poverty, dependency on foreign aid, and political sensitivities. And while a sample of two is hardly adequate in developing a policy framework, it is a starting point. The unique challenges presented by the Uganda and Iraq pilots – and by their respective operating environments – demonstrate the flexibility necessary in building a global system, and the necessity of a policy roadmap that anticipates and accommodates diversity.

The researchers traveled to the field to meet with UNICEF country office staff, government representatives and other implementation partners, end users, local NGOs, and subject matter experts to better understand the complex contexts in which the programmes would be deployed. Through intimate understanding and analysis of the projects’ processes – from rationale, to design, to interaction with governments and implementation partners, to execution strategies – the researchers were able to abstract lessons for extending similar systems to future programmes and states.

The researchers’ recommendations for UN Global Pulse are summarized below:

1. Use a stepping-stone approach to build capacity and earn trust.
   Governments and organizations may, for a variety of reasons, be hesitant to participate in real-time data collection systems, or to participate at the scale requested of them. In these cases, Global Pulse should start with programmes with focuses and at scales (ie. tracking less-sensitive indicators on a smaller scale) they are comfortable with. Once the merits of real-time technology have been proven, partners may be more willing to cooperate in more involved surveillance systems.
    
2. Use the UN’s leverage at both the global and national levels.
   As a UN initiative, Global Pulse has the potential to set transnational priorities and help define protocol and common standards that will have massive impact. This the initiative well recognizes. At a country-level, however, Global Pulse should also use its leverage to secure attractive and otherwise unattainable operator contract terms and partnerships to support national programmes.
3. Involve country governments in determining secondary indicators.
   The global indicators tracked by Global Pulse will need to be supplemented by region- and context-appropriate indicators, whose collection may vary in frequency. These should be jointly selected with state governments to a) respect and utilize individual governments’ expertise, b) encourage political buy-in among partner governments, and c) improve the functioning of existing state-run response systems.
4. Develop suitable data verification systems and determine the appropriate balance of speed and accuracy on a per-case basis.
   One risk of real-time systems is that they may allow information to be publicized before it is verified. Therefore, we stress the importance of implementing appropriate, robust verification systems. The nature of these will depend on the sensitivity of the information being collected and the political sensitivities in the relevant country or region. Since this may potentially slow the flow of information throughout the system, a balance between speed and accuracy should be sought on a case-by-case basis.
5. Recognize that community buy-in is as important as government buy-in, and design the system accordingly.
   For the long-term maintenance and sustainability of Global Pulse, the system needs to have demonstrable value at the community level. The benefits of Global Pulse may not be immediately, locally apparent or locally appreciated; thus, insomuch as it is possible, the system should incorporate auxiliary advantages for the communities (ie. increased access to information, increased peer interaction).
6. Consider local capacity and potential from the outset, and design community-level systems that nurture and build upon both.
   Local ownership of community-scale networks will be critical to a sustainable Global Pulse. Project development should thus, from the start, consider and build upon local ability. Technological training will reduce costs in the long run; analytical training will allow communities to makes sense of and respond to locally collected data, as well as apply the data to other ends. This increases local agency, and multiplies and maximizes the utility of the global system.
7. Develop a practical framework for monitoring and evaluating progress.
   Global Pulse should select and use appropriate performance measures to track progress. The system should also be designed so that it serves as a platform for highlighting and sharing best practices and lessons learned in this sector.
8. Feed information back in to local communities.
   Not only should Global Pulse collect and analyze data for vulnerability monitoring, it should also feed information back into communities that it collects data from in meaningful ways. What are local programmes that would be able to use the data collected by Global Pulse? Could systems set up by Global Pulse be nominally adapted to accommodate additional information collection that could benefit smaller scale programmes? If communities do not receive feedback nor see positive change as a result of their participation, they may withdraw their support.

For more information on the UNICEF-SIPA collaboration, including full details of SIPA’s recommendations and processes, please see the full report.

Professor Elisabeth Lindenmayer, director of the UN Studies Program at SIPA, oversaw the seminar. Acting as advisors were Christopher Fabian and Panthea Lee of UNICEF Innovation, and Robert Kirkpatrick and Chris van der Walt of UN Global Pulse.