The Importance of Big Data Partnerships for Sustainable Development

5 min read

The UN Secretary-General Ban Ki-moon’s High Level Panel has called for a “data revolution” for sustainable development; such a revolution has already arrived in the private sector. But forward-thinking companies can now lead a revolution in Corporate Social Responsibility (CSR) to help us address some of the world’s most pressing issues.

When thinking of engagement with the private sector, the global development community has tended principally to concentrate on volunteer time and donated goods and services, as the Overseas Development Institute (ODI) has highlighted. But there is much more we can learn from the private sector about reducing vulnerability and using real-time insights to boost resilience, mitigate risk and respond to changing global circumstances with greater agility.

Last year, the UN updated its roadmap and agreed on 17 overarching Sustainable Development Goals (SDGs) to see us through to 2030. This new agenda brought an unique opportunity to future-proof the process by using methods that may seem cutting-edge now, but will be considered anything but in a few short years.

The truth is that the data revolution has been underway in the private sector for over a decade.  Citizens today – in both developing economies and industrialized ones – are generating a growing ocean of digital data, every minute of every day, just by going about their daily lives.  As we use mobile devices to communicate, buy and sell goods, transfer money, search for information on the Internet, and share our lives publicly on social networks, we leave digital trails that private sector firms are mining to understand the needs of their customers, track emerging market trends, and monitor their own operations in real-time.

In healthcare, in the auto-industry, and even in entertainment markets this real-time (and occasionally predictive) intelligence has fundamentally altered how companies serve their customers, leading to the emergence of new business models.

This is good news for those seeking to harness big data for social good; the development community is not starting from scratch, but rather seeking to adapt innovative tools and methods to our own needs and learn how to stay ahead of the curve in a fast-changing world.


The Global Pulse initiative is a big data innovation lab for the United Nations. We see that in analysing big data there is potential to generate a real-time understanding of human well-being. Since our inception in 2009, we have been researching, innovating and advocating around many of the principal challenges related to transforming digital data into better outcomes for the poor, discovering new approaches, building tools, and working to demonstrate ways to overcome barriers to adoption and scale.

In time, this practice will be seen as a ‘no brainer’ for the development community: using mobile data to gain insight into urban dynamics to improve transport infrastructure in Sao Paulo or Abidjan; understanding how perceptions spread via social media in order to tailor HIV prevention campaigns, correlating dengue epidemics with data on holiday travel patterns to ramp up prevention strategies or planning where to set up support services based on granular mapping of the routes that communities previously displaced by earthquakes used.

But unlike in the private sector, this is not a case of crunching your own numbers or using your own sales data to identify trends. A great deal of the most insightful real-time digital data is held by private sector companies.  The public sector, therefore, cannot fully exploit big data without leadership and partnership from the private sector. What we need is action that goes beyond corporate social responsibility. We need big data to be treated as a public good.

"The public sector, therefore, cannot fully exploit big data without leadership and partnership from the private sector. What we need is action that goes beyond corporate social responsibility. We need big data to be treated as a public good."


This is not just blue-skies thinking, there are companies doing this already. The Global Pulse network of partners and collaborators includes forward thinking private sector companies willing to engage in “data philanthropy,” by granting access to data and technology tools to the public sector, as well as industry leaders, universities, research institutes and non-profit networks of researchers and innovators who are ready to bring their skills and expertise to bear for the cause of advancing the use of data science across global development and humanitarian fields. This type of collaboration has already enabled big data innovation projects across the UN system with agencies including the World Health Organisation (WHO), the United Nations Development Programme (UNDP), the World Food Programme (WFP), The United Nations Children's Emergency Fund (UNICEF), the Joint United Nations Programme on HIV/AIDS (UNAIDS) and many others.

Of course, protecting individual privacy is paramount when analysing data types of passively generated data including social data, mobile phone data, internet search data, consumer behavior sales data and more. Putting in place mechanisms to protect privacy based on shared guidelines, regulations and technology will help build frameworks in which data can be safely and ethically analysed for insights that can be used to help protect populations. Private sector and public sector must come together to develop these frameworks. 


The 2030 Agenda will require much more of this type of collaboration between public and private sector for social good; and there are many different modalities of working together. In some cases companies interrogate their own data, using their own data scientists, for information on trends that can be used to gain intelligence to solve development and humanitarian problems.

This experience can also prove a positive one for the corporate data scientists. Data scientists are in short supply, and are from a generation to whom ethical behavior and good corporate CSR is extremely important. By allowing these talented young men and women some latitude to use their skills to make the world a better place, companies can boost job satisfaction and retention.

Data philanthropy is also good for business. Take for example, any given mobile phone company operating in a fragile economy – they have an interest in customers’ general wellbeing and also in regional economic health. Huge spikes in food prices, displacement by floods and earthquakes, or a fast-moving disease epidemic all represent a risk to the phone operators customer base. In this case and many others, sharing data with the public sector can help NGOs, foundations and UN agencies work to mitigate risk of harm and  boost resilience – it is a win-win situation.  

Ultimately, we envision a world in which the private sector routinely contributes to a real-time data commons – where information on citizens’ wellbeing could be aggregated and shared, as currently happens with weather data. Yet, we recognize that companies will only participate on their own terms, and doing so must also make good business sense.

The development community can’t tackle global 21st century problems using 20th century techniques. We must find a way for different types of real-time data to be shared and analysed in ways that don’t compromise market competitiveness, and that fully protect privacy in the process.

It will take courage, imagination, new regulatory frameworks, innovative policies, and fresh thinking about how public-private sector partnerships can be structured — but we must bring about this new reality.


Robert Kirkpatrick is Director of the UN Global Pulse initiative. A version of this article originally appeared in the Global Compact Yearbook.

Did you enjoy this blog post? Share it with your networks!

News, thoughts and ideas about big data and AI, data privacy and ethics from across the Pulse Lab Network. Read more on the blog.

Scroll to Top