Big data: Do we need them?

Date: 14/11/2014

In October this year (2014), a 37-year-old Brisbane man was the first arrest and conviction of an undercover online operation targeting pedophiles, using an avatar - computer-generated animation of a 10-year-old Filipina girl - set up by Dutch Children's rights group Terre des Hommes Netherlands.

The avatar known as Sweetie has managed to lure more than 1000 adults around the world; the details are provided to Interpol and an operator was quoted saying the Australian man, Scott Robert Hansen, was not the worst.

Why is the United Nations Population Fund (UNFPA) discussing this story, you may be asking by now?

The arrest is just one example of how big data can be utilised to strategically improve productivity or results regardless of the sector one works in.

Big data for example can inform predictive technologies which could in turn enable us to, for example: predict criminal human behaviour that potentially violates others' human rights and/or predict natural disasters and the impact of climate change that could lead to pro-active targeted preparedness and strategising.

Big data is defined by SAS Institute Inc (North Carolina, USA) as the 'exponential growth of data, both structured and unstructured' and there are five main factors one can narrow big data down to: volume (increased data whether stored transactional or structured data or unstructured data from social media); velocity (unprecedented speed with which data is streaming in); variety (data come in all types of formats - numeric, text, video or audio, etc); variability (can be highly inconsistent with periodic peaks for example event-based trending on social media); and complexity (data come in from multiple sources).

As the United Nations agency mandated to address population and development issues, big data is of interest because of the how we place a lot of importance on evidence-based responses to issues we are dealing with in the 150 countries the organisation is present in.

It is a given that we live in a time in which the connectivity between information sources and its dissemination platforms like the internet-enabled information highway, the world wide web means information overload is almost the norm.

Digital data doubles in quantity (roughly) every three years; in 2003, digital data accounted for 25 per cent of all information in the world, by 2013, only two per cent of all the information in the world were non-digital.

The amount of stored information grows four times faster than the world economy, while the processing power of computers grows nine times faster.

There is acknowledgement of the amount of data humanity is circulating and its implications on its utilisation and management.

Three main types of companies have emerged from this big data phenomenon - those who have the data (for example Google), those who have the skills (like tech companies) and those who have the big data mind-set - individuals or companies and organisations which can analyse the available data and devise innovative ways to strategically recycle or utilise the data.

"As an agency which has data for development as one of its core functions, the UNFPA is very interested in and looking at ways to best utilise this big data phenomenon; big data represents for us a new source of value and a means for innovation," UNFPA Pacific Sub-Regional Office Director and Representative Dr Laurent Zessler said.

"Big data does not replace nor displace the huge body of research work we have already done for example the violence against women prevalence studies.

"The big data phenomenon will however complement the content of existing information repositories."

Big data allows us to parse through huge amounts of data and make decisions based on correlations identified - the "what" instead of the "why" and thus challenges the normative path of establishing causality.

This shift from focusing on causation to establishing correlations merely means we are open to utilising the massive data, whether structured or unstructured, humanity has managed to collate. This datatification - looking at all this data and making it quantifiable to throw up new insights, patterns and predictions - can inform predictive analysis.

For UNFPA, big data can mean the ability to determine people's migratory patterns in real time just from mobile telephone data - texts, calls, etc - and using this data to predict potential disease outbreaks for example or where to best locate a new health centre.

Another example is how the Google Flu trends were used to accurately predict patterns of the flu outbreaks, far ahead of the traditional epidemiological surveillance techniques.

Closer to our socio economic mandate areas, big data enabled the quick correlation between high income mobility (Chetty et al) and less segregation, better schools and more stable families as opposed to teenage births and high school dropouts among indicators of dead-end immobility.

Like all things new, big data does come with risks and challenges - for example accessing data from those who have it, like mobile telephone companies.

Big data are also not necessarily subject to peer review and the issue of privacy is another that will have to be addressed - questions that linger but should be addressed with increased public discourse on big data.

The data revolution is about harnessing new and existing sources of data for enabling results in development work; establishing correlations and making decisions quickly can save lives; more long-term, it will contribute to the reduction of inequality and full realisation of human rights for all come to mind.

Big data and its implications on development approaches is set to be the future; it may be prudent that stakeholders of development work in our various island nations began considering its complementary nature to decision-making and national development planning