The technology solutions currently highlighted in the press are centred around “big data” and the “big data revolution” and relate to the ICT systems that enable the processing, storing and distribution of ever increasing volumes of data, at an ever-reducing cost to the end user. Moore, Kryder, Gilder and Metcalfe foresaw this incredible evolution of ICT systems. Moore (1965) suggested that, “processor speeds or overall processing power for computers will double every two years”. Kryder’s correlate to “Moore’s Law” predicted a similar expected increase in the density and capability of hard drive storage media over time (Walter 2005). Gilder (2000) calculated that the total bandwidth of communication systems triples every 12 months, and Metcalfe added that the value of a network is proportional to the square of the number of nodes, so as a network grows, the value of being connected to it grows exponentially, whilst the cost per user remains the same or reduces (Shapiro and Varian 1999). In summary, our ability to process and store data is getting easier and cheaper. Whilst the cost of sharing data is reducing, the value of sharing and connecting users together is increasing significantly.

The smart phone I carry in my pocket has 488,549 times the amount of memory of the first mainframe I booted up in a warehouse 30 years ago. The mainframe was expensive and affordable by only big corporations and research institutes. It not only took up vast amounts of space, but also required cooling, humidity control and lots of careful preventive maintenance. My home Ultra Fast Broadband (UFB) connection allows me to download data at 14 times the speed of the connection I had a year ago.

Computers and computing have come a long way in 30 years, but the human has not adapted at anything like the same pace. The limiting and enabling factor for extracting value from data whilst minimising consequential risk is the same limiting and enabling factor that has controlled the use of information since the dawn of time—the human, who decides what can be shared and how and when it is shared.

Just over 100 years ago, the lack of data sharing was a contributory factor to the huge loss of life on the sinking of the RMS Titanic. There were many factors that worked together to make the Titanic the disaster it was, but three relate directly to data sharing and interpretation. According to the inquiry by the US Senate, the SS California alerted the wireless operator on the RMS Titanic of pack ice ahead, but it appears that the message was mislaid or ignored. At the time the message was relayed, the team on the Titanic were apparently busy retrieving communications for passengers on board. Later that night, distress flares fired off by the RMS Titanic were seemingly misinterpreted by the SS California crew. The third data related contributory factor was the turning off of the wireless on the SS California overnight. As far as data sharing was concerned, the technology appeared to have been working perfectly. The weak elements were the decisions made by the human operators and their commanders.

And just 30 years ago, at about the time I was working with mainframes, the Challenger shuttle was launched against the advice of the engineers who designed the O-ring joint seals. The decision makers had been advised of the O-ring failures at low temperatures and had been provided with an accurate weather forecast predicting low temperatures for the morning of the launch. And yet, a decision was made to go ahead with the launch. Timing was critical to enable the first Teacher in Space to deliver a lesson during school hours and the fear of reputation loss from a delayed launch was front of mind.

In general, technology and the use of technology to share data has not been an issue—it is still how we share data and the decisions we make around what we share that causes us problems. However, when we magnify the age-old problems of data sharing with the speed, capability and accessibility of big data solutions, we face major breakthroughs and at the same time risk major disasters affecting thousands rather than hundreds. Human interpretation and context is everything.

In a very remote corner of New Zealand, birds from one of our native parrot families, the kakapo, have been busy laying eggs in a highly monitored and protected environment. Total kakapo numbers are down to 150, and they need all the assistance they can get to ensure that the species does not die out entirely. Techniques learnt from big data solutions are helping the parrots to mate and deliver live healthy chicks with a balance of males and females. Each bird wears a smart transmitter that reports position and gives an indication of activity. The technology can determine whether a female is nesting, and the male transmitter provides information about which females he has mated with and when. Intervention is significant. Rangers will intervene if a female makes a poor mating choice, if the pregnant female gets too heavy or too light, and if the nesting female kakapo proves to be an incompetent mother. An array of gadgets and sensors called “the nest kit” monitors eggs and chicks in a nest and provides plenty of useful data to inform the rangers. There has been a record number of live chicks born this year. Thankfully, this is a mast year for the kakapo’s favourite tree, and as long as the fruits continue to ripen, the chicks will have an abundant supply of food.

The kakapo story shows the value of using sensors and monitors to collect data that can be interpreted and used to the advantage of a species…but what if the same technology and methodologies were used to track teenagers at risk and to monitor young pregnant human mothers? Suddenly the idea of intervention becomes disturbing.

Technology is generally neutral and it is how we use it and what we do with it that matters. Big data and big data solutions exaggerate the good or the damage that we can do with ICT systems and data sharing.

How can we assist individuals and organisations in making better choices around the use of big data and big data solutions? Help is available in the form of a Voluntary Code for Data Sharing, developed by Alison Holt during her time as an Academic Visitor at the Oxford Internet Institute, Oxford University. The code provides seven maxims for individuals or organisations to consider when collecting or exposing data, together with references to relevant best practice and examples of organisations already applying each maxim. The code is still in draft form. It will go out for public consultation later this year, to ensure that the final code is fit for purpose and suitable for organisations of all sizes and all sectors.

With so many stories making the headlines due to ill thought out uses of data, it is hoped that the Voluntary Code will assist organisations in considering the consequences of sharing data before lives or livelihoods are put at risk.