Everyone’s talking about it, but what does Big Data mean? David White explains
It’s important to realise that this – the amount of data we are seeing right now – is not really that big.
We are just on the cusp of, what is going to be, a data explosion. To give you an idea of scale, in 2012 there were 2.7 zetabytes (trillion gigabytes) of data on the web. That number is doubling every two years. The internet – much like the universe – is expanding exponentially.
So where is all this data coming from? Part of it is from us.
Individuals are increasingly putting more and more data up on the web. An increasing amount is also coming from corporations and governments.
The internet of things
All this pales in comparison, however, to the amount of data that will be generated as the machines become connected. The internet of things is a rapidly approaching future in which almost everything we interact with, from smart phones to refrigerators, is embedded with sensors which will be constantly gathering data and uploading that data to the internet, 24/7.
These trends and changes will bring about a data universe that is truly Big, and most importantly, diverse.
The diversity of data is something that needs to be embraced and harnessed. Insight is what underpins the Big Data promise and processing the same limited data set that you already have is not going to reveal anything new.
Adding disparate and diverse data is the only way that you are going to start to reveal true cause and effect and begin to realise the insight that was promised.
We haven’t reached the height of big data yet
The true days of Big Data may still be a little while off, but it is rapidly approaching, and it is important that we understand what that is going to mean.
The possibilities of what we can achieve in a data driven world are immense.
Companies have access to customer spending and browsing habits and will be able to provide them with real-time offers that are tailored to their personality, mood, previous behaviour and location. Commerce will become incredibly personalised. Similarly, technology will help students choose majors and careers by analysing previous scores, learning style, and even personality.
Using this method we will be able to tailor the learning experience to each person’s individual needs. Also, the use of machine reporting means that we won’t have to rely on the notoriously unreliable self reported data that humans provide.
Your doctor will be able to track your exercise, food intake, drinking and maybe – if hapi fork takes off – even your rate of consumption. All this will allow for a more accurate picture of who you are and what you are doing, which leads to better analysis and – hopefully – better decision making.
Big Data has the power to move beyond the surface issues of buying behaviour and weight loss, to be able to address macro issues with society. Healthcare is a great example. With such a big (and complex) organisation, getting the right data can be life saving.
For every patient “category”, the back office has a prediction of how long that person should be in hospital and, as of now, doctors have little to no visibility of that data. If they did it would allow them to assess a patient’s progress in relation to other patients, compare treatment methods and determine if a different treatment method needs to be implemented.
Big data comes at a price
All this innovation comes at a price however. In order for companies to tailor deals specifically to you or for your doctor to be able to see your health progress in real time, you will have to give up an enormous amount of personal information. Is this necessarily a bad thing? No. But it is something that people need to become aware of so that they can make informed decisions about when they will and will not give out that data.
At a certain point the equation shifts from a comfortable level to an uncomfortable one. What that point is, no one can predict. As technology becomes smarter, so too, do people need to become smarter about who they allow access to their information.
Big Data is coming. With that will come enormous benefits, both to the individual and the collective whole.
With that will also come a greater responsibility; of the individual to make sure they are protecting themselves, and on the corporations to make sure that what they are providing is worth the data they are asking us to give up.
Big Data is what it is, a tool. How we use that tool is up to us.
David White is CEO at Import.io and will speak at the Innotech Summit on 26th April 2013