There two things we can all agree on when it comes to big data.
1. Startups think they’ll make a fortune accumulating it
2. Big companies think they need to be using it.
Both of these might be true, but there is one thing I know to be true for sure: There is a lot of confusion when it comes to Big Data. And before I go about demystifying it, we should probably agree on what it really is and where it comes from.
What is Big Data?
Well, here is a formal definition:
“Big data is a broad term for data sets so large or complex that traditional data processing applications are inadequate. Challenges include analysis, capture, data curation, search, sharing, storage, transfer, visualization, and information privacy.”
But this definition is kind of useless and somewhat alienating. It almost says, ‘Never you mind about big data, you leave that to us’. Truth be told, as computing power continues its exponential growth curve, anyone with Nasa in their pocket (smart phone) or a cloud connected laptop can use and curate Big Data.
Here’s a better informal definition written by yours truly:
“Big data is the information we (you and me) create every time we interact in the digital world. And these days there is shit loads of it. Which is why it is big.”
Pretty much everything we do which involves electricity creates data these days. Our digital footprints, or information trails we leave behind enable people in business to go from guessing what is happening, to actually knowing. Because almost every human in modern economies is creating data, we can also predict certain behaviours by matching up different data points to create new insights.
Where does Big Data come from?
In the simplest terms Big Data is the story of what we read, what we buy (without cash), where we go, who we follow, what we watch and what we listen to. This story is mostly told by our smart phones and internet interactions. However, I’d add that these interactions are now starting to increase. As we enter the internet of things era, nearly everything we do is tracked and turned into data. Web enabled products from light globes, to cars, to TV’s, to fridges, to running shoes, to watches, to closed circuit cameras in cities to tollways all leave data behind. A good way to think of where big data comes from is this: Anything and everything we do in a public realm creates big data. This certainly includes all of our government interactions, all of our commercial interactions, and all of our digital social interactions.
How can we use Big Data?
So now that we know what it is and where it mostly comes from, the question is how the heck can we use it? Instead of telling you how to use it, I’d rather give you an example of how it has been used which will make it simple.
A great example of Big Data in action is how Google use it to generate live traffic reports on their Google maps app. I’m talking about the traffic visual below which shows how busy the roads are in real time. Red is busy, orange is moderate and green roads are flowing.
It’s easy to think this would involve complex algorithms and satellite imagery to determine how busy our roads are. But they use something much simpler than that. And when I first learned how they generate such accurate live traffic reports it blew my mind.
Here’s how it works
They have all the roads mapped already. When we sign up to the Google maps app, the phone uses our GPS to locate us. It then sees how fast we are moving on the road, and overlays this with the speed limit of that road. They then put these 2 simple data points together to be able to give an accurate account of how fast or slow the traffic is moving. Simple.
It is in essence, 2 simple data points. Made possible by the amount of ‘Big Data’ on our roads as almost everyone has a smart phone. This is the best example I know of using Big Data, with a simple insight, to create real value for end users. It’s way more accurate than any helicopter traffic report has ever been. But more important than that, it shows how imagination and putting the pieces together to make the report is where the real genius lies – not in complex technology and algorithms.
So when thinking of how to benefit from big data we should remember this example. We should remember that we can gain enormous end user benefits from a few simple data interactions, and a bit of imagination. We don’t even need to own the data, but merely put the information together in a new way to solve an old problem.