What do Snowden, Nandan Nilekani and Mark Zuckerberg have in common?

Well, they are all related in one way or the other to Big Data game.

As we all know, Edward Snowden became a bit of a celebrity when he raised a furore about the US National Security Agency’s data-collecting program and went on air with his statement that he doesn’t want to live in a world where everything he says and does is recorded. Wonder what he’d do when he realizes he was “blind’’ to the fact that every single time he swiped his card at the retail store or walked past security cameras he was leaving a trail of his identity, purchase behavior, consumer psychology and more.

Coming to us, the non-whistleblower kinds, every time you hit the ‘LIKE’ button on a casual browse of Facebook, review your favourite book on GoodReads or load your recording on Soundcloud via your Facebook login, you are leaving behind a trail of your identity and preferences.

Closer home, the Aadhar Card initiative is an example of a big data exercise. This nation-wide program captures data right down to your fingerprint and vehicle number plate and the unique id is in turn linked to host of everyday transactions – from loan applications to  LPG gas refills to buying a drug at a pharmaceutical store to buying a ticket on Indian railways.

The above cases only elucidate that data supply has seen a rapid growth in the last decade – in volume, velocity, variety and veracity in virtually every area of our lives.

However, what we need to be cognizant of is that all of this data is just Meta data. To quote what many marketing gurus have commented on earlier, “Data is the new oil“. Data, like crude oil, is very valuable, but cannot be put to profitable use in the unrefined state. It has to be refined and distilled into gasoline, diesel, asphalt etc, to be put to commercial use; so must data. Data needs to be analyzed and insights tailored, for it to drive business value.

Without the analytics, any data would be, just Big. This is where analytics comes into play – Decoding meaningful insights from data to drive big results. It is only when we use data to solve business problems that are we addressing the demand side of Big Data. Over the last couple of years there has been unprecedented attention on data explosion that is now available from multiple sources and in multiple forms (text, video, chat transcripts etc.) as well as the technology used to capture this data. What companies also need, is investment of significant time and resources on data scientists who can slice and dice this data to provide meaningful and actionable insights, address key business questions, help businesses prioritize their focus areas and find areas of opportunity. If these insights and decisions could be provided in real time, that’s even better!  In the long run, data based decision management will emerge as the key differentiator for businesses.

Billy Beane’s artful management of a relatively low funded team – Oakland Athletics to remain competitive in Major League Baseball with the help of player statistics gathered over the years truly explains how data based decision management can be the key, apart from being a great story for the Hollywoood blockbuster Moneyball.

Likewise, Zuckerberg and his team don’t just record activity of users but create an information web and use all of this info to propose a host of interesting recommendations to act on – from friends to network with, to groups to sign-up for to artfully curated and tempting product offers that a user is most likely to click and buy.

What we ought to remember is that we have historically been capturing data.  What has undergone a change, is the kind of data we capture and analyse today and the volume.  About a decade ago, we largely captured transaction related data. While this sort of data was considered unstructured in the late 90s, it is perfectly structured data today. We even have sophisticated ERP systems around this data today and fairly sophisticated analytics tools that can predict sales, given neatly tabulated historical data. Similarly, while chat transcripts, text from discussion threads, video recordings and RFID data is considered unstructured data, difficult to assimilate and analyse, it only a question of time before this kind of data will go through a process of evolution.  We will have soon have sophisticated algorithms that can break down this data into meaningful bytes and also integrate with other sources to provide meaningful insights.

On a closing note I’ll leave you with an interesting anecdote on Big Data, “FBI apparently, closely monitors a lot of pizza delivery outlets. Wondering why? Well, people in hiding tend to have food delivered, and make a lot of calls for pizza!

This blog has been authored by Bhargavi Shankar. Bhargavi is a Sr. Consultant at BRIDGEi2i Analytics Solutions.