Crunching the numbers on big data

One of the big growth areas at the moment is big data.

As we all live in an increasingly digitised world, we are producing massive amounts of data about ourselves that savvy organisations are slicing and dicing to their advantage.

There are a large number of consultancies offering their big data expertise, technology companies building ever smarter and faster big data tools and universities offering Masters degrees in the discipline.

Putting aside concerns about the ethics and privacy aspects of the issue – especially when it’s used by governments – there is no doubt that big data offers companies insight into their customers like never before.

Suddenly, they know that a person who buys cat food is four times more likely to buy English muffins on a Wednesday morning. (I made that up, by the way.)

The kinds of (real) insights that big data generates are tremendously valuable for companies when it comes to their marketing and purchasing and general business strategy. They can become far more scientific in the way that they structure their offerings and activities.

And as a result, there is an arms race among companies to harness the power that these big, hairy algorithms provide.

Some people argue that the whole trend will implode.

They predict that as users of Facebook (etc.) realise that their account information is packaged up and sold to businesses, as shoppers realise that rewards programs help companies manipulate them into buying more and more stuff, that eventually there will be a backlash.

I’m not so sure.

But what happens when every company knows everything about everyone? What happens when companies have crunched all the data there is to crunch?

At that moment, big data will no longer be an advantage. It’s a bit like the first company to rent a photocopier. Instantly, their office became far more efficient and it offered a genuine competitive advantage.

As more and more companies got photocopiers, the advantage dissipated. It became a commodity.

What happens when the same thing happens with big data?

Don’t get me wrong. It’s powerful stuff. The people at the cutting edge of the discipline are incredibly bright and deserve to cash out at big premiums.

But at some point, when data becomes ubiquitous, business is going to have to reach out to that skillset that has always made it uncomfortable – creativity.

Every company will know exactly where I am and exactly what I want. That’s great when you are the only company offering me a widget.

But when there are five of you that have all crunched the same data to work out my exact preferences, you better be offering me something amazing to make me pick you over them.

At that point, design, branding, advertising etc. – the human arts that have been under attack in recent times by the quantification of the universe – will be all powerful again.

Well, that’s my take.

But you would have known that there was a 78.9% chance of that being my view. All you had to do was crunch the data on my LinkedIn page. It says that I used to be an advertising creative and that one of my areas of study was philosophy.

This article first appeared on StartupSmart.