Why big business culture can lead to big data analytics failure

Much of big data comes from people. Web logs, mobile phone usage, financial transactions, insurance claims, you name it: it’s being recorded for potential further analysis to generate business value and improved customer experience.

It goes by the name of customer analytics, and large retailers and service providers, at least in the US, are obsessed with it.

Online businesses are significantly ahead of traditional bricks and mortar businesses when it comes to leveraging data to drive business value. The major reasons are cultural, social and operational.

These online businesses are much closer to a truly scientific culture in which every idea or proposition is automatically considered a hypothesis subject to testing rather than a heavenly insight for which the burden of evidence can be waived. They not only have an obsession with measurement, but also with experimentation.

A scientific approach to business

The design of experiments, data collection, analysis and understanding are what characterise scientific enterprise. So, in order to embrace big data, it’s necessary to embrace science, meaning its values, culture and new methods based on machine learning, which is the automation of hypothesis generation (from data) and testing (against data).

Yet, a scientific culture is not what you will find in a typical bricks and mortar business.

The top online companies have researchers and scientists who seriously understand science and its new machine learning method. Google recently hired Geoffrey Hinton, the father of neural networks and deep learning (instances of machine learning). It has also just reportedly acquired, for US$500 million, a startup comprised of deep learning experts.

Facebook followed suit by catching Yan LeCun, who pioneered the use of neural networks to solve large-scale real-world problems.

Extracting value from data requires not only the right tools but also the right leaders to build the right teams to use these tools (and build the ones that still don’t exist).

Bricks and mortar businesses in general do not have such people on board, although those who are ahead are desperately trying to hire them. The bad news is that the demand for those people is way, way beyond the supply.

Another crucial point is that those giant online properties have an operational model in which the results of the science can make their way into every decision that results in some intervention, with relatively small cost. This is in contrast to traditional businesses that are burdened with a range of channels each with legacy IT systems and human processes.

Story continues on page 2. Please click below.