Big data: The trend that could change your business

Big data is a game changer for business.

In an age of computers, GPS devices and fleet GPS transceivers, RFID tag readers, smart meters, embedded microprocessors and sensors, companies are now confronted with a tsunami of data. In a society where downloadable software applications allow people to do virtually everything from their smart phones, where geospatial applications like Google Maps generate vast quantities of transient data every day, where every transaction is now recorded as well as every potential buying decision and where digital video technology tracks people’s behavior, there are no limits to what’s around.

Smart companies are combining the external data, coming in from everywhere, with the structured existing data in reports and columns. The aim is not just to make faster decisions in real time. It’s also about analysing the data, learning about things that were previously unknown, and incorporating them into business models to build sales and improve customer retention.

The world’s biggest retailer Wal Mart, for example, has been a pioneer in predictive analytics using big data. While other retailers stock their stores with the same products, Wal Mart identifies customer preferences for each neighborhood and stocks its stores accordingly.

Every Wal Mart store is different. It will look at all the data for the neighborhood – what kind of houses people are living in, what sort of jobs they have, earning capacity, family structures and cultural composition to name a few – and then stock their stores to match the neighborhood. Wal Mart handles more than one million customer transactions every hour. Its databases are estimated at more than 2.5 petabytes, the equivalent of 167 times the books in America’s Library of Congress, and it monitors these assiduously to make sure each store fits in with its neighborhood. With this strategy, Wal Mart was able to break away from the pack.

Wal Mart is a trend setter for business and we can expect more companies to follow. We are now seeing the development of a new business nervous system generating a cloud of data about people. This is growing at an exponential rate.

Businesses to cash in via “intelligent devices” usage

According to IBM, Intel and other IT giants, there are more than 60 billion “intelligent devices” in the world today and they forecast this figure will rise to more than 200 billion by 2015. It’s a big opportunity for any business mining these networks to understand who likes what, where they are from and who is interacting with who.

That creates enormous management challenges for companies. It’s not just about accessing the data, it’s also about knowing how to analyse it properly. You need to slice and dice it to find things of statistical significance that were not apparent before, things like customer churn and purchasing decisions.

The amount of data is challenging enough. According to consulting firm McKinsey, companies with more than 1,000 employees store more than 235 terabytes of data, more than what’s contained in the US Library of Congress. Managing big data exceeds the capacity of traditional data management technologies. It not only requires new and exotic technologies to manage the volume, it has to be analysed too.

Over at Qantas, for example, experts believe teams are now trawling through the internet, checking out every tweet, blog, Facebook entry talking about the airline’s nation-rattling decision to ground its aircraft. Social media monitoring services like US-based Radian6, which has software that tracks conversations on the internet and which scours the web for any mention of a company and compiles reports on its findings, are now doing huge business because of big data.

Another piece of software heavily in demand is Hadoop. Started by Yahoo! and released as an open source project in 2007, Hadoop has operating system that can crunch through huge amounts of data that existing systems cannot process.

How much is all that worth? Salesforce.com this year bought the privately held Radian6 for $US326 million ($A314.8 million) to help its clients track customer trends on social media sites such as Facebook and Twitter.

Otto Ottinger, co-founder of big data analysis company Curated Content, puts it bluntly: “I would be absolutely sure that within Qantas, Radian6 is now hot. I am sure they have an in-house package of Radian 6 and probably have been trained by the Radian6 people.”

Ian Bertram, Gartner vice president of research in the Asia-Pacific, says it’s not just a matter of checking out what’s being said, it’s about analysing it and connecting it to the company’s conventional data.

“What they would be doing is trying to put some context around it, they would be trying to mash that up with their structured data which is where you get hybrid data,” says Bertram. “So they would be looking to see if it’s anyone from their loyalty program, a platinum member, or a chairman’s club member. Are they tweeting about us, are they blogging about us? Do we need to go and reach out to them?”

Of course, it’s not just about checking what’s being said about you on blogs and social networks.

Retailers, for example, could use it to track customer likes, dislikes, influences and behaviours. All that will allow them to adjust prices in real time, re-order hot-selling items, shift items from store to store, and better manage their inventory.

Online businesses can use it to get a better understanding of customer preferences and social interactions. In financial services, it would create better risk assessment and allow the company to better tailor its offerings to the individual customer.

Media and entertainment companies will be able to collect large amounts of rich content as well as monitor user and gamer playing behaviour data.

Health care companies can access electronic medical records for instant diagnosis and treatment of diseases.

Transport and telecommunications companies can better track customer preferences and, like financial services companies, create offerings targeting specific customers.

Companies working in this space and managing big data say there are obvious opportunities to tailor offers for customers, making them feel special, creating some loyalty in the hope of generating more return business.

Telcos magange customer churn

Big data analyst Tony Bain, a director at Rock Solid SQL, says “telecommunications companies manage customer churn by analysing call patterns and developing a social graph of who people call, who calls them and who else is called. “

“What they found was that when they analysed the implicit social graph was that they would be very likely to lose someone if someone they knew changed provider,” Bain says. “That allowed them to start offering incentives for customers to stay.”

Similarly, electricity companies have found that customers who contact call centres regularly are more likely to change. Again, it provides them with the opportunity to offer those customers incentives.

He says that in the US, companies are now combing through Facebook and Twitter for their brand analysis.

“People have conversations about brands and make recommendations all the time and the fact is this happens more online. Companies in the US are now starting to analyse Facebook and Twitter to work out whether customers are happy with the product, or not. They’re looking for groups where discussions are going on and whether people are unhappy with the product or service.”

A retailer for example would be dealing with data from RFID devices, over the counter transactions, online transactions and website visits. Some retailers might even add to that social media analysis tracking where their customers are going, what they’re saying and who they are connecting up with. It could find ways to analyse the foot, web and transaction traffic, identify where particular customers are coming from and give them suggestions for what they could buy next. It’s not dissimilar to the approach taken by Amazon, Apple and LinkedIn.

Alec Gardner, industry consultant director for Teradata’s South Pacific operations, says that big data will help any business, particularly retailers, develop closer relationships with their customers.

“The value of big data is that with the pressure on the customer wallet comes a requirement to provide better customer experience and you do that whether you’re a bank, a telco or a retailer, and you do it through many channels,” Gardner says.

He says big retailers in Britain analyse customer browser behaviour and what they are buying. This allows retailers to then follow up with suggestions for the customers, Amazon style.

“That is something we now expect as consumers,” he says. “It’s like the café across the street. They know my name when I go there, and they know what I have. Big data allows you to create something similar.”

He says call centres use big data to better manage their staff during the hectic and slow periods.

“They are using these types of processes to optimise staffing, it makes them more effective.”

Customising your offer

Duncan Bennet, managing director of VMWare, says the big advantage is that it helps the business customise an offer, providing something special for the customer.

“It’s a unique business opportunity. You want to be able to utilise that data in real time and make a decision while the customer is standing there at the counter,” Bennet says.

“If you are a real estate agent, you will have transaction records, browsing patterns, digital pictures and social media analysis sitting and you can get all that together and make decisions while the customer is sitting in front of you. Or maybe the customer in front of you is Mary and you know Mary last time bought all these other items so all that information is coming at you in real time whereas in the traditional world, it’s all stored in a database which you can search through but it’s hard to do it in real time.”
He says it would also allow retailers analysing the traffic to manage their staff better, ensuring they have their gun sales teams on at busy times.

America is the home of big data. Indeed, the US has extraordinary sites like Infochimps https://www.infochimps.com/ which carries everything from Census data to the trustworthiness of Twitter accounts. Still, analysts here expect it will be reshaping Australia in five to 10 years. Indeed, it’s already being picked up here. One of the best examples here is the Australian Bureau of Statistics which rolled out a webapp https://spotlight.abs.gov.au/ that asks you for personal information. It then uses data from the last census to generate a series of infographics comparing you to the population at large. The site includes a laid back narrative by Shaun Micallef. Just one example of how big data can be used and made accessible.

The three V’s – volume, variety and velocity

Speaking from New York, Ovum analyst Tony Baer says it all comes down to the volume, variety and velocity of the data coming in and that creates analysis challenges.

“The variety is with all the different sorts of data coming in,” Baer says. “It can be everything including call records, graph analysis, web log details and social networks. With the usual structured data, it’s data you can manage. But this stuff is unstructured. Variety means lots of different types of data you wouldn’t normally see in databases. You are looking at lots of detail that would not normally be tracked.”

While he does not see small businesses leading the charge to big data, he believes it has enormous benefits for small companies.

According to one Ovum survey, 17% of companies in the US analysing big data had revenues of less than $10 million.

“It’s one area that can be really productive for small business because you can see what people are saying about the product on, for example, Facebook and seeing who the most loyal fans are and who are the opinion leaders.”

“In the past, you would have got a focus group ahead of time where you test a message, or you do a survey after the fact. What you are doing with big data is checking it out in real time.”
He says most small businesses will be turning to cloud based data services to order their analysis. The most obvious contenders, he says, would be retailers and financial services.

But others say any business should use it. Cath Pope, co founder of Curated Content which pulls all the data together and puts it into easy to understand formats, like info-graphics, says it applies to all sectors. “Can you think of an industry that doesn’t produce data that can’t be visualised?”

Curated Content is in the business of collecting data for clients and then putting it into an easy to access format, like info graphics. “We are finding with clients that there is so much data that they can’t see easily and comprehend it so we visualise it for them,” Pope says.

“We are finding that SMEs are attracted to infographics because they are more cost-effective and ever green. They travel better, they are quite modern and look really good in an iPad rather than just putting it all into an article.”

She gives one example of a client, a large data company. Curated Content collects data on what that company’s clients do on websites, what their shopping habits are, who they are competing with, what customers are buying online and who is responding to advertising.

“We are not necessarily talking about thousands and thousands of pieces coming in each time, most SMEs aren’t NASA,” she says. “But there is data coming in from everywhere and this is helping them understand and visualise.”

Curated Content uses software and algorithms to manage the data. It all depends on the client and what they want to do with the data.

She says the clients all have marketing managers who want to assess the success of campaigns.

Bertram agrees that it can be used by any sector, but singles out fast moving consumer goods, fashion, manufacturers, big retailers and even miners.

“Banks are using big data all the time,” Bertram says. “Telstra did a lot of analysis pinpointing influencers across their network, Optus are doing a very similar thing.”

“I can’t think of an industry that wouldn’t want to invest in this type of technology and invest in understanding what’s happening out there in the market place.”

He says the tools are not that expensive but the big challenge is incorporating all the unstructured data, stuff that doesn’t come in rows and columns, with material that is structured. It is creating a system that can manage structured and unstructured data, and incorporate both.

“It’s not a rip and replace,” he says.