Problems with focus groups and market research: Some real world examples

‘The trouble with market research is that people don’t think how they feel, they don’t say what they think and they don’t do what they say’ – David Ogilvy

What are the points David makes and what are examples of this and its implications? Over the years market researchers have had their sacred cows, one example being the classic focus group. Here, we put consumers together in a room and ask them about how they feel, what they think and what they will do.

On a larger scale many millions of dollars have been spent centred around the assumption that asking people, for example in focus groups, will give the information required to design effective brands, products and advertisements.

So let’s look at these assumptions by way of three examples.

Do people think how they feel?

There are many examples that illustrate the gap between thinking and feeling, and for British Airways this resulted in having to throw out lots of beautiful, fresh produce years ago. Why? Because when deciding to introduce a private mini-fridge full of treats as a new service for first-class passengers, British Airways conducted focus groups to find out what people would most feel like when waking up at night craving a snack. Consumers thought their answer was clear: healthy fresh fruit and salads.

It was only because of an experienced stewardess’ concern that British Airways decided to also add a couple of chocolates and cakes to the fridge, to see what happened. Checking the fridge back on solid ground, staff found that when travellers woke up at night they obviously felt no desire for healthy food: all the cakes and chocolates were gone, while all the salads and fruit were still there.

Do people do what they say?

Well, hopefully our friends, family, colleagues and suppliers do what they say when they promise something. But, as far as customers are concerned, there’s no reason to be too optimistic. Even if we think we know what we’ll do in terms of consumer behaviour we are, in fact, terrible at predicting our future actions (see my blog on how imprecisely people can predict what they would actually pay for a product). I’m not saying we don’t all have our great reasons, of course we do, but we don’t know what these reasons actually are.

Therefore, our behaviour more often than not takes a direction we cannot predict in advance. An example? A well-known food and beverage company tried to determine the ideal size of their cookies by way of asking consumers what they preferred. The results suggested clear direction: people said they would prefer the bigger cookie option, because after all, who wants the small piece of pie?

When tested in a real shopping experiment where both a bigger and a smaller cookie size option were presented consumers actually opted for the smaller cookie option, contrary to what they had predicted they would prefer. Why? Because in a real-life comparison they thought there were more cookies in the smaller size option.

Last not least: Do people say what they think?

The answer to this question involves two different problems: conscious thinking makes up only about 5% of what is going on in our mind. So even if we say what we think it is only a very small part of what is actually going on and more to the point it is often not the part that matters when we make purchase decisions (we have explored this in detail in earlier blogs, for example the limbic system, the cognitive mind and the user illusion that misleads, and subconscious mechanisms at the point of sale).

Second, it is unlikely that we even get to the truth, as far as this 5% of conscious thinking is concerned, in a focus group situation. This is because most people don’t say what they really think in social situations. To quantify this problem: when Solomon Asch experimented in 1956 with the impact of what others around us say on what we dare to say in a group situation he found that 75% of people don’t actually hold their ground and express what they really think when everyone else has a different view.  

Solomon Asch, Experiment with the perception of the length of lines, 1956

In other words, when seven people in a room claim that line A and C have the same length, the eighth person in the room will most likely agree in 75% of cases. Some will agree because they actually start doubting their own perception – we tend to think the herd must be right – while others will agree simply because they lack the nerve to express an unpopular opinion.

Regardless of the reason this is the very real situation of a focus group – people tend to express homogenous views about products, needs, etc and will systematically not express more unusual, unpopular or socially undesirable thoughts. And this is only the minor problem compared to the fact that even if people wanted to, they wouldn’t be able to access the subconscious, hidden drivers of their actual behaviour in an interview-type situation.