I’ve just read a research report this afternoon, totally flawed from objective to methodology. So much so that I still find myself completely dumbfounded. A massively huge ‘HOW’ is hovering in my cerebral cortex and it simply can’t seem to find its way out of there.
It’s strange, but part of me can’t help but wish life was as easily understood and our minds as simplistic in their processing as the people behind this study seem to think. Because by their reckoning, it would be a perfectly valid methodology to judge the taste and aesthetics of a cake by tasting and observing its ingredients separately….flour, sugar, eggs, butter and the rest of it.
This ‘experience’ reminded me of a quote that I came across when we pitched for the Sony APAC business in my previous agency. Not only is this quote music to my ears, but its also evidence of an understanding of the severe limitations of methodologies involving direct questions to people. I’ve previously touched on this here and here.
“We don’t ask consumers what they want. They don’t know. Instead we apply our brainpower to what they need, and will want, and make sure we’re there, ready”
Akio Morita, Co-founder, Sony Corporation
April 3, 2008 at 4:29 AM
When people say “consumers don’t kow what they want” it drives me mad, because whilst they may not be able to articulate their needs/wants clearly [because of their frame of reference] you should still be able to understand the underlying need which can lead to the development of a powerful solution.
The famous quote banded about is when Henry T Ford said if he’d asked people what they wanted they’d of replied “A Faster Horse” but if you look underneath that statement, you’d see that people are saying they want to get to their destination faster – which still could of led to the development of the car.
Consumers are stupid, it’s the people who want to be spoon-fed their needs/wants who are.
April 3, 2008 at 7:00 AM
I agree with both. I recently went to a seminar about etonographic studies with a very competent antropologist. she wa´s 100% against focusgroups because nothing good come out of them. I’d like to say that, yes that is often true, but not because nothing good comes out of them but because they’re used wrong, for the wrong reasons. Rob’s comment shows that a focus group can give you a hint, but digging into that answer (using your brainpower and knowing more about what is possible, than the respondent) something good can come out of it. Not as much as clients would like, however.
On the subject above – I love kaviar, kebab, hot dogs and beer. I always mix them and it’s 4 times better tasting than when consumed separately. NOT. What idiots.
April 3, 2008 at 1:24 PM
Yes, that’s exactly the point Rob, and that’s what Morita realised.
O.S. Welcome to this blog and thanks for your input. I’ve actually got nothing against focus groups in general. It’s just those that are used wrongly: those asking consumers to articulate something they can’t be expected to know anything about since they don’t have acccess [literally speaking] to the information they’re being asked for.
April 3, 2008 at 2:25 PM
I wonder if a hybrid form of etnographic research and focus groups is the best of both worlds. For instance having people perform tasks, without them knowing what the thought behind them is and then having moderators ask specific questions to get real (not socialy accaptable) answers?
It’s a bit like the SAS, and I presume every special forces, train their people in reading maps. they have them run and do stuff the whole day, and when they are exhausted they have to read maps to get back to camp. this allows instructors to see and hear from recruits the difficulties they have under changed circumstances.
just a thought
April 8, 2008 at 2:47 AM
Hi Niko…yes, it’s not easy to by-pass the underlying issue with focus groups: what’s said in them rarely have a lot of bearing on reality because of a number of factors. I think as soon as you have lab-like situations with direct questioning, it’ll be hard to avoid mis-leading research since the methodology is predicated on the idea that the respondent is able to give an accurate report of his/her behaviour.
The SAS analogy [solution] is probably a better alternative than what we have in place today, but that map reading exercise that you mention would still be a very different task in peace time from what it would be when you have grenades going off around you and people aiming bullets for your head.
April 8, 2008 at 7:07 AM
that is a good addition on the map reading example. I suspect that the trainers also recognized that flaw, so they build in psychological fears in this specific exercise. Missing a meal for instants, or actually being left behind if the task is not complete. Now this may sound mundane, but as this exercise is the last in week of sleep deprived running, shooting, etc, they have huge effect on soldiers. Some just break down.
I am getting of track here, but the basic point still is something one should investigate: get people to do stuff (to a point where they no longer are self concious-that is the trick i think-) and then you will get true reactions, in actions and words..
You should try the following, if ever in a poker game: when still dealing out the chips and making small talk ask a opponnent his or her favoarite colour, because the guard is down, they are not self aware, you will get a truthfull answer, spoken at a natural tone. Now during the game you can ask them all kinds of questions, and by the tone of their voice you will know if they answer is socialy accepted or not…same goes for focus groups
April 13, 2008 at 3:10 PM
I know what you mean about the psychological fears Niko and it can definitely help a great deal to bring you closer to reality. I was actually subjected to these sorts of things [though it wasn't in the SAS] when I was in the army [a long time ago].
The poker trick is a great one and useful in many situations. But I think it assumes that the change in vocal tonality results from a deliberate lie [a ‘socially adapted answer’
], which is only one problem with focus groups.
The biggest problem with focus groups isn’t ‘lying’ but untruthful answers nevertheless. Respondents don’t have to lie per se to tell untruths. It’s just that poor research often ask respondents to explain their behaviour and motivations behind it when we know scientifically that our behaviour is predominately driven by processes occurring in our subconscious minds – the part of our mind we don’t have access to. The method, however [or poor research I should say], wrongfully assumes that we’re able to give an accurate report of why we do things and what motivates our actions.
April 17, 2008 at 11:40 PM
OK, I know you’re busy and all that but can we have a new post please? Pretty please?
April 24, 2008 at 11:12 PM
I know I’m being a pain but COME ON!