|
Archive for the ‘Surveys’ Category« Previous EntriesUnless I’m Very Much MistakenFriday, September 10th, 2010
This week, Oliver Truman looks at some of the honestly-held misunderstandings we make in everyday life, and at why misconceptions about the market research industry should make us sit up and take notice. Sometimes we all make mistakes. As human beings, we’re loath to admit our failings, particularly when we think we might have got something wrong. Only the other day, a friend was bemoaning the arrival of students back into Manchester for the start of the new academic year. Longer queues at cash machines, processions of drunken youths in fancy dress and buses packed to the rafters were just a few of his misgivings. “Bloody retrobates”, he muttered. At first I hadn’t realised, but after a few seconds it sunk in. “Did you just say retrobates a second ago?”, I enquired. “Yeah, retrobates. You know, delinquents”, he said. After several more verbal exchanges, it became apparent that my chum had been using the word retrobate instead of reprobate for quite some time, possibly even his entire life. As it was perhaps a little too painful to admit it, he gamely attempted (for several minutes) to argue that he was right and I was wrong. However, in the age of instant access to knowledge, a quick mobile web search revealed the error of his ways. The score was settled. Throughout that evening, as several more pints of English Ale were imbibed, me and my friends at the local pub were now alert to the slightest error – whether linguistic, factual or otherwise. Other highlights in the inaccuracy stakes that evening included:
Errors of the retrobate sort are referred to by linguists as “eggcorns” – The term itself involving an idiosyncratic substitution of similar-sounding words to mean acorn. There’s a tremendous website documenting these everyday anomalies at the Eggcorn Database. Some personal favourites include:
In the world of market research we are, to an extent, also on the receiving end of popular misconceptions about our industry and the work we do. There was an interesting article on the BBC website this week about those who respond “don’t know” in opinion polling. Aside from the thought-provoking discussion about how such responses should be treated when reporting survey findings, it was the comments section at the end of the page that really grabbed my attention. Here are a couple of comments that made me realise just how misunderstood the market research process might be amongst the public at large:
Recently on the radio, there was a phone-in on the subject of a recently conducted opinion poll and listeners were asked for their comments. One caller refused to believe the result, citing the fact that it was a survey of “only” 2000 adults. “I’m sure the other 60 million people in this country don’t think that way – It doesn’t capture that they think” was their claim. On the flipside, us survey wonks should also accept that some of the blame rests with the research industry. Market researchers don’t help themselves when we talk to non-research audiences about sample sizes, weighting and quotas. Moreover, research also needs to be conducted in a way that is likely to engage and learn from, rather than alienate the audience. Unless the most appropriate techniques and methods are deployed, the credibility of the research process can be put at risk. The comment below came from the comments in the same BBC article I mentioned earlier. I think it neatly captures an instance in which market research really doesn’t help itself:
Positive Outlook for Industrial CompaniesWednesday, July 21st, 2010  Good news this month for many of our clients! According to a new report, Trends in Industrial Marketing 2010: How Manufacturers are Marketing Today – incidentally published by one of our clients, GlobalSpec – 70% of sales and marketing professionals in the industrial sector expect their companies to increase sales this year, compared with a mere 16% in 2009. However, the online survey of 464 respondents found that only 31% had an increased marketing budget. For 42% of sales and marketing professionals questioned, customer acquisition is their primary goal this year. One-third is focusing on lead generation and 13% specified branding as their main aim. More than two-thirds (68%) plan to increase spending on social media. The same percentage intends to increase their spend on online video this year. Conversely, a quarter plans to decrease spend on trade magazine advertising and 24% will decrease use of printed directories. How Do You Digest The Opinion Polls…..With A Large Pinch Of Salt!!Thursday, April 29th, 2010  When faced with producing a Thursday Night Insight, it’s a challenge to find a suitable subject to write about. However, my challenge was solved when UK Prime Minister Gordon Brown set the date of May 6th 2010 for the General Election. And so the media frenzy began with journalists presenting special election reports every hour of the day and night. But has the prospect of a General Election really gripped the nation? The ad-hoc research I’ve conducted in local pubs and bars (well, someone had to do it!), has highlighted a lack of discussion of party policies and more of a focus on the “lighter” side of politics (more of this later!). This is certainly the case in our office. There was astonishment when my colleague, Oliver Truman, announced that he was taking the day off after the election because he wanted to stay up late watching the results come in. But is it really the electorates’ fault? I would argue not. Another conclusion from my research was a perception that the three main parties and their key issue policies are fairly similar to each other. Gone are the days of the Conservative and Labour parties being at opposite ends of the political spectrum with the Liberal Democrats alone in the centre. This feeling has been exacerbated by the MP expenses scandal when all three parties were seen to be as guilty as each other. One thing that has been different this time has been the introduction of the Televised Presidential Debates, sorry, that should be Televised Leaders Debates! These debates were billed as many things: “History in the making”; “A new age of politics”; “The election in your living room”. However, to quote the Northampton Chronicle & Echo, the first debate was “ultimately a bit of a damp squib, with well-rehearsed answers and fully briefed participants.” This theory was confirmed when Liberal Democrat leader Nick Clegg left his coaching notes in a taxi. Furthermore, “it didn’t help that (presenter) Alistair Stewart interrupted at every opportunity with all the discipline of an ineffective schoolteacher, costing all three leaders the chance to expand on their points.” Comments on the second debate were little better. In particular, it was viewed by some as three simultaneous party political broadcasts with little true debate and all the answers being delivered straight to camera rather than at the opposition. The third debate will focus on the economy and is approaching fast. There is evidence of the general public becoming more involved in the process. However, this involvement might not be what the leaders were looking for. The public wants to hear details of the leaders’ plans on how to solve the problems left by the recession rather than another evening of hand wringing and vague promises. Another problem lies in how the political parties have approached the debates. They have been seen as possible “banana skins” that could lose an election rather than an opportunity to win it. In some respect, the potential minefield led to what colleague Oliver called the X-Factorisation of the election, i.e. the worst performer gets voted off! Turning to the ever-present opinion polls, how much can we rely on their results? Polls have a somewhat chequered past, in particular, in 1992 when they got the result wrong! A POSTnote issued by the House of Commons before the 1997 election suggested a number of improvements. For time and cost reasons, opinion polls typically comprise between 1,000 and 2,000 interviews. This leads to a statistical margin of error in a party’s support. The suggestion was that results should always be presented with associated uncertainties, for example, 33% ±2% or 31%-35%. Furthermore, some polls are conducted using quota sampling based on the UK voting population. However, it is important that the quotas are set accurately on things that affect voter behaviour. A failure to do this led to the opinion poll debacle in 1992. Two other areas that can affect the reliability of a poll are whether the respondent will actually go on to vote; and what will those who are currently undecided do on Election Day. A further problem lies in the way that the results are then used. Newsreaders frequently report on a party’s share of the vote having falling by 1% when, in fact, this fall lies within the uncertainty in the share of vote figures and could simply have occurred by chance. It also needs to be remembered that opinion polls merely present a snapshot of public opinion at the time the poll was carried out. For example, a typical question is “How would you vote if there were a General Election held tomorrow?” Headlines then claim findings that are not there, e.g. “Poll Predicts Conservative Victory”, when in reality, a poll never predicts anything. Consequently, my suggestion is that we should take polls as an indication rather than as a prediction, i.e. with a large dose of salt!! And finally, I mentioned the “lighter” side of politics earlier, in other words, election silly season! The first story I noticed was when Unilever launched legal action against the British National Party after it used a jar of Marmite in an election broadcast without permission. Then the company which licenses children’s TV character Peppa Pig withdrew her from Labour’s launch of their manifesto for families at a children’s centre. Schools Secretary Ed Balls joked: “Unfortunately Peppa Pig is a global media star, acclaimed around the world, very busy, with many dates in her diary.” A spokesman for Gordon Brown said the prime minister and his family were “big fans” of Peppa Pig. Other ways of measuring voting intentions using food have emerged. At Pizza Express, the Gordon Brown pizza includes fiery chilli and ground beef, while David Cameron is represented with cheese and rocket. Liberal Democrat leader Nick Clegg gets fresh tomato and spicy sausage on his pizza. Ben & Jerry’s have renamed their iconic flavours with a political twist and are selling limited edition versions…Gordon Fudge Brownie, Cameron Chew Chew and Cheesecake Clegg. However, the lighter side prize must go to ex-boxer Terry Marsh who is standing as an independent candidate. He has changed his name by deed poll to “None Of The Above X” (electoral law bans parties from using the name, but not individuals). Onwards to decision day on May 6th!!! Be Vocal—It’s Good Medicine For EverybodyFriday, April 9th, 2010  During a week of laryngitis, Eve Lenkowsky reflects on how frustrating it is to lose your voice—and how market research can be a powerful cure for millions of people worldwide. Wouldn’t you bet my luck that the week the weather turns beautiful and everybody is ready to go outside and shout, that I should lose my voice! Since Saturday, I have been croaking, whispering, and wheezing at anybody who can stand being within earshot of my raspy voice. Luckily, I have people who care for me and who patiently crane an ear to hear what I am saying. But after a while, whether I’m trying to communicate with a loved one or a stranger, I wind up screaming but my words barely come out. Eventually, when you keep on yelling but nobody hears, you give up on trying to get someone to listen. It becomes very frustrating, and sometimes disheartening, when your voice is lost. I think that it is times like these that make me appreciate being a market researcher the most. That’s because I spend most of my time listening to the voices of other people who might otherwise go unheard. Whether it’s a construction worker or a printer, a doctor or a lawyer, business owner or a scientist—these are the people whose voices really have something to say. They are the end-users, the experts, the consumers and people closest to the products and services that our clients provide. They have a vantage point that our clients can only guess at. Sometimes it’s good feedback, sometimes it’s negative—all of it is important. I listen to people’s opinions and requests for improvement in many ways. Sometimes I have the pleasure of speaking with respondents on the phone, either asking them a specific list of questions or having an in-depth discussion to focus on subjects with which they have the most experience. Sometimes, we’ll do focus groups with a bunch of people saying what they think and commenting on each others’ views in a conversation. Other times, I’ll read through comments that hundreds of people type into online surveys when we ask them open-ended questions. Market researchers call these people’s comment quotes ‘verbatims,’ because the person literally tells us his or her point of view—verbatim. Have you ever taken a survey that asks you to answer a question by typing in a comment? Or given some of your time to answer a survey over the phone? Well, rest assured, your voice will be heard! There’s going to be a market researcher out there like me who reads through all of your complaints, compliments, and suggestions, and then communicates your key points directly to the person who has the power to make things better. Market research creates an open dialog that allows consumers to communicate back to the businesses that sell and advertise to them. Consumers are bombarded every day with messages from companies, and market research is one of the key ways that they can speak out and bring about change. Think of it as activism that is actively sought by companies, that benefits everybody. So basically, my job lets me be the voice of thousands of people every year, sharing their opinions with our clients so they can make their products and services better. I can’t ask for anything more—and this week, with this sore throat, I mean that literally! Keep It Simple & Delight Your CustomersFriday, March 26th, 2010
After a week of globetrotting, Nick Hague reflects on his experiences to explain how a good questionnaire can be the first step in achieving satisfied – or, even better, delighted – customers. As I sat at Heathrow Airport on Monday evening waiting for a connecting flight to Athens, I was confronted by a lady wanting to ask me a few quick questions about my experience of T5. My flight had been delayed but the gate was about to been called. However, since it was a ‘short survey’ I said I would help. She asked me the usual questions about shops, eateries, toilets and spaces to sit and relax, but since I wasn’t looking to shop, I had already eaten a sandwich on the previous flight and was on a quick turnaround on my connection, none of the questions were relevant to me. It would have been good if I had chance to freely say that I am always deeply frustrated with the lack of plug sockets near seating areas in airports to allow business travelers to charge their ever-depleting laptop batteries for the forthcoming flights, but this highly structured questionnaire didn’t allow such feedback. I was now being called to the gate and so it won’t surprise you that I rattled my answers off without much thought. I then boarded the plane for the usual humdrum flight experience in cramped surroundings with little space to stretch out, never mind get some work done! However, on this occasion I was wrong to think this way as one of the air stewards must have seen my discomfort and, once airborne, I was offered the chance to move to one of the exit seats where I would have more room. That was very perceptive I thought! It was now a few hours since I had eaten and I was getting hungry. I predicted that the usual dried out meal would be as exciting as ever and would be washed down by the accustomed cheap wine – nonetheless, it would at least curb the onset of my hunger. Then the second thing that I wasn’t expecting happened – I was given a palatable meal but with the usual cheap plonk. I didn’t complain (it isn’t in the English nature) but my face must have spoken a thousand words. The air steward fittingly came back with a very nice Argentinean red – wow I thought! Now I know in-flight entertainment isn’t anything new but I had flown to Athens before and never had a movie, but as I tucked into my meal it came across the speakers that tonight’s movie would be ‘The Blind Side’. I had just read some good reviews on this movie and so sat back and relaxed to watch it. After 3½ hours I arrived in Athens at 1am and realized that even though I was a little bleary eyed, I had in fact enjoyed my flight! The next morning when I vacated my hotel and was waiting for my taxi, I was given a feedback form that asked me about my satisfaction with the courteousness of the staff, the speed of check-in, my bed, the temperature of the room, the amenities (pool, bar etc) and breakfast. I noticed that there wasn’t a question about being kept up all night by the traffic outside my window or for the fact that I didn’t have an iron in my room to iron my creased shirt and trousers for my impending meeting. Like at Heathrow, I had to dash off a response as my taxi was waiting – however, I did point out to the receptionist that there was no scope for me to add comment to the form outside of the tick box questions. That same evening I then boarded my plane home. With vivid memories of my inbound flight I quietly looked forward to my flight back to the UK. However, I should have known better. We boarded with the usual German efficiency (row numbers at a time) and we even set off on time (unlike my flight the previous day), but yes, you guessed it. I had a cramped seat, the meal was a dried sandwich, the wine was like vinegar and there was no movie! Since there was no movie, it did at least give me time to pen my looming Thursday Night Insight piece. Thinking back over the last couple of days’ events, it hit home to me what the difference was between customer satisfaction, delighting customers and customer loyalty. Was I satisfied with my flight back from Athens; yes, it got me back without crashing and on time. However, was I delighted; no, and if I ever have a choice again I will definitely choose a different airline. My reflective time also allowed me to think back to the surveys I took part in at the airport and the hotel and emphasized the importance of not only asking the right questions but also picking the right time so respondents are in the right frame of mind to answer them properly. The market research industry is probably its own worst enemy at times and latest thoughts often swing from one extreme to another. Either questionnaires are designed so detailed to get to the heart of what customers think that you run the risk of not surveying the busy but important customer; or with the desire for simplicity becoming worldwide, does a CSI (Customer Satisfaction Index) or NPS (Net Promoter Score) oversimplify things and not actually mean anything other than a number used for internal benchmarking? All researchers like their lists and so here are my 5 top tips from this week’s experience to take into account when designing a customer satisfaction or customer loyalty project:
To conclude, in designing customer satisfaction and loyalty surveys I think we should all remind ourselves of the words of Albert Einstein “Things should be made as simple as possible, but no simpler” – but remember; it can be the simple things in life that can actually delight your customer. Find out more about some of the subjects touched in today’s Thursday Night Insight. Read our:
« Previous Entries |
|
