Archive for the ‘Questionnaire Design’ Category
Friday, March 26th, 2010
After a week of globetrotting, Nick Hague reflects on his experiences to explain how a good questionnaire can be the first step in achieving satisfied – or, even better, delighted – customers.
As I sat at Heathrow Airport on Monday evening waiting for a connecting flight to Athens, I was confronted by a lady wanting to ask me a few quick questions about my experience of T5. My flight had been delayed but the gate was about to been called. However, since it was a ‘short survey’ I said I would help. She asked me the usual questions about shops, eateries, toilets and spaces to sit and relax, but since I wasn’t looking to shop, I had already eaten a sandwich on the previous flight and was on a quick turnaround on my connection, none of the questions were relevant to me. It would have been good if I had chance to freely say that I am always deeply frustrated with the lack of plug sockets near seating areas in airports to allow business travelers to charge their ever-depleting laptop batteries for the forthcoming flights, but this highly structured questionnaire didn’t allow such feedback. I was now being called to the gate and so it won’t surprise you that I rattled my answers off without much thought.
I then boarded the plane for the usual humdrum flight experience in cramped surroundings with little space to stretch out, never mind get some work done! However, on this occasion I was wrong to think this way as one of the air stewards must have seen my discomfort and, once airborne, I was offered the chance to move to one of the exit seats where I would have more room. That was very perceptive I thought! It was now a few hours since I had eaten and I was getting hungry. I predicted that the usual dried out meal would be as exciting as ever and would be washed down by the accustomed cheap wine – nonetheless, it would at least curb the onset of my hunger. Then the second thing that I wasn’t expecting happened – I was given a palatable meal but with the usual cheap plonk. I didn’t complain (it isn’t in the English nature) but my face must have spoken a thousand words. The air steward fittingly came back with a very nice Argentinean red – wow I thought! Now I know in-flight entertainment isn’t anything new but I had flown to Athens before and never had a movie, but as I tucked into my meal it came across the speakers that tonight’s movie would be ‘The Blind Side’. I had just read some good reviews on this movie and so sat back and relaxed to watch it. After 3½ hours I arrived in Athens at 1am and realized that even though I was a little bleary eyed, I had in fact enjoyed my flight!
The next morning when I vacated my hotel and was waiting for my taxi, I was given a feedback form that asked me about my satisfaction with the courteousness of the staff, the speed of check-in, my bed, the temperature of the room, the amenities (pool, bar etc) and breakfast. I noticed that there wasn’t a question about being kept up all night by the traffic outside my window or for the fact that I didn’t have an iron in my room to iron my creased shirt and trousers for my impending meeting. Like at Heathrow, I had to dash off a response as my taxi was waiting – however, I did point out to the receptionist that there was no scope for me to add comment to the form outside of the tick box questions.
That same evening I then boarded my plane home. With vivid memories of my inbound flight I quietly looked forward to my flight back to the UK. However, I should have known better. We boarded with the usual German efficiency (row numbers at a time) and we even set off on time (unlike my flight the previous day), but yes, you guessed it. I had a cramped seat, the meal was a dried sandwich, the wine was like vinegar and there was no movie!
Since there was no movie, it did at least give me time to pen my looming Thursday Night Insight piece. Thinking back over the last couple of days’ events, it hit home to me what the difference was between customer satisfaction, delighting customers and customer loyalty. Was I satisfied with my flight back from Athens; yes, it got me back without crashing and on time. However, was I delighted; no, and if I ever have a choice again I will definitely choose a different airline. My reflective time also allowed me to think back to the surveys I took part in at the airport and the hotel and emphasized the importance of not only asking the right questions but also picking the right time so respondents are in the right frame of mind to answer them properly.
The market research industry is probably its own worst enemy at times and latest thoughts often swing from one extreme to another. Either questionnaires are designed so detailed to get to the heart of what customers think that you run the risk of not surveying the busy but important customer; or with the desire for simplicity becoming worldwide, does a CSI (Customer Satisfaction Index) or NPS (Net Promoter Score) oversimplify things and not actually mean anything other than a number used for internal benchmarking?
All researchers like their lists and so here are my 5 top tips from this week’s experience to take into account when designing a customer satisfaction or customer loyalty project:
- Tip number 1: Make sure you allow scope to get feedback on what really matters. Therefore always try and build in a qualitative stage upfront so you can design your quantitative questionnaire with confidence. If budget and timings don’t allow for a qualitative stage, make sure you allow for scope within the questionnaire for open-ended answers so respondents aren’t infuriated in not being able to give their fullest answers.
- Tip number 2: Don’t use fancy vocabulary or at least use layman’s language (qualitative research can also help to validate how customers think and speak in this area). By way of example, the questionnaire from my hotel in Athens asked about housekeeping. Should they not have asked about cleanliness and comfort in the room to make it more understandable to a wider audience?
- Tip number 3: Don’t just look to understand what satisfies customers. Design your questionnaire to look at what delights customers as this will drive loyalty and therefore drive upwards profitability.
- Tip number 4: Use a research design that allows for a range of customers to be interviewed; not just the extremes of customers who are either highly satisfied or have an axe to grind, and make sure you don’t just analyse stated answers. Using statistical tools can help you infer what is really important to customers and therefore driving customer satisfaction.
- Tip number 5: Make sure the survey design is fitting to the marketplace you are looking to get information from. If you only have a couple of minutes, ask the really important questions only. If the respondent will have more time, design a wider ranging survey that looks to get to the crux of the matter (and make sure it is at a convenient time to collect the most thorough information you are looking for).
To conclude, in designing customer satisfaction and loyalty surveys I think we should all remind ourselves of the words of Albert Einstein “Things should be made as simple as possible, but no simpler” – but remember; it can be the simple things in life that can actually delight your customer.
Find out more about some of the subjects touched in today’s Thursday Night Insight. Read our:
Posted in
Customer Insight, Interviews, Market Research, Nick Hague, Questionnaire Design, Service, Surveys, Thursday Night Insight, White Papers |
Comments:
None Yet »
Friday, May 8th, 2009
David Ward this week uses his data processing expertise to show us how we can spot and weed out ‘rogue respondents’ to get the most reliable and valuable data from our online surveys.
The internet is a very useful tool in market research. We can reach a much wider and larger audience than using traditional pen and paper or CATI interviewing (spam filters allowing). We can make the interviewing experience more visual, route respondents to the questions relevant to them, view results in real time before the tabulation processes have begun, program logic checks in to the survey to catch errors before the analysis stage…the list of positives goes on. Of course there are, as with any interview method, pros and cons to choosing a particular method of interviewing.
On our travels on the internet we’re never too far away from someone wanting to collect our opinions about this and that. For example, when I was looking online for a new car recently, nearly every website I went to had some sort of pop-up window asking me to take part in a survey and, more often than not, some incentive was provided to entice me to spend my time completing the survey. Personally I’ve never been tempted to complete one of these surveys appearing in pop-up windows, but that’s just me. However, what is to stop someone seeing the incentive, thinking ‘I’d quite like a new iPod’, and just randomly clicking through the survey in double-quick time with no thought to their responses? For the respondent, the incentive is there to get the survey completed and have a chance of winning the prize, but not necessarily to give each question the thought required. There isn’t a lot we can do to stop this happening; however, there are things we can do after the data is collected to spot these rogue respondents.
The scope for logic checks on online surveys would be vast if you took it to the nth degree. We might lose a large number of respondents if we did do this though – through frustration at constantly having their responses questioned – but there is a balance to be found. As Head of Data Processing at B2B International, what steps can I take to ensure the quality of our data? There are no guarantees but we can take steps when setting up online surveys and reviewing the data to highlight suspect records. As I have said, we can program logic checks into our online surveys to make sure the survey is filled in correctly. We can make sure numbers add up to 100% where needed, or that respondents select the correct number of items from a list.
One of the easiest ways to catch potentially bad records in the data is to quickly check the time stamp for each interview. Did someone manage to complete a 20-minute interview in 5 minutes? This would suggest someone has just clicked through the survey with a click happy mouse finger, without giving due consideration to their answers.
Another telltale sign to look out for is something termed as “straightlining”. Has a respondent gone through a grid or battery of questions and given the same response each time?
We can also look for inconsistencies in the logic of the answers and for unlikely values in numeric questions. Part of this can be done during the survey and more checks can be run once the data have been collected.
We could also add in questions purely for verification purposes to allow us to judge whether or not the respondent is reading the question. For example, in a grid of questions we could add one that simply says “for verification purposes please answer strongly agree”. Along the same lines as this we could use data from any panels we have purchased to ask respondents to verify certain details. Comparisons can then be made between the original panel data and our own data collected online. Differences between the two could be viewed as suspect.
Finally, we can look at responses to open questions. We can check that fields do not contain random characters, or single character answers. If we find this, can we be sure we can trust the other answers given?
Failing one of these checks does not necessarily mean the data is not to be trusted but failing two or more may be grounds for removing that respondent. Perhaps the strongest guide to base the decision on is the time taken to complete a survey, but whichever method or combination of methods are used, having the checks in place gives us added confidence in the findings we present to our clients.
I’m not sure we will ever be able to completely stop respondents just clicking through an online survey giving responses that are illogical, poor quality and clearly not much use to market research, but knowing that there are telltale signs we can look out for which can indicate a respondent we may need to exclude is certainly reassuring.
Posted in
Data Quality, David Ward, Esurveys, Questionnaire Design, Thursday Night Insight |
Comments:
None Yet »
Wednesday, August 27th, 2008
Today we’re publishing the eleventh (and final!) chapter from our eBook – Questionnaire Design, by Paul Hague.
This week’s closing segments looks at some "Examples Of Questions", which provide illustration of all the principles outlined in the rest of the eBook. These questions can also act as a useful template for constructing different types of questionnaire and include:
- Questions for starting a questionnaire
- Questions for testing awareness
- Questions for gleaning behavioural information
- Questions for obtaining attitudinal information
- Questions looking at buying motivations
- Customer preference and satisfaction questions
- Price testing questions
- Classification questions
As always, this week’s chunk of the eBook is available both as a pdf and as a podcast. To download each, just click on the appropriate links below.
That’s all for now, but be sure to check back very soon, as we’ve plenty more podcast (and eBook) goodness lined up. Watch this space…
Posted in
Paul Hague, Questionnaire Design, Research Design, eBook |
Comments:
None Yet »
Monday, August 11th, 2008
This Monday sees the publication of the tenth, and penultimate chapter of our latest eBook – Questionnaire Design, by Paul Hague.
This week’s installment deals with the topic of "Special Questionnaires", by which we mean those questionnaires that depart from the usual telephone- or face-to-face- administered, structured survey.
The first type of special questionnaire to be explored are "check list" style surveys, used predominantly in qualitative research as a means of roughly structuring or guiding a conversation with the respondent. The second main "other" type of survey is the self-completion questionnaire, whether conducted by post or online. Such surveys work very differently to those guided by an interviewer, and as such, the questionnaire designer must tailor the questionnaire accordingly to achieve the maximum quality of response.
As always, this week’s chunk of the eBook is available both as a pdf and as a podcast. To download each, just click on the appropriate links below.
And last but not least, don’t forget that next week we’re publishing the final chapter of the eBook – be sure not to miss it!
Posted in
Online Research, Paul Hague, Qualitative Research, Questionnaire Design, Research Design, eBook |
Comments:
None Yet »
Monday, August 4th, 2008
It’s a Monday, which must mean that it’s time for another chapter from our latest eBook Questionnaire Design.
This week’s eBook chapter (with accompanying Podcast, courtesy of the Market Research Podcast), looks at "Getting The Questionnaire Right", both in terms of adequately testing the survey through piloting, as well as investigating how software can assist in putting the questionnaire together.
Don’t forget to come back next Monday when we’ll be releasing Chapter 10 for your reading (and listening) pleasure!
Posted in
Paul Hague, Questionnaire Design, Research Design, eBook |
Comments:
None Yet »
|
