Article: Good & Bad In Questionnaire Design
How often have you struggled to respond to a questionnaire which just isn't quite right for your answers? No doubt you became annoyed, frustrated and maybe even cynical about market research. These problems arise because of poor questionnaire design.
The most frequent errors in questionnaires are:
These errors can be eliminated or at least minimised if the researcher keeps two things clearly in mind: who will be answering the questions and what the research is aiming to achieve.
The researcher must understand four fundamentals of questionnaire design:
What Types Of Question?
The researcher's tool box contains three different types of question: behavioural, attitudinal and classificatory.
Answers to behavioural questions tell the researcher where a brand is now and where it could be in the future. They show who is doing the buying and the pattern of purchasing amongst companies using the product. They allow the researcher to position a company or brand in terms of its market share. Here are some examples of the subjects which behavioural questions can address:
Attitudinal questions establish respondents' opinions and their image or perception of a subject. Their perceptions may, of course, be totally at variance with reality, but that may make them all the more valuable as marketing data. Attitudinal research can help the researcher plan a campaign to win sales by:
The third type of question in the researcher’s armoury is that which classifies the respondent. Such questions allow answers from different companies to be grouped and compared. Classification questions enable the researcher to segment the market and decide where marketing effort will be most rewarded. Common classifications used in industrial market research are:
There may well be classifications which are special to a particular survey. Most industrial surveys, for example, classify according to the size of the respondent company’s purchases, so that the researcher can see the difference in attitudes or purchasing patterns between large and small companies.
What Types Of Response Should Be Built In?
Having selected the type of question (attitudinal, behavioural or classificatory), the researcher must decide on the response option. The questions can be either open ended or closed.
Open ended questions leave the respondent free to give any answer, and verbatim responses are recorded. This type of question and answer is extremely flexible, as the researcher has not committed to a list of pre-coded responses.
However, open ended questions are sometimes an easy option by researchers too lazy to think of possible pre-codes. Open enders should only be asked where the response is truly unknown.
A closed question is one that has pre-coded answers. The simplest is the dichotomous question to which the respondent must answer yes or no.
Closed response questions save the respondent having to think of possible replies. They also make life easier for the interviewer who simply has to tick a box or circle a number. Moreover, they spare the coding staff difficult judgements which, if wrong, can skew the findings.
A special type of pre-coded question is a scale. Scales can be of different types.
Verbal rating scales. Typical of these is the five point scale running from very good through to very poor. Verbal rating scales can be applied to almost any adjective - good/poor, happy/unhappy, satisfied/ dissatisfied, pleased/displeased, suitable/unsuitable, and so on.
Instead of words, the researcher can use a numerical weighting scale. This usually runs up to five or 10. The respondent needs to know the direction of the scale, viz:
Problems can occur if the researcher wants a rating on too many questions. Answers deteriorate as the questions grind on and the respondent repeats the same score in an effort to speed the completion of the interview.
Diagrammatic rating scales. These, as the name suggests, ask the respondent to mark an answer on some continuum. This could be a line, a balance, or boxes. Diagrammatic scales are self completed by the respondent and so they need to be clearly marked at each end of the scale. They can be more fun and faster to complete than the verbal or numerical scale.
How To Lay Out The Questionnaire
The questions must now be organised logically and attractively. This is for the benefit of the interviewers, the respondents and the coders. The layout depends very much on who will be administering the questionnaire, who will be answering it, and the type of questions which are being asked.
A well laid out questionnaire must have a sensible sequence of questions. The smooth flow of the interview depends largely on the order in which questions are asked.
Questions on image, purchasing behaviour and classification should all be arranged together. They should lead from one to another with the general subjects at the beginning and the more particular later on. This allows the respondent to relax and loosen up.
For the same reasons, easy questions should precede the difficult ones. Sensitive questions should be placed towards the end, so that the respondent is "warmed up" and in autonomic response mode.
The questions and responses should be clear and easy to read. If the boxes to tick (or numbers to circle) are out of line or placed too far from the pre-coded responses, the interviewer could mis-cue an answer. There should be ample space for the interviewer to record verbatim responses.
A clear and attractive layout is even more important in a self-completion questionnaire, where it has a significant effect on the response rate.
How To Test The Questionnaire
Questionnaires may have to be designed with only a modicum of knowledge about the subject. In these circumstances mistakes are bound to occur unless the questionnaire is tested. The test should be carried out using the interviewing medium - ie telephone, face-to-face, postal – for which the questionnaire has been designed. The number of interviews required to test a questionnaire could be as few as five to 10, though the more the better. Cost and time are invariably the constraints. In industrial market research, the project may use semi-structured questionnaires with a very small number of interviews. Piloting here is a continuous process, and modifications are made from the moment the study gets under way.