Chapter Nine Questionnaire design
Introduction
In research investigations, information can be collected through the application of a variety of techniques such as interviews, questionnaires, observation, direct physical measurement and the use of standardized tests. This chapter focuses on questionnaires and questionnaire design, since they are frequently used to collect data in health sciences research.
Questionnaire construction
A questionnaire is a document designed with the purpose of seeking specific information from the respondents. Questionnaires are best used with literate people. The design of the questionnaire is crucial to its success. The process of design and implementation is usually termed questionnaire construction.
Questionnaire construction usually involves the following steps:
As with all research, the ethics of conducting surveys and designing questionnaires must be considered. For example, respondents should not be misled concerning the aims of a survey. A blatant example of unethical conduct is if one is asked to respond to a general ‘market survey’ and then finds a high-pressure salesperson on the doorstep. If the survey is said to be anonymous, then it is questionable practice by the investigator to code the forms secretly. The follow-up of non-responders can cause a dilemma; people choosing not to participate in a survey should not be pestered. However, forms are sometimes mislaid or forgotten and it is necessary to follow these up to ensure that a representative sample is obtained.
In clinical research, the ethical issues relating to the possible effect of the contents of the questionnaire on the respondent must be taken into account. As an example, one of the present authors was once involved in a survey aimed at establishing levels of knowledge of Huntington’s disease and certain attitudes of people at risk for the condition (Teltscher & Polgar 1981). Before the survey was undertaken, a pilot study was carried out to establish whether or not the questions were upsetting to the subjects. The actual subjects were randomly selected from a ‘pedigree chart’. However, the questionnaires could not be sent out before it was clearly established that each of the prospective subjects already knew that they were at risk of developing the condition. It would have been appalling if people learned from receiving this questionnaire that they were at risk of a severe genetic disorder.
Questions and questionnaire formats
Questions and questionnaires come in a variety of formats. The researcher must decide which format is the most appropriate for the purpose of the study. Let us first consider the issue of the questionnaire format.
In some instances, researchers will not prepare a formal questionnaire to be filled in by the respondent, but will design an interview schedule to guide the interviewer who asks the questions. (Interviews are discussed in more detail in Ch. 10.) There are costs and benefits in both approaches, as shown in Table 9.1.
Table 9.1 Costs and benefits of interviews and questionnaires
| Costs | Benefits | |
|---|---|---|
| Interview schedule administered by interviewer | ||
| Self-administered questionnaire |
The interview schedule approach requires expert interviewers to administer the questions and this is expensive and time-consuming. Further, interviewer bias has been shown to influence responses, as some respondents may modify their responses to fit in with what they perceive to be the opinions of the interviewer. It is important to note that the structure and content of a questionnaire convey a lot of information about the researcher’s agenda to the respondent. The respondent generally has little opportunity to influence the agenda. A questionnaire is not really a conversation or dialogue but is essentially a monologue from the researcher to which the potential respondent may or may not respond.
However, the self-administered questionnaire approach is cheap, is less susceptible to interviewer bias and can be administered by post. The disadvantages of this approach include higher rejection or refusal rates and much less control over how the response forms are filled out. Anyone involved in self-administered questionnaire analysis will attest to the sometimes remarkable talents of respondents in returning incomplete questionnaire response forms.
Having decided whether the questionnaire will be self-administered or administered by interviewers, the researcher must then decide upon the format for the individual questions.
Open-ended and closed-response formats
There are two major question formats, the open-ended and closed-response types. The distinction between the two is best illustrated by example (see Table 9.2).
Table 9.2 Open-ended and closed-response formats
| Open-ended | Q1. How do you feel about the standard of the treatment you received while you were a patient at this hospital? | |
| Closed-response | Q2. How would you rate the standard of the treatment you received while you were a patient at this hospital? (circle one number) | |
| excellent | 1 | |
| good | 2 | |
| moderately good | 3 | |
| fair | 4 | |
| poor | 5 | |
In Table 9.2 the first question is an open-ended question whereas the second question is a closed-response question. In an open-ended question, there is no predetermined response schedule into which the respondent must fit a response. In a closed-response question, the respondent is supplied with a predetermined list of response options. The advantages and disadvantages of both question types are represented in Table 9.3.
Table 9.3 Costs and benefits of open-ended and closed-response formats
| Costs | Benefits | |
|---|---|---|
| Open-ended | More detailed answers elicited | |
| Closed-response |
Although open-ended questions elicit more detailed responses, there are some possible disadvantages associated with this type of question. The responses generated by such questions require a large amount of effort to encode for data analysis and, when they are coded, tend to give rise to categorical scales. These scales necessitate the use of less powerful statistical methods. Further, some respondents may take a long time to answer this type of question.
Of course, to a researcher employing a qualitative orientation, these ‘disadvantages’ may not be seen as such. The opportunity to study respondents’ interpretations expressed in their own words might lead a qualitative researcher to advocate extensive use of open-ended questions. It is more likely that interview techniques rather than a written questionnaire would be employed. Questionnaires, particularly of the self-administered variety, are generally used for convenience and speed, not for depth of analysis.
It is important that the lists of options for closed-response questions are carefully designed by the questionnaire designer. It is very easy to bias responses by restricting the range of answers in this type of question.
This brings to mind a short questionnaire distributed by an insurance company to one of the authors (see Table 9.4). Two features are remarkable about this question. First, it does not allow for any answers other than the ones listed. Second, the range of available answers is very limited. What we wanted to say was that the insurers were cheap and reliable, i.e. were likely to pay up in the event of a claim. Clearly, the survey designer was a marketing person who had not satisfactorily trialled the questions with a non-marketing audience. While the designers may well have obtained the answers they wanted, the answers may not have been the ones the respondents wished to give. In the health sciences, there are often large differences in the ways in which health professionals and consumers approach the same problems. In questionnaire design, it is vital that the researchers do not impose their own conceptualizations of the situations under investigation to the extent that validity is compromised.
| Why did you choose XYZ Insurance to insure your car? |
A further example of this danger is provided by the results of a survey conducted at a major teaching children’s hospital in Australia (Thomas et al 1989). The survey was designed to study why many parents chose to stay with their children in the hospital, in order to plan better facilities and services. One of the questions asked of the respondents was ‘Why did your child come to hospital today?’ The investigators deliberately chose an open-ended response format for this question in order to tap into the parents’ interpretations of the situation in their own words. What a treasure trove of answers! The answers provided considerable insight into the issues of importance to the parents, most of which we could not have predicted. Thus, the ways in which the questions are asked and the answers sought can have a major impact on the value of the information collected.
Likert and forced-choice response formats
In attitudinal questions, two possible response formats may be chosen: the traditional five- or seven-point Likert-type format, or the four-point forced-choice format. These are best illustrated by examples, as shown in Table 9.5. The first example is a conventional five-point Likert-type scale. The second is a four-point forced-choice type. The advantages and disadvantages are summarized in Table 9.6.
Table 9.5 Likert and forced-choice response formats
| Q7. My medical practitioner always explains the chosen treatment to me (circle one number). | |
| strongly agree | 1 |
| agree | 2 |
| undecided | 3 |
| disagree | 4 |
| strongly disagree | 5 |
| Q8. My medical practitioner always explains the chosen treatment to me (circle one number). | |
| strongly agree | 1 |
| agree | 2 |
| disagree | 3 |
| strongly disagree | 4 |
Table 9.6 Advantages and disadvantages of response formats
| Response format | Advantages | Disadvantages |
|---|---|---|
| Likert-type | Allows middle ‘undecided’ response | Acquiescent response mode |
| Forced-choice | Respondent forced to give either a positive or a negative response | ‘Undecided’ response not allowed |
The forced-choice format does not allow respondents to give a ‘middle of the road’ or undecided answer. This is to guard against respondents using an acquiescent response mode. Acquiescent response mode refers to the phenomenon that occurs when respondents give middle responses all the time. Extreme response mode occurs when a respondent never selects an intermediate point on the rating scale.
The wording and design of questions
The writing of good questions is an art, and a time-consuming art at that. In order to obtain valid and reliable responses one needs well-worded questions. There are a number of pitfalls to be avoided:
Finally, it should be kept in mind that even a good questionnaire might be invalidly completed. For example, a survey on ‘attitudes to migration’ might be answered less than honestly by respondents if the interviewer is obviously of immigrant background.
The overall structure of questionnaires
Questionnaires may be structured in different ways, but typically the following components are included.
It is best to avoid complicated structures involving, for example, many conditional questions such as ‘If you answered yes to Question 6 and no to Question 9, please answer Question 10’. Conditional questions usually confuse respondents and ought be avoided where possible.
Summary
Questionnaires are frequently used for data collection in health sciences research. This chapter has reviewed the principles of questionnaire design, including issues arising from the selection of appropriate questions and response formats.
Self-assessment
Explain the meaning of the following terms: