With survey software platforms available free of charge and easy to use, it can be enticing to run your own survey- perhaps amongst your stakeholders, clients or staff members. However, it takes someone with specialised skills to design a survey well, to ensure you gather information that you need and you can use. If you do decide to design your own survey and you are not an expert in the field, here are some common errors to avoid.
Mistake #1. You haven’t decided how you will use the information
It is a common error to think it is a good idea to ask everyone everything. It won’t cost you anything, and since you have the attention of your respondents, why not ask them everything you have always wanted to know? To do this, respondents are likely to feel drained and exploited. Additionally, you will not know what do with the results. When you think your survey is ready, read back over every question. If you do not know what you intend to do with the findings to each and every question, drop those questions from the survey.
Mistake #2. You ask two questions in one
Two questions can be asked as one because a survey designer conflates two ideas they do not see as mutually exclusive. If two ideas are asked in one question, when you look at the results it will be impossible to know which aspect of the question respondents are referring to. An example of two questions asked in one: “How satisfied are you with the speed and accuracy of our service?” Speed and accuracy are different concepts; some respondents may be satisfied with the speed, others just with accuracy. So, instead, two questions are required: one about speed, the other dealing with accuracy.
Mistake #3. You use leading questions
Based on their structure, questions can be leading if respondents are likely to feel they are being guided to a particular response. It may be unintentional and can come about from you being too close to the subject matter of your survey. For example: “How important do you think it is for funding to be provided to continue our very successful arts and cultural program for another term?” The issue here is that respondents are being subtly manipulated into feeling a positive disposition towards the arts and cultural program and may feel unable to dispute the argument for its ongoing funding. It is important to remain neutral and avoid using persuasive language.
Mistake #4. Your grammar is inconsistent
It is important to take great care in the wording of your questions that your grammar is consistent to give respondents the best chance of understanding exactly what you are asking. If you ask a question using a particular adjective, it is important to use the same adjective in the responses. For example, this question has inconsistent grammar: “How happy were you with the follow-up support provided by our service?” [response options range from very dissatisfied to very satisfied]. Happiness and satisfaction are subtly different concepts. Be sure to use the same adjective in your response options as in your question so that your question is reinforced as respondents consider the answer options.
Mistake #5: Leading with demographics
It is not uncommon to see surveys begin with demographic questions (age, sex, location, education etc). You may think it is better to start with easy questions to get respondents into the flow of the survey without having to think too much. Respondents can often find these questions to be intrusive, and even unnecessary; they are not always comfortable answering them on a whim at the beginning of a survey. It is better to wait until you have earned their trust, demonstrated the legitimacy of the survey before asking personal questions. Additionally, if respondents drop out of the survey, you hope they have at least answered the most important questions. Demographic should always come at the end of a survey.
Mistake #6. Giving respondents too many choices
We have all experienced shopping at a supermarket, perhaps looking for our favourite item such as plain tuna, only to be bombarded with a myriad of choices offering variations and embellishments. Tuna with chili, tuna with pepper, tuna with lemon, pepper and chili. Too much choice can be overwhelming and confusing. Providing respondents with a variety of choices for a particularly question is a good idea, but it is important to do so within reason. If there are going to be too many possible answers, it is a good idea to anticipate the most likely ones in a list, then add ‘Other, please specify’.
Mistake #7. Too many open ended questions
Open ended questions can provide a valuable opportunity to gather free-flowing responses from respondents. When a topic is complex, open ended questions provide freedom in expression and a chance to explain complexities. But too many open ended questions in a survey can be a burden. Additionally, open ended questions take considerably more time and effort to analyse. If you can anticipate likely responses to a question, add them as closed options. Use open ended questions sparingly and only when they add value such as depth of meaning.
Conclusion
It can be tempting to take on your own survey design, particularly when using free online survey software. But the quality of your findings are dependent on the standard of your survey design. If your survey is important enough to do in the first place, get expert help. If you must embark on it yourself, try to avoid these common mistakes to give yourself a good start.