Surveys are the great, misunderstood research method and there are many reasons for this, as surveys are often:
- Poorly done
- Poorly timed
- Right tool/wrong time
- Wrong tool for the job
However, the most important reason why surveys often fail is that surveys are deceptively simple to create and thus can be thrown together with just a moment’s notice and just as little thought. Got a problem? Do a survey. Users or customers unhappy about something? Do a survey. Don’t have time/money/resources for other methods of user research or testing? Do a survey! With so many great, free, online, survey tools available in the time it takes you to read this article you could have created and launched a survey. That’s both the upside and the downside of using surveys as a research tool.
Garbage In/Garbage Out
The only way to get significant, usable, data from the surveys you conduct is to start with an excellent survey. As the old saying goes: garbage in/garbage out. So, follow these basic rules of survey design and the data you get back in return will be data you can use:
- Define an objective: If you are going to conduct a survey, you must first have an objective. Don’t just aim for a “quick pulse check” and hope you will uncover something interesting along the way. Before you start building a survey have a stated problem you are trying to solve, and a reason why you are using this particular tool to address that specific problem. It would help if you defined the problem so that you will know what to do with the data you collect.
Two things must be crystal clear before you begin to design a survey:
- What do you want to learn?
- Who is the target audience?
- Make sure your goal is to collect opinion-based data: Surveys can only uncover attitudinal data—opinions—what people think or how they feel. You cannot discover how people will or do use a product or service with a survey tool. People are notoriously poor reporters of what they will or will not do in the context of an action. So if you are trying to identify failings in the UI or functionality of a product or service, a survey is not the right tool for the job. Nielsen Norman Group’s UX Cheat Sheet can help you determine which research method is best for the problem you are trying to solve.
- Keep it simple: Only ask what you need to ask. There is no ideal number of questions or ideal survey length, but the more questions you ask, the less time respondents will likely spend on each question so keep your survey tightly focused.
- Write great questions: There are many guidelines available online to help you write great survey questions. These guides will help you choose the right type of question, help you create a great mix of questions and help you understand how the questions you ask affect the answers you receive. When you start to craft your survey make sure you:
- Ask direct questions: This is not a context where flowery language will help you. Be direct.
- Avoid leading questions: Make sure the way you word the questions doesn’t inadvertently tip the scales to how you would like it answered.
- Understand the difference open-ended and closed-ended questions: Choose carefully, keeping in mind what type of data each type will yield.
- Test before you launch: Don’t throw something at the wall and hope it sticks. Create the survey, then test it and refine it before you send it out into the world.
Here are some other resources to help you create a great survey:
- Writing Usable Survey Questions: 10 Things All UX Researchers Should Know
- The Best Online Survey Tools of 2018
- When is a Sample Size Statistically Significant?
You can cut corners, throw something together, “quick and dirty” to just get that “pulse check” everyone is always asking about, or you roll up your sleeves and do the work that needs to be done to create a survey that will yield data you can use.
The choice is yours, but remember: good survey design leads to good survey data.