Idea Validation Field Guide

Exercise 4.
Create a survey to test hypotheses

When you sit down and talk to potential users, you’re in the perfect setting to learn about the problem you’re tackling with your idea. Interviews are especially well-suited to answer questions around ‘why something happens’ and ‘how something happens.’ In other words, it’s a qualitative research method.

But if you’re working on a complex problem, in a complex vertical (like healthcare, or financial, or education, or even a habit-building product), and your budget is limited for your MVP, you may need to dig deeper to understand which pain points you should prioritise. This is where a quantitative research method, like a survey, may come in handy.

Surveys aren’t the fix-it-all for everything

Surveys aren’t a magic validation tool where just because some questions give you answers with numbers, those suddenly become meaningful and ‘objective.’ In fact, it’s harder to build a good survey than a bad one, and the only way to verify if you’ve done a good job is by checking your results against a secondary source of research. Or you can build something on bad survey insights and it will flop.

What you need:

What to do:

Decide on 1-3 hypotheses to test from the interview insights you’ve gathered.

Screener questions make sure the people responding to your survey are part of the audience of your future product. If one of your survey goals is to segment your audience into buckets to identify which to target first, a series of screener questions are doubly important:

• To screen out any people who don’t belong to your audience in the first place.
• To place the responder into their respective audience bucket

What you’ll want to cover in screener questions is usually sociodemographic information, like where someone works, their education, income, marital status, age, location, gender, and any other information of the type that might be relevant for you. Don’t ask for this type of personal information if it’s not relevant.

To write down the questions for a survey, start with the answers you want to get. You already have a start on the answers you want to get, from the hypotheses you set out to test.

So, let’s start out from one of those as an example of the process.

“People managing small and medium businesses struggle more with managing finances than people who are freelance in a regular month.”

There are several variables here, so let’s break it down a little. First, there are the screener questions for the job position:

Then there’s what we’re actually trying to measure ‘struggle with managing finances’. To see what that actually means, we’d refer to the interview insights. As we’re writing the questions of a survey, we’re focusing on measuring frequency and severity, so let’s say struggle = time spent + money spent. And because we need two answers, we’ll ask about two separate questions:

That’s how the process goes for all the questions of your survey. It’s important that you try to focus on the least amount of questions possible, and not include questions in the survey about things you might be able to find out from different sources.

Have the specialist review your research hypotheses and survey to make sure you’ll gather the right insights. For more insights on writing good survey questions, check out the dedicated section.

You can use a free tool like Google Forms, SurveyMonkey or Typeform.
If you’re unsure how you’ll gather responses from your audience organically, you can also set up your survey on a platform like Pollfish, where you can pay to gather responses from a preselected audience. 

If your product has more than one core audience, you’ll need a tailored survey for each of them, and more than 120 responses from each type.
Have the specialist get insights out of the answers you gathered.

While you work

Good rules of thumb when writing survey questions

1. Go from general questions towards specific questions.
2. When writing the questions and answer options, use simple and to-the-point language that is familiar to the audience. Referring back or simplifying expressions you’ve heard in the interviews is not a bad thing.
3. Ask about one thing at a time. Don’t combine two or more questions into one, because the responder won’t know what to answer, or will only answer about one and you won’t be able to interpret the data.
4. People like to be liked. If you load your questions or response options with assumptions or preferred answers, they’ll answer that they perceive you want to hear.
5. Humans are very good at being… well,  human. That means we’re bad at recalling the distant past, and whatever we say about our future intentions should be taken with a grain of salt. If you want reliable predictors for future behaviours, ask if the behaviour has happened in the recent past.
6. Closed questions are easier to answer – especially if you offer the right number of options. Balanced to cover both extremes, and enough to cover all likely answers without being overwhelming.
7. If you’re writing a multiple-choice question, make sure the answers are mutually exclusive and that they cover all possible categories. If you’re unsure about that, an ‘other’, unspecific category will save the day.

Need help with
your survey?

Email us