Think!

The hardest question is the one you don’t ask

“I didn’t have time to write a short letter, so I wrote a long one instead.” Often attributed to Mark Twain.

It’s one of those quotes that’s stuck with me because it’s true. Writing less is harder than writing more. So is asking fewer questions.

In social research, there’s always a temptation to squeeze in one more question. If we’re already talking to people, why not ask about a few more attitudes? Or future intentions? We might need it later.

One extra question rarely feels like much. But it’s rarely just one extra question.

Interestingly, the clients who get the most from research are usually the ones asking us to simplify. They’re clear about the decisions they need to make and disciplined enough to let go of the “nice to know” in favour of the “need to know”.

That’s because research isn’t really limited by how much we can ask. It’s limited by how much people can reasonably process. People complete surveys in the middle of busy lives. Between meetings. On their phones. While juggling a dozen other things. They don’t arrive fresh, focused and excited to answer 40 carefully crafted questions.

As surveys get longer, or questions become harder to read, people naturally put in less effort. They skim. They choose the first answer that seems reasonable. They stop reading every option. Some leave altogether, but many simply keep going on autopilot.

That’s the real risk.

A long survey doesn’t just increase drop out. It increases low quality responses. And low-quality data can be more dangerous than no data at all because it gives us confidence in conclusions that aren’t as reliable as they appear.

The irony is that removing a question often improves a survey more than adding one. Every unnecessary question gives participants one less opportunity to disengage. Every sentence rewritten in plain English makes it easier for people to answer what we’re actually trying to ask. Simplicity isn’t about dumbing research down. It’s about respecting the people we’re asking to help us. When we make research easier to understand and easier to complete, everyone benefits, including the quality of the decisions that follow.

We’re constantly challenging ourselves to make our research tools simpler, clearer and easier for people to engage with. The evidence tells us that when we do, the quality of the data improves too.

What the evidence says

  • Longer surveys increase respondent fatigue. As respondents become tired, they are more likely to speed through questions, skip open-ended responses, or give less thoughtful answers.
  • People often don’t drop out. Instead, they disengage while continuing to complete the survey. Researchers refer to this as satisficing, where respondents provide an answer that is “good enough” rather than their best answer.
  • Plain language improves data quality. Questions that are shorter and easier to understand reduce misunderstanding and improve the consistency of responses across all literacy levels.
  • More data isn’t always better. A smaller number of well-designed questions often produces more reliable insights than a longer questionnaire completed with less care.

 

If you work with us, challenge us to come back with the simplest, most focused version of the research. Ask us whether the questionnaire could be shorter. Ask whether every question has earned its place. It takes more effort to write a shorter survey. It takes more conversations, more difficult decisions and more discipline. But we’re happy to put in that effort, because focused research produces better quality data, and better quality data leads to better decisions.

Square Holes is a cultural insight studio.

We design mixed method explorations of people and culture beyond the category,  uncovering the patterns, tensions and shifts shaping behaviour to inform strategy, inspire innovation and enable confident decisions. Our studio model brings together the right mix of thinkers, researchers and specialists for each exploration. If you’re navigating change, entering a new market, or seeking deeper understanding of people and culture, let’s start a conversation >

Share this: