Leading questions can wreak havoc on your survey results, leaving you with data that isn’t representative of your respondents’ opinions.
In this article, we’ll explore what leading questions are, why they’re bad for your data collection, and how to avoid leading questions in surveys.
What is a leading question?
Leading questions are types of questions that influence respondents to answer in a biased way. Usually, the question is phrased in a way that prompts respondents to select a particular answer, rather than allowing respondents to give an unbiased answer.
This is where the name comes from, as they lead respondents to express a particular opinion that may not be their own.
This can be intentional, where an organization wants to confirm something they want to believe, or unintentional, where the survey hasn’t been tested from a respondent’s perspective. Either way, leading questions will never collect honest and accurate data.
What is an example of a leading question
Here’s a very clear example of a leading question:
“How much did you enjoy today’s session?”
- Extremely enjoyed it
- Very much enjoyed it
- Enjoyed it
- Somewhat enjoyed it
- Did not enjoy the event
It’s unlikely you’d ever produce something like this accidentally. It’s glaringly obvious this question is pushing survey respondents to agree that enjoyed the event in some way.
To fix this question, you’d need to balance the scale-out with an equal number of positive and negative ratings (along with a neutral answer option.
What is a leading question?
There is an argument that leading questions can help optimize surveys, namely that they are:
- Time-efficient: The nature of leading questions is to push respondents to certain answers, meaning that people are spending less time considering each question.
- Focused: They keep the survey focused on the specific objectives of the survey creator. For example, you can create a survey revolving around why your product is great, without ever asking if a customer likes your product.
- Specific: They exceed at avoiding ambiguity in surveys, as they are hyper-focused on collecting certain answers.
But you have to consider the impact of using leading questions on your relationship with respondents. These aren’t just a source of data, they’re real people who should be treated as such.
If you’re a business, these respondents are your customers. And if they don’t believe their opinion is represented in your survey questions, your relationship with them will take a hit. This doesn’t just mean you’ll lose their feedback, you may lose their loyalty.
Even if your project is purely academic, the data you collect absolutely has to be accurate or your research is void.
This brings us to the other downside to using leading questions; incurring survey bias.
What is an example of a leading question
Survey bias (or response bias) is where respondents answer questions dishonestly or inaccurately due to an internal or external influence.
The problem with collecting this type of data is that you’re unable to draw any accurate conclusion because the responses themselves are false.
Bias can be a result of a number of factors in a survey, not just leading questions, including:
- Survey design
- Question order
- Survey fatigue
Your responsibility as a survey creator is to reduce all sources of bias by as much as possible.
Types of leading questions
There are a few types of leading questions you should avoid in surveys. We’ll go through them below, provide some examples, and then some suggestions on how they can be improved to collect open and honest data.
Leading questions based on assumptions
This is where questions are asked based on some assumption made by the survey creator.
For example, in education surveys a leading question based on assumption would be:
“How much did you enjoy your time in class?“
This makes the assumption that students did enjoy their time in classes, rather than asking whether they enjoyed it.
A better phrasing would be:
“How was your time in class?” (Not at all enjoyable – Extremely enjoyable).
Leading questions based on interconnected statements
Leading questions based on interconnected statements are those where a statement closely related to the question is included in the question itself.
In a customer satisfaction survey, a leading question would look like this:
“Most customers think our product is good value for money. Do you think our product is good value for money?“
This puts pressure on respondents to agree with the other customers you’ve referred to and may influence how honest they are in their feedback.
It’s understandable that you would want customers to think your product is good value for money, but by pushing them to pick agree, you aren’t actually collecting useful data. If customers think your product should be less expensive, this is something you should know.
A better example of this would be:
“Do you think our product is good value for money?” (Not at all good value for money – Extremely good value for money).
By removing the interconnected statement, you allow respondents to give an honest answer.
Leading questions with direct implications
Leading questions with direct implications are those that prompt respondents to consider something that may happen based on something that is implied to have happened.
Sounds a little convoluted, but an example for an employee satisfaction survey will help provide some clarity.
“If you love working at this company, would you recommend us as a place to work?“
The first phrase in this question has assumed that the employee loves working at the company. Or at least, hasn’t actually asked whether they do enjoy working there.
The best practice for this question would be to split it into two separate questions:
“How do you feel about working at this company?“
Firstly, you want to remove the emotive language from the first question (like above). You want employees to be able to give their honest opinion to be able to address the reason behind them.
Then you can ask a follow-up question:
“Would you recommend this as a place to work?“
You might even find that you don’t need to ask both questions, which gives you an opportunity to optimize your survey.
Coercive leading questions
These types of leading questions attempt to coerce respondents into choosing a particular answer.
An example of a coercive question in a market research survey would be:
“You have heard of our product/ company, haven’t you?“
When a question is phrased like this, it can make respondents feel like they’re being told to agree with something. This will alienate your respondents, and will likely result in inaccurate data or survey drop-outs.
Questions aren’t always written like this intentionally. You can easily read the above as a statement, which a respondent can either agree or disagree with. But you have to consider your respondents’ perspectives.
If there’s even a possibility they may interpret a question in a way other than you intended, you should rephrase it to something more neutral.
For example:
“Have you ever heard of our product/ company?“
Coercive leading questions
Scale-based leading questions are those that lean towards either a positive or negative sentiment, meaning they aren’t inclusive of all opinions.
For example, an event survey might ask how satisfied attendees were with the event:
“How satisfied were you with the event?“
- Extremely satisfied
- Satisfied
- Somewhat satisfied
- Dissatisfied
- Extremely dissatisfied
There are more options above that lean towards a positive (satisfied) answer than there are negative.
A more balanced scale would look like this:
“How satisfied were you with the event?“
- Extremely satisfied
- Satisfied
- Neither satisfied nor dissatisfied
- Dissatisfied
- Extremely dissatisfied
This gives respondents a neutral answer to provide if they don’t feel strongly enough about the event to express a positive or negative opinion.
How to avoid leading questions in surveys
If you want to collect honest and open feedback, here are some tips on how to avoid leading questions in surveys.
- Avoid emotive language that may influence a respondent’s answer choice.
- Write simple and clear questions that encourage honest survey answers.
- Be specific when asking questions, vague wording can lead to incorrect interpretations.
- Avoid jargon or technical terms that are not easily understood by the average respondent.
- Always have a friend or colleague test your survey and give feedback about your questions.
- Provide an ‘Other’ answer option to allow respondents to write their own answer rather than choose one of your own.
- Provide neutral answer choices that allow respondents to opt out of a question.
- Ensure your answer options are exhaustive and aren’t leaning in a positive or negative manner.
- Don’t write double-barrelled questions that ask two things in one space.
Wrapping up
If you want to collect accurate data that is representative of your respondents’ true opinions, you have to avoid asking leading questions in surveys. This also goes for the answer choices you provide in close-ended questions, if they aren’t exhaustive then you are still leading some respondents.
Adding a neutral answer option (e.g. “Prefer not to answer” or “Not applicable”) is another great way of giving survey participants more agency. People are more likely to drop out of your survey if they feel they aren’t going to be heard.
