The most common survey question mistakes to avoid are leading wording, questions that assume the answer, confusing rating scales, and answer options that do not cover the real range of answers. Each one pushes respondents toward a particular response, and the bias then flows straight into your tables, tests and conclusions without ever producing an error message.
A questionnaire is a measurement instrument. If the instrument is bent, the measurement is bent, and no amount of clever analysis afterwards can straighten it out. Most of the errors below are cheap to fix at the drafting stage and expensive to live with after fielding.
Here is what each mistake looks like, how to detect it, and how to rewrite the question.
Table of Contents
- 1Survey Question Mistakes to Avoid at a Glance
- 2Survey Question Mistakes to Avoid: Quick Review Checklist
- 31. Asking Leading Questions
- 42. Using Loaded or Emotionally Biased Language
- 53. Writing Questions That Assume the Answer
- 64. Creating Confusing Response Scales
- 75. Forcing Respondents to Choose the Wrong Answer
- 86. Mixing Several Ideas Into One Question
- 97. Using Jargon, Slang, or Undefined Terms
- 108. Asking About Memories That Are Unreliable
- 119. Writing Questions That Overlap Too Much
- 1210. Skipping a Final Review and Pilot Test
- 13Frequently Asked Questions
- 14How can I tell if a survey question is leading?
- 15What is the best way to write neutral survey questions?
- 16How many questions should a student survey include?
- 17Should every survey question use the same answer scale?
- 18How do I pilot test a questionnaire before collecting data?
- 19How should I ask sensitive questions without making respondents uncomfortable?
- 20Conclusion: Fix the Questions Before Collecting Data
Survey Question Mistakes to Avoid at a Glance
| Mistake | Why it damages the data | Quick fix |
|---|---|---|
| Leading questions | Signals the answer you want, so agreement inflates your result | Strip evaluative words and name a time period |
| Loaded or emotional wording | Triggers social desirability and defensive responding | Swap adjectives for plain nouns and verbs |
| Assuming the answer | Forces a rating from people with no experience of the thing | Add a neutral screener and route non-users past it |
| Confusing response scales | Endpoints and directions drift, so responses mean different things | Label every scale point and keep one direction |
| Forced choice | Respondents pick the least wrong option, not the true one | Add not applicable, other and prefer not to say where they fit |
| Double-barreled questions | No single answer exists when the halves disagree | Split it into one question per idea |
| Jargon and undefined terms | Respondents answer from their own interpretation | Use plain words and define the technical term |
| Unreliable recall | People estimate counts they cannot remember | Shorten the reference period or offer ranges |
| Overlapping questions | Fatigue turns into straightlining and inconsistent answers | One construct, one item, unless you need a scale |
| No review or pilot test | Skip logic, typos and layout issues reach real respondents | Think-aloud pretest with 5 to 8 people, then soft launch |
Worth saying plainly: these are question-level errors. A beautiful questionnaire still cannot rescue a bad sampling frame, and a clumsy one will not save a good sample. Fix the item wording first, then worry about who answered.
Survey Question Mistakes to Avoid: Quick Review Checklist
Run this list in about ten minutes before you field anything.
- Search every stem for the words and, or, because and since. Each one marks a candidate double-barreled item.
- Delete any adjective that no reasonable person would disagree with.
- Check that every question names who, what and over what period.
- Read each item aloud. If you run out of breath, split it.
- Confirm response options are mutually exclusive and collectively exhaustive, and add other when the list is closed.
- Check scale direction runs the same way through the whole questionnaire.
- Add a neutral midpoint or a don’t know wherever an opinion is genuinely optional.
- Test every screener and skip path with someone who does not qualify.
- View the questionnaire on a phone screen before you send it.
- Give the draft to a colleague outside the project and ask them to paraphrase five items.
Step ten is the cheapest test there is. If two people describe the same question differently, the wording is doing work you did not intend.
1. Asking Leading Questions
A leading question is any question whose wording contains or implies the answer you are hoping for. It is the most common of the survey question mistakes to avoid because it is usually written by someone who already believes something.
How to spot it: look for superlatives such as best, most important or major; for presupposition, where the question takes something as given; for second-person evaluative words like you love, you found easy or you agree; and for tag questions such as don’t you think.
Bad: How helpful was our support team in resolving your issue?
Fixed: Thinking about your most recent contact with our support team, how satisfied were you with the help you received?
The fixed version still measures helpfulness, but it lets a dissatisfied customer give a low number instead of picking the least-bad answer on a scale built around praise.
2. Using Loaded or Emotionally Biased Language
Loaded wording is language with an emotional charge attached. It does not have to argue for anything; a single word such as excellent, painful or unfair can do the work of a whole paragraph of bias.
How to spot it: any adjective inside the question stem that only one direction of answer would ever support. If the stem praises the product, agreement measures politeness.
Bad: How important is excellent customer service when you choose a provider?
Fixed: How important is each of the following to you when you choose a provider? Customer service, price, product range, location.
Words like helpful, fast and unreliable also carry a built-in benchmark. Replace them with something countable, such as minutes or hours, or with a rating scale whose points are defined.
3. Writing Questions That Assume the Answer
An assumptive question quietly states that something is true and then asks about it. The trouble is that people who do not share the assumption have no honest exit, so they answer anyway and inflate your numbers.
How to spot it: look for embedded presupposition. Your recent purchase, the support you received and the course you took all assume the respondent did the thing.
Bad: How much did you enjoy your recent purchase?
Fixed: First, did you buy from us in the past three months? If yes: How satisfied were you with that purchase? If no: skip to the next section.
The screener matters as much as the rewrite. Keep it factual and short, and route the people who fail it past the question entirely rather than letting them guess.
4. Creating Confusing Response Scales
Bad scales have a short shelf life. Respondents learn the pattern after two or three items, and past that point they stop reading the labels and start picking a number that feels right.
How to spot it: unlabelled scale points, an odd or inconsistent number of points, direction that reverses mid-scale, endpoints that do not match the question, and a single agree-disagree block doing the work of a frequency scale.
Bad: Never, Rarely, Sometimes, Often, Always, with the last two points reversed in the printed form.
Fixed: Never, Rarely, Sometimes, Often, Always, plus not applicable for people who have no experience to judge.
Label both endpoints, keep one direction throughout, and use one scale type per block of items. If you mix scales, say so in the instructions so a respondent knows a change is coming.
5. Forcing Respondents to Choose the Wrong Answer
Forced choice happens whenever the options on offer do not cover the answers people actually hold. The respondent then picks the nearest fit, and your dataset records a preference they do not have.
Three escapes are worth building into most questionnaires:
- Not applicable when the respondent has no experience to judge, for example if they never used the feature.
- Other, please specify when you have listed options but cannot promise the list is complete.
- Prefer not to say when a question touches something sensitive and the study has an ethics board or review process that expects an opt-out.
Test your list against the question: are the options mutually exclusive, and do they collectively exhaust the plausible answers? Most weak lists fail the second test, usually in the direction of money, time or frequency.
6. Mixing Several Ideas Into One Question
A double-barreled question, also called a compound question, asks two or more things in one item. Sometimes both barrels point the same way and nobody notices. When they disagree, the respondent has no valid answer and picks one at random.
How to spot it: scan for and, or, because, since, plus semicolons; check whether the question names two different objects, two different time frames or two different groups of people.
Bad: How satisfied are you with the quality and price of our products?
Fixed, item one: How satisfied are you with the quality of our products?
Fixed, item two: How satisfied are you with the price you paid?
A customer who loves the quality and dislikes the price cannot answer the original question honestly. Splitting it costs you one extra item and saves you an unusable number.
7. Using Jargon, Slang, or Undefined Terms
Technical terms are fine in a methods section and dangerous in a question stem. Each one carries a household definition that differs from yours, and the data you get back measures the respondent’s version.
How to spot it: a word your target group would not use in conversation. Churn, onboarding flow, latency, throughput and likelihood to recommend all qualify, and each one means something different in different organisations.
Bad: How likely are you to churn within the next two quarters?
Fixed: How likely are you to stop using our service in the next six months?
Where the technical term is the point of the research, keep it and define it in the questionnaire itself, or add a familiarity screener so you can analyse experienced and inexperienced respondents separately.
8. Asking About Memories That Are Unreliable
Recall questions fail quietly. Asked how many times they called support last year, most respondents do not retrieve the count, they reconstruct it, and the reconstruction drifts toward round numbers and recent events.
How to spot it: a long reference period, a request for an exact count, or a question about frequency of something irregular.
Three repairs work well: shorten the reference period to the last week or month, convert counts into bounded bands such as none, one to two, three to five, more than five, or replace the recall with a behavioural indicator such as whether a transaction appears in the records you already hold.
Sensitive topics need the same caution. Income, health and illegal activity are best asked in ranges, with a prefer-not-to-say option and wording that makes clear why you are asking.
9. Writing Questions That Overlap Too Much
Redundant items are not free. They add reading load, they bore respondents, and they produce the satisficing behaviour that shows up later as straightlining and speeding.
How to spot it: build a quick construct table before drafting. List the idea each item is meant to measure, and any two items that share a construct need a reason to exist. Sometimes they do, because a multi-item scale gives you a reliability estimate and a spread of responses. Sometimes they do not, and you have just made the survey longer.
Bad: three separate grids asking how satisfied, how happy and how pleased respondents are with the same service.
Fixed: one multi-item scale with the three wordings as rows, so you can report an average rather than three near-identical numbers.
Matrix grids save time for respondents and punish careless ones. Keep grids narrow, and never repeat the same scale label set twice in one questionnaire.
10. Skipping a Final Review and Pilot Test
The last stages are where avoidable errors survive into the field. A typo in one response option, an instruction that never got written, a skip condition pointing at a deleted question, or a layout that collapses on a phone screen.
How to spot it: you have never watched a real person answer it. Proofreading your own draft does not count, because you already know what every item means.
Cognitive interviewing is the formal version of this. You sit with 5 to 8 people from your target group, ask them to think aloud while answering, and then ask what each question meant to them. If their paraphrase differs from your intention, you rewrite the item, not the respondent.
After that, run a soft launch to a small slice of the real sample. Watch completion rate, time taken and item non-response per question, not just the overall total. A question with a 30 percent skip rate is usually a wording problem wearing a layout costume.
Frequently Asked Questions
How can I tell if a survey question is leading?
Read the question stem and ask whether a respondent could plausibly disagree with it without feeling awkward. Signs include superlatives such as best or most, second-person praise, tag questions like don’t you think, and any wording that treats your hypothesis as settled. A quick test: if two colleagues describe the item differently, or if you would be annoyed to see the opposite answer in your results, rewrite it before fielding.
What is the best way to write neutral survey questions?
Keep the stem short, name the referent and the time period, and remove every evaluative word. Put the object of the question in plain language, then offer response options that are mutually exclusive and collectively exhaustive. Neutral does not mean vague: vague questions create measurement error, while neutral questions create accurate measurement. Review each item against the criterion that a reasonable person could give any answer without it feeling rude or contradictory.
How many questions should a student survey include?
For a student project, ten to fifteen questions is usually enough to answer the research question and short enough to finish in five to ten minutes. Long surveys produce fatigue, straightlining and drop-off, which show up as missing data rather than as complaints. Cut anything that will not appear in your write-up. If you need many items, use a short multi-item scale instead of repeating similar questions one at a time.
Should every survey question use the same answer scale?
Not always. Consistency helps within a block of items that measure the same construct, but a single agree-disagree scale cannot measure frequency, satisfaction and knowledge equally well. Match the scale to the question: frequency items for how often, defined endpoints for quality, and a don’t know or not applicable option wherever an opinion is optional. If you switch scale mid-questionnaire, tell respondents in the instructions.
How do I pilot test a questionnaire before collecting data?
Run a cognitive interview with 5 to 8 people from your target group, ask them to think aloud while answering, and compare their paraphrase of each item with your intention. Then soft launch the questionnaire to a small slice of the real sample and check completion rate, time taken and item non-response question by question. The purpose is to test the instrument, not to gather findings, so treat the data as a diagnostic rather than evidence.
How should I ask sensitive questions without making respondents uncomfortable?
Say why you are asking, keep it brief, and offer a prefer not to say option. Use bounded ranges rather than exact figures, ask about behaviour instead of identity where you can, and place sensitive items after rapport has been built, not in the first screen. If your study goes through an ethics review, follow what it requires. For anything touching health, income or legal history, keep the language plain and never imply that a particular answer is expected.
Conclusion: Fix the Questions Before Collecting Data
Start by reading every item aloud and deleting anything that praises, blames or assumes. Then split the double-barreled ones, label the scale points, check that every answer option covers the range of real answers, and hand the draft to someone outside the project for a think-aloud pretest.
Those five steps catch most of the survey question mistakes to avoid before a single response arrives. The rewrite takes an afternoon. Recreating the study because the data is unusable takes a term.
