Surveys can help your company identify opportunities, assess challenges, and set direction. If you're looking to design a survey for your own organization, you should be aware of common pitfalls that novice survey creators can run into.

Here are three survey question development mistakes made by startups and well-established companies alike. Caveat lector, or “let the reader beware”.

The Biased or Leading Question

“Our product reduces your tension by 10%. Would you like to buy it?”

This rhetoric might be fine for a home gym infomercial, but the question above is biased. A biased survey question prompts or leads a respondent to respond in favor or against a specific outcome, resulting in inaccurate data.  The example above mentions a reduction in tension that might influence the respondent to indicate they would make a purchase. A better question would be "How likely are you to buy this product?" Examples don’t need to be as extreme as the one highlighted to be considered biased.

Tips to avoid bias:

• Use neutral language. Do not favor one option –explicitly or implicitly –and do not lead a respondent to an answer.
• Vary the ordering of options in a list. This should be done across questions with similar answers to ensure a respondent will not make a decision based on chronological sequencing.

The Ambiguous Question

“How do you feel about your purchase?”

This question uses ambiguous and imprecise language. Quantifying or assessing subjective attitudes is difficult, and the burden shouldn’t fall on the respondent. A better approach is to provide options for the respondent: "How satisfied are you with your purchase? Extremely satisfied; somewhat satisfied; neutral; somewhat unsatisfied; extremely unsatisfied".

Tips to avoid ambiguity:

• Think critically and develop precise questions. Even better – don’t reinvent the wheel and look at existing metrics, such as those available at Survey Legend.
• Consider tools other than surveys. Conjoint analysis, for example, can help reveal how much a customer values a certain product or service.

The Complex Question

“If you had to get to work using a bicycle, bus, train, car, or on foot, which would you choose? Consider annual precipitation, your transportation budget, and carpooling opportunities.”

The question above assesses propensity for transportation methods, perhaps to help a city decide whether to allocate funding to bike share programs, commuter rail services, or bus routes. However, this question is overly complex, making it difficult for the respondent to answer.

Tips to avoid complexity:

• Trim the fat. Cutting unnecessary qualifiers could ease the intellectual burden. Understand, however, that you risk perpetuating the second question type by making assessments more open-ended.
• Qualitative interviews and focus groups are an alternative method of receiving detailed answers if time and effort allow. A pair-wise ranking system might also be appropriate in this scenario.

When your company conducts surveys, look for red-flags that could lead to inaccurate conclusions. Low response rates or an unrepresentative sample of respondents are especially sharp objects that could poke a hole in your metaphorical tire. These three tips are only the beginning of becoming a proficient survey designer. Knowing the best way to design assessments – and being aware of when other tools may be more appropriate – will lead to more effective company research.

Interested in learning more about survey design and other principles of Business Analytics? Take HBX CORe to strengthen your knowledge of Business Analytics, Financial Accounting, and Economics for Managers.

You’ve probably been in a meeting where a lot of ideas are circulated about how to improve an existing product or service. In these meetings, differing opinions can quickly turn into a battle of long-winded defenses. Fortunately, the emergence of A/B testing – once thought to be exclusive to tech firms – has become a viable and cost-effective way for all types of businesses to identify and test value-creating ideas.

A/B Testing: in statistical terms, A/B testing is a method of two-sample hypothesis testing. In laymen’s terms, this means comparing the outcomes of two different choices (A and B) by running a controlled mini-experiment.

Although the concept of A/B testing was galvanized by Silicon Valley giants, the rationale behind A/B testing is not new. The practice borrows from traditional randomized-control trials to create smaller, more scalable experiments.

As a very basic example, let’s say you are an abstract artist. Your paintings are informed by the world around you, but you cannot merely mimic landscapes. You are confident in your technique, but you still aren’t sure how the outside world—and more importantly art critics—are going to respond to your new paintings. Assessing the quality of art is a famously challenging process.

If you were to employ A/B testing for this scenario, you would start by creating two different paintings that are exactly alike. As you continue working, you would decide to change one small thing—let’s say you add a red square to one painting and not the other. Again, this means that everything about the paintings are alike except for this one modification. Once the change is made, you display the two paintings in randomly selected art galleries across the country and wait for your art agent, or another unbiased third party, to gather the reactions and report back to you.

After each painting has been placed in a reasonable amount of art galleries, perhaps you are informed that the painting with the small change received significantly more praise, or maybe it did not. The hypothetical outcome does not matter. Rather, what matters is that you can be reasonably confident that your change will or will not make the painting better, and you can go on to create better art as a result.

The randomization aspect of this design is explicitly emphasized because randomization is the gold-standard for eliminating biases. Art is a subjective field and evolves over time, and so do the preferences and opinions of customers, clients, or coworkers. A/B testing is not a static process, and tests can be repeated or complemented if companies believe that findings may not be valid or applicable anymore.

Companies like Google, Amazon, and Facebook have all used A/B testing to help create more intuitive web layouts or ad campaigns. Customers benefit and companies can reap measurable monetary returns by catering to market preferences. Momentum is now building to use this method outside of Silicon Valley. Jim Manzi, the founder of Applied Predictive Technologies, has advocated for the use of randomized experiments in other aspects of business, politics, and society in his book Uncontrolled.

As a final note, it is imperative that the design of A/B testing be rigorous to ensure the validity of your results. Furthermore, there may be some decisions where internal opinions are more cost-effective or timely.

Interested to learn more about the technical and conceptual aspects of A/B testing and how it can be used? Take HBX CORe and discover the basics of Business Analytics, Financial Accounting, and Economics for Managers.