You’ve probably been in a meeting where a lot of ideas are circulated about how to improve an existing product or service. In these meetings, differing opinions can quickly turn into a battle of long-winded defenses. Fortunately, the emergence of A/B testing – once thought to be exclusive to tech firms – has become a viable and cost-effective way for all types of businesses to identify and test value-creating ideas.

A/B Testing: in statistical terms, A/B testing is a method of two-sample hypothesis testing. In laymen’s terms, this means comparing the outcomes of two different choices (A and B) by running a controlled mini-experiment.

Although the concept of A/B testing was galvanized by Silicon Valley giants, the rationale behind A/B testing is not new. The practice borrows from traditional randomized-control trials to create smaller, more scalable experiments.

As a very basic example, let’s say you are an abstract artist. Your paintings are informed by the world around you, but you cannot merely mimic landscapes. You are confident in your technique, but you still aren’t sure how the outside world—and more importantly art critics—are going to respond to your new paintings. Assessing the quality of art is a famously challenging process.

If you were to employ A/B testing for this scenario, you would start by creating two different paintings that are exactly alike. As you continue working, you would decide to change one small thing—let’s say you add a red square to one painting and not the other. Again, this means that everything about the paintings are alike except for this one modification. Once the change is made, you display the two paintings in randomly selected art galleries across the country and wait for your art agent, or another unbiased third party, to gather the reactions and report back to you.

After each painting has been placed in a reasonable amount of art galleries, perhaps you are informed that the painting with the small change received significantly more praise, or maybe it did not. The hypothetical outcome does not matter. Rather, what matters is that you can be reasonably confident that your change will or will not make the painting better, and you can go on to create better art as a result.

The randomization aspect of this design is explicitly emphasized because randomization is the gold-standard for eliminating biases. Art is a subjective field and evolves over time, and so do the preferences and opinions of customers, clients, or coworkers. A/B testing is not a static process, and tests can be repeated or complemented if companies believe that findings may not be valid or applicable anymore.

Companies like Google, Amazon, and Facebook have all used A/B testing to help create more intuitive web layouts or ad campaigns. Customers benefit and companies can reap measurable monetary returns by catering to market preferences. Momentum is now building to use this method outside of Silicon Valley. Jim Manzi, the founder of Applied Predictive Technologies, has advocated for the use of randomized experiments in other aspects of business, politics, and society in his book Uncontrolled.

As a final note, it is imperative that the design of A/B testing be rigorous to ensure the validity of your results. Furthermore, there may be some decisions where internal opinions are more cost-effective or timely.

Interested to learn more about the technical and conceptual aspects of A/B testing and how it can be used? Take HBX CORe and discover the basics of Business Analytics, Financial Accounting, and Economics for Managers.