Office for National Statistics
Want to Make Better Decisions, Start Experimenting
Article

Want to Make Better Decisions, Start Experimenting

Four lessons for using randomized controlled experiments to create value for your company and customers.

Michael Luca and Max H. Bazerman
MIT Sloan Management Review, 2020
Read the article

Read or listen offline

Amazon Kindle
  • 0.5×
  • 0.75×
  • 1×
  • 1.25×
  • 1.5×
  • 1.75×
1×

Editorial Rating

9

Qualities

  • Overview
  • Concrete Examples
  • Engaging

Recommendation

The era of gut instinct as the sole source of information for making decisions is over. Companies now use randomized controlled experiments to resolve issues involving sales or algorithms. Writing for the MIT Sloan Management Review, Michael Luca and Max H. Bazerman report four lessons gleaned from research and interviews to highlight the basics of making your experiments relevant and valuable. From integrating testing into your decision-making processes to ensuring that your experiments provide the results you need, their analysis can help you build your business.

Summary

To leverage experiments to make good business decisions follow four lessons:

Google executives don’t debate whether adding a certain color could increase ad revenue; they run an online experiment. Annually, the company conducts more than 10,000 tests to resolve issues from sales to algorithms. Randomized controlled experiments are ubiquitous in the tech industry.

Corporations such as Amazon, Facebook, Uber and Yelp also act only after they’ve tested a planned change by capturing user behavior and reactions. The same holds for many companies that are not internet-based. For example, the Campbell Soup Company has used experimentation to gain “evidence-based insight” for more than 20 years.

To get the most utility from your test results, determine precisely what decision you want to make. Then you can make sure that the experiments you design and the information you collect will provide the answers you want.

As more business decisions come to depend on test data, companies must develop processes for creating tests and interpreting the results. A test must help the business while taking customer privacy...

About the Authors

Harvard business professor Max H. Bazerman and associate professor Michael Luca co-wrote The Power of Experiments: Decision-Making in a Data-Driven World.

Comment on this summary or 开始讨论