Introduction

The Power of Experiments, as the book title suggests, is about experimentation. The authors write this book to show managers how to make the most of experimental results. I read it to find out how experiments can impact on management practices.

Author

Michael Luca is is the Lee J. Styslinger III Associate Professor of Business Administration at Harvard Business School. He serves on the board of directors at the National Association of Business Economics, the academic advisory board of the Behavioural Insights Team, and the advisory board of the CNBC Technology Executive Council, and is a faculty research fellow at the National Bureau of Economic Research.

Max H. Bazerman is Jesse Isidor Straus Professor of Business Administration at the Harvard Business School. He received the Lifetime Achievement Award from the Organizational Behavior Division of the Academy of Management in 2019.

Contents

The Power of Experiments has 15 chapters which are divided into 3 parts.

Part I is Breaking Out of the Lab and has 3 chapters. They are The Power of Experiments, The Rise of Experiments in Psychology and Economics, and The Rise of Behavioral Experiments in Policymaking.

Part II is Experiments in the Tech Sector. The chapters here include From the Behavioral Insights Team to Booking.com, #AirbnbWhileBlack, eBay’s $50 Million Advertising Mistake, Deep Discounts at Alibaba, Shrouded Fees at StubHub, Market-Level Experiments at Uber, and The Facebook Blues.

Part III is Experimenting for the Social Good and contains 5 chapters. These are Behavioral Experiments for the Social Good, Healthy, Wealthy, and Wise, The Behavior Change for Good Project, The Ethics of Experimentation, and A Final Case for Experiments and Some Concluding Lessons.

Review

What are experiments? Experiments are a way to systematically and objectively try new ideas. If they are efficiently and effectively run, the evidence will be more accurate and we might get a valid conclusion.

According to the authors, the 4 purposes of experiments are:

1. testing theory and mechanisms,

2. understanding magnitudes and tradeoffs,

3. evaluating policies or products, and

4. fact finding

How do experiments help in management? Experiments complement intuition and guesswork with evidence-based decision making. Thus, they are valuable for determining what type of intervention is likely to improve decision making in a particular context.

When we run experiments, we need to run them in a variety of contexts and focus on developing frameworks to apply across decisions, instead of relying on a single data point from an experiment. We must think about the relationship between short- and long-term outcomes, and invest heavily in long-term outcomes. The length of time for tracking an outcome depends on the question at hand, and on the relationship between short-term and long-term outcomes.

Effective experimentation requires managerial judgment, careful interpretation of results, and an appreciation of both the strengths and limitations of experiments. The value of an experiment is limited by the measurable outcomes. Besides that, we need to accept uncertainty when running experiments as we might end up concluding that a new idea is not effective

Experiments offer behavioral insights but we need to think about the context, design choices, and unintended consequences of the interventions. Default options and other elements of the choice environment matter in shaping human behavior. We also have to remember that lab experiments helps us know whether and when an effect might be relevant, but would not predict the exact effect in any particular real-world setting.

The authors tells a story about experimentation rather than bore the readers with the technical detail. This might be the result of experiments that showed that humans are more easily persuaded by stories, rather than facts alone.

Overall, The Power of Experiments shows us the value of experiments. Besides that, the authors also discuss about some common pitfalls of experiments. So, how does this book help in management? I think it shows us how experiments can help in making an evidence-based decision rather than simply making a decision based on intuition. By running an experiment before implementing an action, it will allow us to see if the measure would really work.

Quotes

1. We like to have faith in our intuition, but the fact is, it’s often flat-out wrong.
2. There is no single perfect way to run an experiment.
3. Secrecy is a bigger danger than publicity.
4. Good leaders have enough humility to admit what they don’t know, identify the best options in an uncertain world, and experiment.
5. But, as with all evidence gathering, the value of the evidence depends entirely on the people running the experiments.

Rating

Interested in The Power of Experiments?

You may get the book from through the link below.

Get the ebook from Rakuten kobo here

Share on Facebook

Tweet