Stefan Thomke, a Harvard Business School professor and author of "Experimentation Works," delves into the transformative power of experimentation in business. He discusses how successful companies like Netflix and Booking.com leverage A/B testing for competitive advantage. Thomke highlights the pitfalls of relying on intuition, advocating for a data-driven decision-making approach. He shares insights on cultivating an experimental culture within organizations, encouraging all employees to engage in small-scale experiments to drive innovation and growth.
23:28
forum Ask episode
web_stories AI Snips
view_agenda Chapters
menu_book Books
auto_awesome Transcript
info_circle Episode notes
question_answer ANECDOTE
Bing's $100M Experiment
An employee at Microsoft's Bing insisted on running an ad display experiment despite skepticism.
The experiment generated over $100 million in additional revenue in its first year, proving the power of testing.
insights INSIGHT
Risk Misconceptions of Experiments
Business leaders often overestimate the risks and underestimate the rewards of running experiments.
Proper stoppage rules and acknowledging uncertainty help mitigate perceived risks of experimentation.
question_answer ANECDOTE
Kohl's Opening Hour Experiment
Kohl's ran experiments to test if opening an hour later would save money without hurting revenue.
The results showed neutral impact on revenue, illustrating the value of testing big decisions.
Get the Snipd Podcast app to discover more snips from this episode
In "Experimentation Works," Stefan Thomke provides a comprehensive guide to implementing experimentation in business settings. The book explores various case studies, demonstrating how companies of all sizes can leverage data-driven insights to improve decision-making and drive innovation. Thomke emphasizes the importance of building a culture that embraces experimentation and learning from failures. The book offers practical strategies and tools for designing, conducting, and analyzing experiments, helping businesses to make more informed decisions and achieve better outcomes. It also addresses the challenges and potential pitfalls of experimentation, providing guidance on how to mitigate risks and maximize the benefits.
Stefan Thomke, professor at Harvard Business School, says running experiments can give companies tremendous value, but too often business leaders make decisions based on intuition.
While A/B testing on large transaction volumes is common practice at Google, Booking.com, and Netflix, Thomke says even small firms can get a competitive advantage from experiments. He explains how to introduce, run, and learn from them, as well as how to cultivate an experimental mindset at your organization. Thomke is the author of the book Experimentation Works: The Surprising Power of Business Experiments and the HBR article “Building a Culture of Experimentation.”
Key episode topics include: experimentation, data and analytics, risk management, decision-making, strategy
HBR On Strategy curates the best case studies and conversations with the world’s top business and management experts, to help you unlock new ways of doing business. New episodes every week.
HBR On Strategy 
