Ronny Kohavi

Ronny Kohavi, a leading expert in A/B testing with a rich history at Airbnb, Microsoft, and Amazon, shares invaluable insights into the world of experimentation. He emphasizes how fostering a culture of experimentation can lead to significant product innovations. The discussion covers common pitfalls in A/B testing, including sample ratio mismatches, and highlights the importance of trust and data integrity. Ronny also shares surprising results from his experiments and best practices for optimizing testing processes, making data-driven decisions essential for growth.

Data scientist Ronny Kohavi discusses A/B testing statistics concepts all experimenters must know, including the Overall Evaluation Criterion, Twyman's law, and the amount of traffic needed for A/B tests. He shares insights on handling tests that turn out to be wrong and presenting experiment results to the CEO of Amazon. The podcast also highlights the benefits of AV tests and variety in designs for experimentation and CRO, while cautioning against the pitfalls of long-term testing.

Ronny Kohavi, an expert in experimentation and personalization, discusses the role of online experiments during the current crisis and the need for a trustworthy environment for testing. He shares great examples from Amazon and Microsoft, and explores the connection between experimentation, personalization, and machine learning.