The Data Scientist Show - Daliana Liu Stop abusing A/B testing, toxic experimentation culture, how to run A/B tests with rigor - Che Sharma - The Data Scientist Show #071
17 snips
Nov 4, 2023 Che Sharma, former data scientist at Airbnb and founder of Eppo, discusses toxic behaviors in experimentation culture, A/B testing best practices, and A/B testing for ML models on The Data Scientist Show. Topics include statistical power, effect size, monitoring metrics, alternative methods to A/B testing, difference in differences method, and A/B testing in ML and AI.
Chapters
Transcript
Episode notes
1 2 3 4 5 6 7
Introduction
00:00 • 2min
Toxic Experimentation Behaviors and the Importance of Statistical Power
01:47 • 4min
Understanding Effect Size and Selecting Meaningful Metrics
06:05 • 18min
Monitoring and Guarding Metrics
24:34 • 20min
Alternative Methods to A/B Testing
44:48 • 6min
Using Difference in Differences in Measuring Effects
50:55 • 9min
The Importance of A/B Testing in ML and AI
59:43 • 4min