Adventures in Machine Learning
A/B Testing with ML ft. Michael Berk - ML 181
Jan 2, 2025
Michael Berk, a data scientist at Tubi, specializes in A/B testing and machine learning for streaming services. He dives into how A/B testing proves causality, emphasizing its crucial role in data-driven decisions for businesses. The discussion contrasts frequentist and Bayesian methodologies, highlighting sample size importance. Berk also shares insights on adapting to varying business environments and the shift from viewer count to watch time metrics in streaming. He wraps up with thoughts on the need for clear communication of data science principles to engage interest in machine learning.
45:41
Episode guests
AI Summary
AI Chapters
Episode notes
Podcast summary created with Snipd AI
Quick takeaways
- A/B testing, through randomized control trials, serves as a crucial method to establish causality and derive actionable insights in data science.
- Utilizing both frequentist and Bayesian approaches for hypothesis testing can help organizations optimize sample sizes and validate decision-making frameworks effectively.
Deep dives
The Importance of A-B Testing in Data Science
A-B testing serves as a critical method in data science for establishing causality through randomized control trials. This process involves comparing different treatments to a control group to analyze statistically significant lifts, allowing data scientists to derive actionable insights from experiments. The discussion highlights the choice between frequentist and Bayesian approaches, with the former being preferred for its simplicity and efficiency in hypothesis testing. By optimizing sample sizes and evaluating control metrics, organizations can derive significant business value and validate their decision-making frameworks.
