The Data Scientist Show cover image

The Data Scientist Show

A/B testing at Airbnb, building next-gen experimentation platform at Eppo - Che Sharma - The Data Scientist Show #068

Aug 25, 2023
Che Sharma, former data scientist at Airbnb and founder of Eppo, talks about A/B testing best practices, A/B testing for ML models, and his career journey. They discuss successful A/B testing, interpreting and communicating test results, A/B testing best practices for ML models, centralizing experiment analysis, preparing data scientists for the future, developing communication skills, transitioning to a manager role, and the future of experimentation.
01:14:15

Episode guests

Podcast summary created with Snipd AI

Quick takeaways

  • Sufficient data collection is crucial in experimentation to avoid inconclusive results and inaccurate institutional knowledge.
  • Experimentation can transform a company's culture and decision-making process, enabling data-driven decision-making and fostering innovation.

Deep dives

The Importance of Sufficient Data in Experimentation

One of the main ideas discussed in the podcast is the need for sufficient data in experimentation. The speaker highlights a past experiment at Airbnb that was prematurely terminated before enough data was collected, leading to inconclusive results. This emphasizes the importance of running experiments for an adequate duration to detect meaningful insights. The failure to handle this failure mode can create inaccurate institutional knowledge and hinder decision-making.

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode