The Data Scientist Show - Daliana Liu A/B testing at Airbnb, building next-gen experimentation platform at Eppo - Che Sharma - The Data Scientist Show #068
31 snips
Aug 25, 2023 Che Sharma, former data scientist at Airbnb and founder of Eppo, talks about A/B testing best practices, A/B testing for ML models, and his career journey. They discuss successful A/B testing, interpreting and communicating test results, A/B testing best practices for ML models, centralizing experiment analysis, preparing data scientists for the future, developing communication skills, transitioning to a manager role, and the future of experimentation.
AI Snips
Chapters
Transcript
Episode notes
Airbnb's Neighborhood Guides Fail
- Airbnb launched Neighborhood Guides as a glossy content feature backed by PR and enthusiasm.
- AB testing revealed removing it boosted bookings, showing content distracted users from booking.
New Tab Boosted Airbnb Bookings
- Opening listings in new tabs at Airbnb, despite design opposition, led to the largest uplift in bookings.
- This simple UI change preserved search context and boosted user engagement effectively.
Encourage Experimentation with Uncertainty
- Convey the uncertainty in running experiments to encourage teams to try them.
- Wrap product work in experiments to gain valuable learning without extra effort.