Experiment Nation: The Podcast cover image

S3E15: A/B Testing Statistics issues CROs should ignore with Timothy Chan

Experiment Nation: The Podcast

00:00

Facebook's Culture of Overlapping Tests

In the world of product-based experimentation, that just fundamentally doesn't work. If you assume every test is actually like an AP test split at 50-50, that's actually two to the power of 50 combinations. There's not enough users in the world to make sure that everybody is represented properly. So I think in some ways, folks who are really worried about interaction effects are making too much of a big deal.

Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app