Experiment Nation: The Podcast cover image

S3E15: A/B Testing Statistics issues CROs should ignore with Timothy Chan

Experiment Nation: The Podcast

00:00

Facebook's Culture of Overlapping Tests

In the world of product-based experimentation, that just fundamentally doesn't work. If you assume every test is actually like an AP test split at 50-50, that's actually two to the power of 50 combinations. There's not enough users in the world to make sure that everybody is represented properly. So I think in some ways, folks who are really worried about interaction effects are making too much of a big deal.

Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner