Experiment Nation: The Podcast cover image

S3E17 - CRO Insights from a Metaanalysis of over 1000 A/B Tests with Georgi Georgiev

Experiment Nation: The Podcast

00:00

The Risk of Losses in A.B. Tests

When you have a big sample size, that also means big risk. So per unit time that you're running the test, you have bigger risk of losses if these variants are worse than the control. Or if they are truly better, you're running a higher risk of missing out on revenue. I would actually recommend obviously to just run the test on average revenue per user when you can,. But yeah, that would be a side topic to consider.

Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner