
S3E17 - CRO Insights from a Metaanalysis of over 1000 A/B Tests with Georgi Georgiev
Experiment Nation: The Podcast
00:00
The Risk of Losses in A.B. Tests
When you have a big sample size, that also means big risk. So per unit time that you're running the test, you have bigger risk of losses if these variants are worse than the control. Or if they are truly better, you're running a higher risk of missing out on revenue. I would actually recommend obviously to just run the test on average revenue per user when you can,. But yeah, that would be a side topic to consider.
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.