Experiment Nation: The Podcast cover image

S3E17 - CRO Insights from a Metaanalysis of over 1000 A/B Tests with Georgi Georgiev

Experiment Nation: The Podcast

CHAPTER

The Power Distribution of a Single Test

In most cases, you really want to have more than 30% power there for 4% lift. Small changes like this are very unlikely to result in meaningful differences. Most practitioners understand that unless they have a drastically different variant, which has sort of an independent chance of being better than both the control and their first variant, then adding the variant seriously.

00:00
Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner