Experiment Nation: The Podcast

S3E17 - CRO Insights from a Metaanalysis of over 1000 A/B Tests with Georgi Georgiev

May 20, 2023

01:00:28

Creator website

AI Chapters

Episode notes

Introduction

3min

The Importance of Analyzing Results

3min

How to Be an Expert in Statistical Methods

2min

The Importance of Deep Diving in A.B. Testing

3min

The Evolution of Analytics

3min

The Consequences of Running Tests Longer

6min

The Importance of Concurrent Tests

6min

The Importance of 80% Power in Testing

3min

The Power Distribution of a Single Test

6min

10.

The Risk of Losses in A.B. Tests

6min

11.

The Importance of Sequential Testing

6min

12.

The Relationship Between Bandits and Equation Tests

3min

13.

The Benefits of Running a Test Early

4min

14.

The Importance of Minimum Effective Interest in Tests

6min

15.

How to Be a Successful Statistician

3min

Georgi Georgiev performed a meta-analysis of over 1000 A/B tests and has a great deal to share about: - What you should expect your win rate to be - The benefits of sequential testing - Why you should report on confidence intervals

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.