App Revenue Growth Podcast with Michael Stysin

Ep.2. Klarna lead on what A/B testing really is, why testing everything is a bad idea, and why you should never rely on A/B testing results only

Dec 19, 2022

32:01

Mateusz Wrzeszcz, ASO lead at Klarna and ex-Phiture.
In this episode, we'll discuss what A/B testing really is, what are the most common pitfalls and misconceptions regarding A/B testing, which tools to use, and how to test to get fair results.

In this episode:

01:08 What should mobile apps A/B tests?
01:32 Why testing everything is a bad idea
02:48 How to approach data/statistics when doing A/B testing
03:30 The difference between simple A/B testing and advanced A/B testing with probabilistic methods
06:14 The most common pitfalls & misconceptions when it comes to A/B testing and how to avoid them
12:40 Best practices for A/B testing
14:00 How to validate the results of A/B testing
18:39 A/B testing tools
21:00 Case study

The podcast is supported by Qonversion Inc., the leading european revenue & analytics platform for mobile apps with subscriptions.

YouTube