The Product Podcast cover image

All the Ways That A/B Testing Sucks by Yammer PM

The Product Podcast

00:00

Yammer - How Many Users Do You Need?

A lot of people who are a lot smarter than me have determined how many users you need before you can consider something significant. Right now, I have a project I just kicked off with our Android team for something that's kind of buried a little bit deep in our product. And we only have about 31,000 users within a period of a month that engage with that feature. So if I wanna be able to A. test that, I'm gonna have to run that for an entire month, more or less. If I'll have 15,000 users on each side of the treatment and I'll hopefully be able to detect some significant changes,. But at that scale, I'll really only

Play episode from 13:53
Transcript

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app