Daliana's Game Why 80% of A/B tests fail, how to 10X your experimentation velocity - Kristi Angel - The Data Scientist Show #088
23 snips
Apr 8, 2024 Kristi Angel shares expertise on boosting experimentation velocity, designing impactful experiments, and building knowledge repos. Topics include choosing metrics, team structures, reducing noise with CUPED, future of A/B testing, and scaling experimentation programs.
AI Snips
Chapters
Transcript
Episode notes
Why Most Experimentations Fail
- Most online experiments fail because humans are bad at predicting what will work.
- Experimentation helps check biases but success depends on tailored context and user base differences.
Boost Experiment Velocity Early
- Align stakeholders early and provide experiment planning support to speed experimentation.
- Creating a community of learning removes decision paralysis and accelerates test launches.
Choose Metrics Wisely
- Choose one or two primary metrics to avoid multiple hypothesis problems that reduce power.
- Avoid using revenue as a primary metric if it's far downstream and noisy; pick metrics closer to the feature impact.