
#254: Is Your Use of Benchmarks Above Average? with Eric Sandosham
The Analytics Power Hour
00:00
Rethinking Benchmarks in Experimentation
This chapter critiques the common 30% win rate benchmark used in experimentation, arguing for its irrelevance in enhancing organizational intelligence. The discussion highlights the need for more meaningful metrics and draws a connection between the limitations of benchmarks and the philosophical implications of generative AI in problem identification.
Transcript
Play full episode