Jason has been worried that too many people take A/B Testing too seriously, or at least take it seriously without understanding what it can do for your business. He is worried that not enough people know when to use AB Testing and when you should do something else.
In this talk, he makes the case for optimising for the big wins in any start up, before you start to dig too deep into data:
“When you’re a small company you need to seek very large outcomes from any optimization because any subtle things like a 10% increase or whatever you literally can’t measure them”.
Business of Software Podcast