The Reality of the Marketing Experimentation Is You Have to Throw Away Your Code

Matt: I'm not sure I agree on the part with the experimentation that you have to throw things away right so like let's say you have a good experimentation setup and I don't know how you guys are doing it but like we're shipping about 10 to 15 experiments per quarter and because of the success rate is just not that good right so like 30 to 35% success rate on the initial hypothesis you kind of get used to throwing code away. Matt: The reality for me is like most experiments I think we ran at credit karma anything worked so we thought a lot about like multivariate hypotheses most solutions found some lift in what they were doing. Even creating two or three different versions of a