Experimentation is not about right or wrong. It’s about learning things that you genuinely didn’t know. The secret is to become comfortable with the uncomfortable and to make room for your own sense of vulnerability, says Holly Hester-Reilly. When you’re able to embrace not knowing something, or have experiments come back that disprove your hypotheses, you’re going to discover amazing insights that benefit you, your team, and your organization.
In this episode of the Product Momentum Podcast, Sean and Paul welcome Holly Hester-Reilly, Founder and CEO of H2R Product Science. In this dynamic and fast-paced conversation, Holly discusses her approach to the product science method, one that focuses on using science and empathy to manage risk while building high-growth products and teams.
“Our job as product people is to manage the risk of product failure,” Holly says. “Part of that risk is to avoid looking bad in front of our teams, peers, and managers. We have to shift the mindset and the conversation away from right or wrong so that we can begin to pride ourselves on learning new things.”
Product leaders have an enormous role to play in encouraging experimentation, Holly adds. “The only way for us to make that mindset shift is for us to be the example by calling out when the people around us learn something new and saying, ‘that’s what we want to see more of!’”
Listen in to catch more from Holly Hester-Reilly:
[02:16] The product science method. It’s really about the difference between what people say they will do and what they actually do.
[02:58] Design experiments around past behaviors, not abstracts and hypotheticals.
[04:51] The role of data and metrics. The cool thing about software is we can actually measure how users behave. The right metrics …that’s the best possible predictor of future behavior.
[07:42] Why smart companies with reams of data still make flawed product launches. They’re too comfortable.
[08:25] The Emperor’s New Clothes. Do we have the willingness to be uncomfortable, to be the person who will stand up and say to the boss, “here are the reasons why your pet project is going to fail.”?
[10:12] Confirmation bias. Channeling Richard Feynman, “you must not fool yourself, and you are the easiest person to fool.”
[10:45] Rapid research. You have to be super-focused on the most important thing to learn and let go of the idea that you might not learn the other things.
[12:38] Exposure therapy. The more times that you’re exposed to something, the more comfortable you become with it.
[15:05] Optimism bias. Gets in the way of making good business decisions like so, so much.
[16:14] How long does it take to change somebody’s mind about their pet project?
[17:49] The role of experimentation. It’s not about being right. It’s about learning things we don’t already know.
[21:19] Premortem risk assessment. Put yourself in a place where risk is already assumed to be real.
[22:34] Our job as product people is to manage the risk of product failure.
[24:02] The difference between good and fantastic product research.
[25:46] Take a snapshot. Make sure that your team is situating who your customer is within the strategy of the product.
[27:22] Practicing discovery. As a product leader, you should have a strategy that is a series of product-market fits.
[28:27] Measuring the value of research. Two parts: quantify the value of research and know when you’ve done enough of it.
[32:10] “Faster horses.” At least you know what outcome your users want.
[33:48] Innovation. Innovation drives a significant change. It doesn’t just increase the amount of something you’re selling: the revenue, the number of users. It changes the rate of that.