On the Trail of Evidence-Guided Product Success | Itamar Gilad, Product Coach and Author
Oct 3, 2023
auto_awesome
Guest Itamar Gilad, author of 'Evidence-Guided: Creating High-Impact Products in the Face of Uncertainty,' discusses the importance of evidence-based decision-making in product management, introduces the AFTER model for evaluating and testing ideas, addresses misconceptions about OKRs, and shares insights on goal-setting and alignment. They also talk about missed opportunities with Google+ and the pitfalls of the planning waterfall approach.
Making evidence-based decisions in product management is crucial for success.
The AFTER model provides a comprehensive framework for evaluating and testing ideas effectively.
Outcome-based OKRs are more effective than output-based OKRs, and creating too many OKRs can hinder team commitment and trust.
Deep dives
Gmail's Reckoning with Inbox Overload
In the early 2010s, Gmail faced the challenge of users' inboxes being flooded with transactional emails, social notifications, promotions, and ads. Recognizing the need to make Gmail more engaging, the team sought ideas to automatically organize the inbox. This led to the development of the tabbed inbox, which categorizes emails into primary, social, promotions, and other tabs. The team conducted assessments, fact finding, tests, and experiments to validate the impact and feasibility of the tabbed inbox. The approach helped Gmail address the problem of inbox clutter and provided users with a more organized and engaging experience.
Evidence-Guided Product Management
The podcast episode features Ida Mar-Galad, author of the book 'Evidence Guided: Creating High Impact Products in the Face of Uncertainty.' The episode explores the concept of evidence-guided product management and highlights the importance of making decisions based on data and insights rather than opinions. It emphasizes the need to validate ideas through assessments, fact finding, tests, and experiments. The podcast also discusses the pitfalls of using output-driven goals and the benefits of setting outcome-driven goals. The conversation aims to expand listeners' understanding of evidence-guided product management and provides practical insights for avoiding common industry mistakes.
The Story of Google+ and Lessons Learned
The podcast episode delves into the story of Google's attempt to create its own version of Facebook called Google+. The company launched Google+ in response to the growing popularity of social networks and the fear that Facebook's advertising capabilities would threaten Google's dominance. Despite its initial success, Google+ eventually failed to gain traction and was shut down after nine years. The episode underscores the importance of evidence-guided decision-making, highlighting the need to validate assumptions, assess market demand, and test ideas before committing extensive resources to a project.
The Pitfalls of the Planning Waterfall
The podcast episode challenges the traditional planning waterfall approach commonly used in product management. It highlights the inherent uncertainty in the industry and the limitations of relying on long-term planning and consensus-based decision-making. The episode presents the concept of evidence-guided development as an alternative approach to product management, emphasizing the need to embrace uncertainty, continuously test ideas, and use evidence to guide decision-making. By adopting a more iterative and flexible mindset, product managers can navigate the complexities of the market and increase the likelihood of creating high-impact products.
The After Model: A Framework for Validation and Testing
The podcast episode introduces the after model, a framework for assessing, fact finding, testing, experimenting, and releasing ideas. It highlights the different stages of validation and testing that can be applied to idea development. Starting with quick assessments and fact finding, teams can gather insights, analyze data, and identify potential risks. This is followed by various testing methods, such as smoke tests, usability tests, and experiments. The after model promotes a systematic and evidence-based approach to product development, enabling teams to validate ideas and make data-informed decisions.
In this episode of Dreams with Deadlines, host Jenny Herald dives deep into the world of product management with guest Itamar Gilad, author of the newest book, "Evidence-Guided: Creating High-Impact Products in the Face of Uncertainty."
Key Things Discussed:
Evidence-Guided Product Management: Itamar emphasizes the importance of making evidence-based decisions in product management. He shares how to set clear goals and align them with company objectives, enabling teams to focus on what truly matters.
The AFTER Model: Itamar introduces the AFTER model—a comprehensive framework that covers Assessment, Fact-finding, Tests, Experiments, and Release Results. This model guides product managers in evaluating and testing ideas effectively, leading to better decision-making.
Common Mistakes with OKRs: Itamar addresses misconceptions about Objectives and Key Results (OKRs), emphasizing the need for outcome-based OKRs rather than focusing on output. He also warns against the common mistake of creating too many OKRs, which can hinder commitment and trust within teams.
Show Notes
[00:00:07] Introduction to "Evidence-Guided: Creating High Impact Products in the Face of Uncertainty. Jenny Herald introduces the newest book "Evidence-Guided: Creating High Impact Products in the Face of Uncertainty" by Itamar Gilad. Itamar shares the opening story of Google+ and why it was chosen to lead the book.
[00:03:43] The Pitfalls of Planning Waterfall in Product Development. Jenny and Itamar discuss the concept of the planning waterfall and why it's not an effective approach in product development.
[00:06:36] Understanding Evidence-Guided Development. Jenny and Itamar delve into the concept of evidence-guided development and explore why Itamar is so passionate about sharing this approach with the world.
[00:08:21] The GIST Model: A Framework for Evidence-Guided Development. Jenny and Itamar discuss the GIST model, a foundational framework in evidence-guided development, and its importance in helping organizations embrace change and innovation.
[00:11:22] From Idea to Reality: The Tabbed Inbox and Evidence-Based Development. Jenny and Itamar discuss the development of Gmail's tabbed inbox as an example of evidence-based development within the GIST model framework.
[00:18:10] Prioritizing Ideas with the Confidence Meter. Jenny and Itamar discuss the confidence level meter, a framework for prioritizing ideas based on evidence and confidence. Itamar explains how this tool helps product teams avoid investing in ideas with low confidence and guides them towards incremental development.
[00:24:55] Setting Evidence-Guided Goals. Jenny and Itamar discuss the importance of setting evidence-guided goals and how to choose the most important outcomes. Itamar explains why goals are crucial, regardless of whether you use OKRs or other frameworks, and emphasizes the need for both qualitative and quantitative aspects in goal setting.
[00:31:22] Balancing Local Optimization with a Bigger Picture. Jenny and Itamar delve into the challenge of avoiding local optimization while setting evidence-guided goals. They discuss the importance of aligning team goals with the company's overarching strategy and how metrics and impact assessments can play a role in achieving this alignment.
[00:37:03] Using the AFTER Model for Effective Idea Validation. Jenny and Itamar explore the AFTER model, a framework Itamar developed to help teams define steps for idea validation. The AFTER model stands for Assessment, Fact-Finding, Test Experiments, and Release Results. Itamar explains how each stage in the model contributes to the validation process, from initial idea assessment to controlled experiments and gradual product releases.
[00:43:58] Quick-Fire Questions for Itamar:
What is your dream with a deadline? Itamar's dream with a deadline is to help product people bring evidence-guided thinking into their product development process, ultimately leading to better products, higher value, and greater job satisfaction for everyone involved.
What do people get wrong about OKRs? People often make two common mistakes with OKRs: using them to capture a plan (output OKRs) instead of focusing on outcomes, and creating too many OKRs, which leads to lack of commitment and trust in the process.
When can we expect your book to be released? The target release date for Itamar's book is September 20th, with a possibility of it being released a few days later or by the end of September.
What would you say is the book that largely shaped how you think? Two books have significantly shaped Itamar's thinking: "INSPIRED" by Marty Cagan, which is highly influential in the product management field, and "The Lean Startup" by Eric Ries, which introduced the concept of evidence-guided development and the interplay between business and product development.
