Scrum Master Toolbox Podcast: Agile storytelling from the trenches

How A/B Testing Can Derail Product Development, A Product Leadership Story | Eli Goodman

Oct 1, 2024

Eli Goodman, the Head of Product at Tuple with 17 years of software development experience, dives into the pitfalls of over-relying on A/B testing. He reveals how this dependence can stifle creativity and slow down progress in product development. Eli shares strategies for product teams to reclaim their decision-making by blending data insights with intuition. He also highlights the influence of 'The Idea Factory' on his perspective as a product leader, emphasizing the need for creativity in a fast-paced tech environment.

11:58

Creator website

Episode guests

Eli Goodman

AI Summary

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

Over-reliance on A/B testing can inhibit progress and creativity within product teams, leading to decision-making delays and frustration.

Embracing individual insights and alternative de-risking methods, such as prototyping, can enhance agility and innovation in product development.

Deep dives

The Value of Empirical Testing

A fixation on A/B testing has emerged within many product organizations, driven by the pursuit of empirical data to validate decisions. Initially, the excitement surrounding the implementation of A/B tests often stems from their ability to provide concrete insights into user interaction and business impacts. However, this enthusiasm can lead to significant costs and resource allocation that may not align with the organization's scale or needs. As teams grow overly focused on A/B test results, they may lose sight of other important factors influencing product development and decision-making.

Intro

3min

The Overdependence on A-B Testing in Product Development

5min

The Dangers of Relying Solely on A/B Testing in Product Development

4min

Eli Goodman: How A/B Testing Can Derail Product Development, A Product Leadership Story

NOTE: We want to thank the folks at Tuple.app for being so generous with their stories, and supporting the podcast. Visit tuple.app/scrum and share them if you find the app useful! Remember, sharing is caring!

Eli Goodman, Head of Product at Tuple, discusses a recurring anti-pattern in product development: the over-reliance on A/B testing. Reflecting on his experiences at two different companies, Eli illustrates how A/B testing, when misused, can slow down product progress and lead to a bloated team structure. He also shares strategies on how to avoid this trap and take responsibility for product decisions. How can product teams avoid hiding behind A/B testing and instead move forward with confidence? Listen in to find out!

Featured Book of the Week: The Idea Factory by Jon Gertner

In this episode, Eli Goodman, Head of Product at Tuple, shares the profound influence of The Idea Factory, a book about Bell Labs, on his career as a product manager. Eli delves into how the book’s lessons on creativity, innovation, and the long-term impact of foundational ideas have shaped his thinking. What can today’s product leaders learn from the story of Bell Labs? How does creativity fuel product success, even in today’s fast-paced tech world? Listen in to find out.

About Eli Goodman

Eli Goodman has been working on software teams for 17 years. He’s been a full-stack developer and engineering manager at both large and small companies, including Etsy and Headspace. A few years ago, Eli transitioned to product management and is now the Head of Product at Tuple, a remote pair programming service used by companies such as Figma, Shopify, and many others in the software industry.

You can link with Eli Goodman on LinkedIn, or email Eli at Eli@Tuple.app.

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more