Lenny's Podcast: Product | Growth | Career

The UX research reckoning is here | Judd Antin (Airbnb, Meta)

Jan 4, 2024

Judd Antin, a veteran in UX research with a rich background at Yahoo, Meta, and Airbnb, tackles the pressing challenges in the field. He critiques past user research trends and argues for three distinct types of research—macro, middle-range, and micro. Antin emphasizes the need for researchers to be integral to product development and critiques the effectiveness of the Net Promoter Score, advocating for better customer satisfaction metrics instead. His insights shed light on how to genuinely align research with business objectives for meaningful improvements.

01:14:34

Creator website

Episode guests

Judd Antin

AI Summary

Highlights

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

User-centered research demands diverse skills and deeper understanding of business and product metrics to drive better outcomes.

Researchers and product managers should collaborate closely to integrate research fully into the product process and leverage the expertise of researchers for informed decision-making.

Prioritizing micro and macro research types, such as usability testing and strategic product planning, respectively, can lead to more meaningful outcomes for the business.

Deep dives

The need for a new version of user-centered research

User-centered research is undergoing an evolution, and researchers need to adapt to the changing landscape. This evolution involves developing diverse research skills and deepening their understanding of the business and product metrics. Additionally, researchers should focus on building strong relationships with product managers and designers to ensure integration from the beginning to the end of the product process. By doing so, they can have a larger impact and drive better outcomes for the business.

User Centered Performance in Product Development

02:17

UX researchers aren’t integrated enough to make impact

00:36

The Flaws of NPS as a Marketing Metric

02:45

The Importance of Analytic Thinking and Diverse Perspectives in Decision-Making

01:28

Intro

5min

Reckoning in User Research

9min

Integrating UX Research in Product Development

18min

Collaboration in User Research

5min

The Essential Role of Usability Research in Product Development

4min

Balancing A-B Testing and User Research

23min

Rethinking NPS: The Case for Customer Satisfaction Metrics

3min

Navigating Product Management and Personal Insights

8min

Judd Antin has spent 15 years leading research and design teams at companies like Yahoo, Meta, and Airbnb. His direct reports have gone on to lead user research at Figma, Notion, Slack, Robinhood, Duolingo, AllTrails, and more. In our conversation, we unpack the transformation that the user-research field is experiencing. Specifically:

• Where user research went wrong over the past decade

• The three types of research—macro, middle-range, and micro—and the purpose of each

• How to effectively integrate researchers into the product development process

• The “user-centered performance” phenomenon and why it’s a waste of time

• Common tropes about PMs, from researchers

• The ideal ratio of researchers in a company

• Why Judd says NPS is useless, and what to use instead

—

Brought to you by Teal—Your personal career growth platform | Vanta—Automate compliance. Simplify security | Ahrefs—Improve your website’s SEO for free

—

Find the full transcript at: https://www.lennysnewsletter.com/p/the-ux-research-reckoning-is-here

—

Where to find Judd Antin:

• LinkedIn: https://www.linkedin.com/in/juddantin/

• Website: https://juddantin.com/

• Blog: https://medium.com/onebigthought

—

Where to find Lenny:

• Newsletter: https://www.lennysnewsletter.com

• X: https://twitter.com/lennysan

• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/

—

In this episode, we cover:

(00:00) Judd’s background

(04:16) Critiques and responses to Judd’s post “The UX Research Reckoning Is Here”

(07:33) The state of user research

(08:53) Macro, middle-range, and micro research

(14:05) What teams get wrong when it comes to research

(15:46) The importance of integrating research from the beginning

(17:30) Traits of great researchers

(19:53) Advice for evaluating user researchers

(21:10) Balancing business and product focus

(23:55) User-centered performance

(26:42) The role of intuition in product development

(30:15) Checking your gut instincts

(32:54) Common tropes about PMs, from researchers

(41:02) A/B testing vs. user research

(43:15) Hindsight bias and narrative fallacy

(44:55) Making recommendations based on research

(47:26) Advice for teams on how to leverage researchers

(51:18) How product managers can be better partners to user researchers

(56:53) The ideal ratio of researchers in a company

(59:43) Empowering user researchers to drive impact

(01:03:39) The limitations of NPS as a metric

(01:06:48) The risks of dogfooding

(01:08:51) Lightning round

—

Referenced:

• Matt Gallivan on LinkedIn: https://www.linkedin.com/in/mattgallivan/

• Janna Bray on LinkedIn: https://www.linkedin.com/in/janna-bray-a4046a25/

• Celeste Ridlen on LinkedIn: https://www.linkedin.com/in/celesteridlen/

• Rebecca Gray on LinkedIn: https://www.linkedin.com/in/rebeccagray2/

• Hannah Pileggi on LinkedIn: https://www.linkedin.com/in/hannah-pileggi-43169314/

• Louise Beryl on LinkedIn: https://www.linkedin.com/in/louise-beryl-13225833/

• The UX Research Reckoning Is Here: https://medium.com/onebigthought/the-ux-research-reckoning-is-here-c63710ea4084

• The end of the “free money” era: https://www.theguardian.com/technology/2023/apr/11/techscape-zirp-tech-boom

• Cognitive biases: https://en.wikipedia.org/wiki/List_of_cognitive_biases

• IDEO design thinking: https://designthinking.ideo.com/

• Everything Is Obvious: How Common Sense Fails Us: https://www.amazon.com/Everything-Obvious-Common-Sense-Fails/dp/0307951790

• Patrick Collison’s tweet: https://twitter.com/patrickc/status/1443215022029619200?lang=en

• Brian Chesky on LinkedIn: https://www.linkedin.com/in/brianchesky/

• Brian Chesky on Lenny’s Podcast: https://www.lennyspodcast.com/brian-cheskys-new-playbook/

• NPS: https://en.wikipedia.org/wiki/Net_promoter_score

• What is CSAT and how do you measure it?: https://www.qualtrics.com/experience-management/customer/what-is-csat/

• Michael Murakami on LinkedIn: https://www.linkedin.com/in/michaelhmurakami/

• Bad Leadership: What It Is, How It Happens, Why It Matters: https://www.amazon.com/Bad-Leadership-Happens-Matters-Common/dp/1591391660

• Demon Copperhead: https://www.amazon.com/Demon-Copperhead-Novel-Barbara-Kingsolver/dp/0063251922

• All Systems Red: The Murderbot Diaries: https://www.amazon.com/All-Systems-Red-Murderbot-Diaries/dp/0765397536

• The Last of Us on HBO: https://www.hbo.com/the-last-of-us

• Belay glasses: https://www.amazon.com/Belay-Glasses-Climbing-Comfortable-Sturdy/dp/B08GSBYDKQ/

• Epictetus: Control What You Can—Especially Yourself: https://www.shortform.com/blog/epictetus-control/

• The 7 Habits of Highly Effective People: Powerful Lessons in Personal Change: https://www.amazon.com/Habits-Highly-Effective-People-Powerful/dp/0743269519

—

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.

—

Lenny may be an investor in the companies discussed.

This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.lennysnewsletter.com/subscribe

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.

Get the app

Lenny's Podcast: Product | Growth | Career

The UX research reckoning is here | Judd Antin (Airbnb, Meta)

Episode guests

Podcast summary created with Snipd AI

Quick takeaways

Deep dives

The need for a new version of user-centered research

Challenges between researchers and product managers

Reevaluating the value of different research types

Moving from validation to falsification

Importance of Integrated Insights Function

The Role of Researchers in Product Development

User Centered Performance in Product Development

UX researchers aren’t integrated enough to make impact

The Flaws of NPS as a Marketing Metric

The Importance of Analytic Thinking and Diverse Perspectives in Decision-Making

Remember Everything You Learn from Podcasts