Converting Taste Into Evaluators for Non‑Determinism

What does it take to build products that feel thoughtful, emotionally resonant, and unmistakably high-quality, especially inside a company that ships fast and experiments constantly?

In this episode of Supra Insider, Marc and Ben sit down with Nickey Skarstad, Director of Product at Duolingo, to unpack how one of the world’s most beloved consumer apps maintains its bar for craft, clarity, and delight while operating at massive scale. Nickey shares how Duolingo operationalizes quality across teams, how they dogfood relentlessly, and why “unreasonable hospitality” applies just as much to software as it does to service.

She also shares the internal rituals Duolingo uses - like their “hot trash” Friday forum for sharing early experiments, and how these lightweight mechanisms help PMs, designers, and engineers learn from each other. Nickey walks through Duolingo’s product review process, how senior leaders give feedback, and how the company ensures teams move quickly without shipping work that falls below their quality standards.

Whether you’re a PM trying to strengthen your intuition, a design-driven leader aiming for higher quality bars, or a product builder exploring how AI should (and shouldn’t) fit into your workflow, this episode is full of concrete lessons you can use immediately.

All episodes of the podcast are also available on Spotify, Apple and YouTube.

New to the pod? Subscribe below to get the next episode in your inbox 👇

Interviewing at Uber, Figma, Stripe, or DoorDash?

If you’re a PM preparing for interviews at Uber, Figma, Stripe, and DoorDash… we think you’re going to love Insider Loops. You can read about it in Ben’s LI post, Marc’s LI post or check out the site directly below.

In this episode, we covered the following topics:

* (01:35) How Nickey keeps up with AI: newsletters, podcasts, internal Slack channels, and intentional learning time

* (07:15) Internal rituals for sharing AI learnings: Slack posts, demos, and Duolingo’s “hot trash” culture

* (11:12) Duolingo’s product review process as a mechanism for scaling quality and judgment

* (14:12) How Duolingo balances speed (“clock speed”) with rigorous product reviews

* (23:31) What makes feedback actually useful—and when it should block shipping

* (29:05) The importance of heavy dogfooding for generative AI experiences

* (32:22) Turning product taste into evaluators for non-deterministic systems

* (37:50) Why slapping “AI” into marketing often hurts more than it helps

* (41:52) How to intentionally develop better product taste

* (51:28) Why AI has re-energized builders and product teams

* (55:01) Why hobbies, fiction, and being “in the world” matter for product taste

* (58:40) Nickey’s closing advice: investing in taste is never wasted time

* And more!

Links:

* Nickey’s Newsletter: https://nickey.substack.com/

* Nickey Skarstad: https://www.linkedin.com/in/nickeyskarstad/

* Marc: https://www.linkedin.com/in/marcbaselga/

* Ben: https://www.linkedin.com/in/benerez/

Resources Mentioned:

* Duolingo’s Product Review Process: https://www.linkedin.com/pulse/how-product-review-works-sneak-peek-key-decision-making-process-lnpoe/

* Peter Yang Podcast: https://creatoreconomy.so/

* Lenny’s Newsletter: https://www.lennysnewsletter.com/

* Book Suggestions:

* The Thursday Murder Club - recommended by Nickey

* Hyperion - recommended by Ben

If you’re a new subscriber, we encourage you to check out some of the recent episodes you might have missed:

To support the podcast, please check out the links below:

* Supra has teamed up with Maven to bring you something special – courses that our own community members have personally curated. And because you’re part of the Supra family, you can get up to 35% in some of these handpicked selections with code SUPRAxMAVEN.

* Preparing for Meta-style Product Sense & Analytical Thinking interviews? Check out Ben’s AI Practice Copilot for Meta-Style PM Interviews and top-rated Maven course: Product Sense & Analytical Thinking Bootcamp for Meta-Style PM Interviews (selected by Lenny as a top course in product 🔥). Get 25% off the course with promo code `suprainsider` (auto-applied here).

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit suprainsider.substack.com