LLM Rewards & RLHF for Ranking

The best way to build a horrible search product? Don’t ever measure anything against what a user wants.

Search veterans Doug Turnbull (Led Search at Reddit + Shopify; Wrote Relevant Search + AI Powered Search) and John Berryman (Early Engineer on Github Copilot; Author of Prompt Engineering for LLMs), join Hugo to talk about how to build Agentic Search Applications.

We Discuss:

* The evolution of information retrieval as it moves from traditional keyword search toward “agentic search“ and what this means for builders.

* John’s five-level maturity model (you can prototype today!) for AI adoption, moving from Trad Search to conversational AI to asynchronous research assistants that reason about result quality.

* The Agentic Search Builders Playbook, including why and how you should “hand-roll” your own agentic loops to maintain control;

* The importance of “revealed preferences” that LLM-judges often miss (evaluations must use real clickstream data to capture “revealed preferences” that semantic relevance alone cannot infer)

* Patterns and Anti-Patterns for Agentic Search Applications

* Learning and teaching Search in the Age of Agents

You can find the full episode on Spotify, Apple Podcasts, and YouTube.

You can also interact directly with the transcript here in NotebookLM: If you do so, let us know anything you find in the comments!

👉 Want to learn more about Building AI-Powered Software? Check out our Building AI Applications course. It’s a live cohort with hands on exercises and office hours. Here is a discount code for readers. 👈

Doug and Hugo are also doing a free lightning lesson on Feb 20 about How To Build Your First Agentic Search Application! You’ll walk away with a framework & code to build your first agentic search app. Register here to join live or get the recording after.

Links and Resources

Guests

* Arcturus Labs (John’s website)

* Software Doug (Doug’s website)

* John Berryman on LinkedIn

* Doug Turnbull on LinkedIn

Books

* Relevant Search by Doug Turnbull (Manning)

* AI-Powered Search by Doug Turnbull (Manning)

* Prompt Engineering for LLMs by John Berryman (O’Reilly)

Blog Posts

* Incremental AI Adoption for E-commerce by John Berryman

* Roaming RAG – RAG without the Vector Database by John Berryman

* Agents Turn Simple Keyword Search into Compelling Search Experiences by Doug Turnbull

* A Simple Agentic Loop with Just Python Functions by Doug Turnbull

* Agentic Code Generation to Optimize a Search Reranker by Doug Turnbull

* LLM Judges Aren’t the Shortcut You Think by Doug Turnbul (Hugo’s 5 minute video below)

* Malleable Software by Ink & Switch (inc. Geoffrey Lit)

* Patterns and Anti-Patterns for Building with AI by Hugo Bowne-Anderson

Other Resources

* The Rise of Agentic Search, a recent VG Podcast with Jeff Huber

* Karpathy on Cognitive Core LLMs

* Cheat at Search with Agents course by Doug Turnbull (use code: vanishinggradients for $200 off)

* Upcoming Events on Luma

* Vanishing Gradients on YouTube

* Watch the podcast video on YouTube

* Join the final cohort of our Building AI Applications course in Q1, 2026 (25% off for listeners)

Timestamps (for YouTube livestream)

00:00 How to Build Agentic Search & Retrieval Systems02:48 Defining Search and AI03:26 Evolution of Search Technologies08:46 Search in E-commerce and Other Domains12:15 Combining Search and AI: RAG and LLMs23:50 User Intent and Search Optimization29:47 Levels of AI Integration in Search32:25 Exploring the Complexity of Search in Various Domains33:49 The Evolution and Impact of Agentic Search34:07 Defining Terms: RAG and Agentic Search34:52 The Research Loop and Tool Interaction35:55 Formal Protocols and Structured Outputs38:39 Building Agentic Search Experiences: Tips and Advice41:50 The Importance of Empathy in AI and Search Development54:30 The Role of UX in Search Applications01:01:15 Future of Search: Malleable User Interfaces01:02:38 Exploring Malleable Software01:04:20 The Coordination Challenge in Software Development01:05:23 The Impact of Claude Code & Claude Cowork01:06:22 The Future of Knowledge Work with AI01:12:39 Evaluating Search Algorithms with AI01:15:15 The Role of Agents in Search Optimization01:29:55 Teaching AI and Search Techniques01:34:25 Final Thoughts and Farewell

👉 Want to learn more about Building AI-Powered Software? Check out our Building AI Applications course. It’s a live cohort with hands on exercises and office hours. Here is a discount code for readers. 👈

https://maven.com/hugo-stefan/building-ai-apps-ds-and-swe-from-first-principles?promoCode=vgpod

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit hugobowne.substack.com