How AI Is Built  cover image

How AI Is Built

RAG is two things. Prompt Engineering and Search. Keep it Separate | S2 E28

Mar 6, 2025
In this discussion, John Berryman, an expert who transitioned from aerospace engineering to search and machine learning, explores the dual nature of retrieval-augmented generation (RAG). He emphasizes separating search from prompt engineering for optimal performance. Berryman shares insights on effective prompting strategies using familiar structures, testing human evaluations, and managing token limits. He dives into the differences between chat and completion models and highlights practical techniques for tackling AI applications and workflows. It's a deep dive into enhancing interactions with AI!
01:02:44

Episode guests

Podcast summary created with Snipd AI

Quick takeaways

  • RAG encompasses both retrieval and generation, and treating them as separate elements enhances the optimization of each process.
  • Effective prompt engineering requires using familiar structures and correct formatting to align with the LLM's training data for improved model responses.

Deep dives

Separation of Retrieval and Generation

It's essential to recognize that retrieval and generation are two distinct components of information processing. By treating them separately, one can optimize the retrieval process before focusing on how to present information to the model. Prioritizing retrieval enhances the system's performance, enabling better context selection for subsequent model interaction. This separation also assists in diagnosing issues when they arise, providing clarity on which part of the process needs attention.

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode