#038 AI-Powered Search, Context Is King, But Your RAG System Ignores Two-Thirds of It

Today, I (Nicolay Gerold) sit down with Trey Grainger, author of the book AI-Powered Search. We discuss the different techniques for search and recommendations and how to combine them.

While RAG (Retrieval-Augmented Generation) has become a buzzword in AI, Trey argues that the current understanding of "RAG" is overly simplified – it's actually a bidirectional process he calls "GARRAG," where retrieval and generation continuously enhance each other.

Trey uses a three context framework for search architecture:

• Content Context: Traditional document understanding and retrieval
• User Context: Behavioral signals driving personalization and recommendations
• Domain Context: Knowledge graphs and semantic understanding

Trey shares insights on:

• Why collecting and properly using user behavior signals is crucial yet often overlooked
• How to implement "light touch" personalization without trapping users in filter bubbles
• The evolution from simple vector similarity to sophisticated late interaction models
• Why treating search as a non-linear pipeline with feedback loops leads to better results

For engineers building search systems, Trey offers practical advice on choosing the right tools and techniques, from traditional search engines like Solr and Elasticsearch to modern approaches like ColBERT.

Also how to layer different techniques to make search tunable and debuggable.

Quotes:

• "I think of whether it's search or generative AI, I think of all of these systems as nonlinear pipelines."
• "The reason we use retrieval when we're working with generative AI is because A generative AI model these LLMs will take your query, your request, whatever you're asking for. They will then try to interpret them and without access to up to date information, without access to correct information, they will generate a response from their highly compressed understanding of the world. And so we use retrieval to augment them with information."
• "I think the misconception is that, oh, hey, for RAG I can just, plug in a vector database and a couple of libraries and, a day or two later everything's magically working and I'm off to solve the next problem. Because search and information retrieval is one of those problems that you never really solve. You get it, good enough and quit, or you find so much value in it, you just continue investing to constantly make it better."
• "To me, they're, search and recommendations are fundamentally the same problem. They're just using different contexts."
• "Anytime you're building a search system, whether it's traditional search, whether it's RAG for generative AI, you need to have all three of those contexts in order to effectively get the most relevant results to solve solve the problem."
• "There's no better way to make your users really angry with you than to stick them in a bucket and get them stuck in that bucket, which is not their actual intent."

Trey Grainger:

• LinkedIn
• AI Powered Search (Community)
• AI Powered Search (Book)

Nicolay Gerold:

• ⁠LinkedIn⁠
• ⁠X (Twitter)

00:00 Introduction to Search Challenges 00:50 Layered Approach to Ranking 01:00 Personalization and Signal Boosting 02:25 Broader Principles in Software Engineering 02:51 Interview with Trey Greinger 03:32 Understanding RAG and Retrieval 04:35 Nonlinear Pipelines in Search 06:01 Generative AI and Retrieval 08:10 Search Renaissance and AI 10:27 Misconceptions in AI-Powered Search 18:12 Search vs. Recommendation Systems 22:26 Three Buckets of Relevance 38:19 Traditional Learning to Rank 39:11 Semantic Relevance and User Behavior 39:53 Layered Ranking Algorithms 41:40 Personalization in Search 43:44 Technological Setup for Query Understanding 48:21 Personalization and User Behavior Vectors 52:10 Choosing the Right Search Engine 56:35 Future of AI-Powered Search 01:00:48 Building Effective Search Applications 01:06:50 Three Critical Context Frameworks 01:12:08 Modern Search Systems and Contextual Understanding 01:13:37 Conclusion and Recommendations