#020 The Evolution of Search, Finding Search Signals, GenAI Augmented Retrieval

23 snips

Sep 5, 2024

Doug Turnbull, a search engineer at Reddit and author of "Relevant Search," dives into the transformation of search from keyword basics to advanced methods like semantic search and LLMs. He highlights the ongoing challenges of defining relevance based on user intent and context. Doug also discusses the importance of integrating various search techniques for better results, emphasizing the role of operational concerns in shaping search technology. With insights on the resurgence of underappreciated methods like LambdaMART, he shares how understanding user perspectives can significantly enhance search performance.

Ask episode

AI Snips

Chapters

Transcript

Episode notes

INSIGHT

Relevance Is User-Defined

Relevance has no single definition; it depends on user intent and context at scale.
Doug Turnbull stresses using user engagement and labels to discover what relevance means for your product.

INSIGHT

Query-Document Relationship Is Core

The core of good search is modeling the relationship between query and document.
Doug calls query understanding plus content understanding the secret sauce of search quality.

ADVICE

Tier Retrieval For Precision And Safety

Use tiered retrieval: cheap, high-recall first passes and stronger high-confidence tiers afterwards.
Filter known-bad items early and boost exact high-confidence matches like title-exact results.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

In this episode of How AI is Built, Nicolay Gerold interviews Doug Turnbull, a search engineer at Reddit and author on “Relevant Search”. They discuss how methods and technologies, including large language models (LLMs) and semantic search, contribute to relevant search results.

Key Highlights:

Defining relevance is challenging and depends heavily on user intent and context
Combining multiple search techniques (keyword, semantic, etc.) in tiers can improve results
LLMs are emerging as a powerful tool for augmenting traditional search approaches
Operational concerns often drive architectural decisions in large-scale search systems
Underappreciated techniques like LambdaMART may see a resurgence

Key Quotes:

"There's not like a perfect measure or definition of what a relevant search result is for a given application. There are a lot of really good proxies, and a lot of really good like things, but you can't just like blindly follow the one objective, if you want to build a good search product." - Doug Turnbull

"I think 10 years ago, what people would do is they would just put everything in Solr, Elasticsearch or whatever, and they would make the query to Elasticsearch pretty complicated to rank what they wanted... What I see people doing more and more these days is that they'll use each retrieval source as like an independent piece of infrastructure." - Doug Turnbull on the evolution of search architecture

"Honestly, I feel like that's a very practical and underappreciated thing. People talk about RAG and I talk, I call this GAR - generative AI augmented retrieval, so you're making search smarter with generative AI." - Doug Turnbull on using LLMs to enhance search

"LambdaMART and gradient boosted decision trees are really powerful, especially for when you're expressing your re-ranking as some kind of structured learning problem... I feel like we'll see that and like you're seeing papers now where people are like finding new ways of making BM25 better." - Doug Turnbull on underappreciated techniques

Doug Turnbull

Nicolay Gerold:

Chapters

00:00 Introduction and Guest Introduction 00:52 Understanding Relevant Search Results 01:18 Search Behavior on Social Media 02:14 Challenges in Defining Relevance 05:12 Query Understanding and Ranking Signals 10:57 Evolution of Search Technologies 15:15 Combining Search Techniques 21:49 Leveraging LLMs and Embeddings 25:49 Operational Considerations in Search Systems 39:09 Concluding Thoughts and Future Directions