Tool Use - AI Conversations The Hard Truth About RAG in Production (ft Kirk Marple)
11 snips
Feb 4, 2025 
Kirk Marple, Founder and CEO of Graphlit, shares his expertise on building production-ready RAG systems. He dives into the significance of knowledge graphs and effective reranking strategies. Kirk emphasizes the importance of scaling RAG beyond basic implementations and reveals the critical differences between demo projects and production systems in AI. The conversation highlights challenges and best practices for developers, offering valuable insights on enhancing language models through sophisticated data retrieval techniques.
AI Snips
Chapters
Transcript
Episode notes
RAG Explained
- Retrieval Augmented Generation (RAG) enhances LLMs by incorporating external data into prompts.
- This allows LLMs to access information beyond their training data, enabling more accurate and context-aware responses.
Chunking Strategies
- Don't overthink chunk size; defaults with overlap work well.
- Focus on retrieval expansion and re-ranking for relevance.
Metadata and Embeddings
- Metadata in vector embeddings can create false positives.
- A hybrid approach using separate JSON metadata stores is more effective.