Navigating AI Memory and Retrieval Systems

Jeff Huber of Chroma joins us to talk about what actually matters in vector databases in 2025, why “modern search for AI” is different, and how to ship systems that don’t rot as context grows.

Full show notes: https://www.latent.space/p/chroma

00:00 Introductions
00:48 Why Build Chroma
02:55 Information Retrieval vs. Search
04:29 Staying Focused in a Competitive AI Market
08:08 Building Chroma Cloud
12:15 Context Engineering and the Problems with RAG
16:11 Context Rot
21:49 Prioritizing Context Quality
27:02 Code Indexing and Retrieval Strategies
32:04 Chunk Rewriting and Query Optimization for Code
34:07 Transformer Architecture Evolution and Retrieval Systems
38:06 Memory as a Benefit of Context Engineering
40:13 Structuring AI Memory and Offline Compaction
45:46 Lessons from Previous Startups and Building with Purpose
47:32 Religion and Values in Silicon Valley
50:18 Company Culture, Design, and Brand Consistency
52:36 Hiring at Chroma: Designers, Researchers, and Engineers