Jeff Huber is the CEO of Chroma, working on context engineering and building reliable retrieval infrastructure for AI systems.
Context Engineering, Context Rot, & Agentic Search with the CEO of Chroma, Jeff Huber // MLOps Podcast #348.
Join the Community:
https://go.mlops.community/YTJoinIn
Get the newsletter: https://go.mlops.community/YTNewsletter
// Abstract
Jeff Huber drops some hard truths about “context rot” — the slow decay of AI memory that’s quietly breaking your favorite models. From retrieval chaos to the hidden limits of context windows, he and Demetrios Brinkmann unpack why most AI systems forget what matters and how Chroma is rethinking the entire retrieval stack. It’s a bold look at whether smarter AI means cleaner context — or just better ways to hide the mess.
// Bio
Jeff Huber is the CEO and cofounder of Chroma. Chroma has raised $20M from top investors in Silicon Valley and builds modern search infrastructure for AI.
// Related Links
Website: https://www.trychroma.com/
~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~
Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore
Join our Slack community [https://go.mlops.community/slack]
Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)]
Sign up for the next meetup: [https://go.mlops.community/register]
MLOps Swag/Merch: [https://shop.mlops.community/]
Connect with Demetrios on LinkedIn: /dpbrinkm
Connect with Jeff on LinkedIn: /jeffchuber/
Timestamps:
[00:00] AI intelligence context clarity
[00:37] Context rot explanation
[03:02] Benchmarking context windows
[05:09] Breaking down search eras
[10:50] Agent task memory issues
[17:21] Semantic search limitations
[22:54] Context hygiene in AI
[30:15] Chroma on-device functionality
[38:23] Vision for precision systems
[43:07] ML model deployment challenges
[44:17] Wrap up
