"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

RAG & Beyond: Semantic Storage and Retrieval

117 snips

Oct 24, 2023

Anton Troynikov, Cofounder of Chroma, shares insights on the urgency of maintaining the retrieval-augmented generation (RAG) loop internally, framing Chroma's current phase as 'wartime.' He discusses the burgeoning need for semantic storage and retrieval technologies, especially for handling data that has never been stored in databases before. Troynikov emphasizes innovative strategies for optimizing semantic retrieval and the importance of adapting to the fast-evolving AI landscape, making complex data management accessible for all.

Ask episode

AI Snips

Chapters

Transcript

Episode notes

ADVICE

Measuring RAG Performance

Measure RAG application performance with offline evals and human feedback.
Prioritize returning all relevant information while minimizing irrelevant data.

INSIGHT

Vector Database Differentiation

Vector database differentiation is unclear early in the adoption curve. Chroma differentiates by enabling easy experimentation and horizontal scaling.

ADVICE

Easy RAG Experimentation

Start RAG experiments with easy tools like ChromaDB. Leverage its simple setup and sensible defaults for quick demos.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

Anton Troynikov, cofounder of Chroma, joins Nathan Labenz to discuss the importance of keeping the retrieval-augmented generation (RAG) loop in house, what it means for Chroma to be in “wartime” mode right now, and how much of the data going into Chroma has never been in a database before. If you need an ERP platform, check out our sponsor NetSuite: http://netsuite.com/cognitive.

SPONSORS: NetSuite | Omneky

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

LINKS:

Part 1 with Anton: https://youtu.be/ogy37CdIljg

X/SOCIAL:

@labenz (Nathan)

@atroyn (Anton)

@eriktorenberg (Erik)

@CogRev_Podcast

TIMESTAMPS:

(00:00:00) - Introduction by Nathan, setting up the conversation with Anton

(00:02:16) - Anton articulates Chroma's mission to build a horizontally scalable system

(00:03:06) - Rise in popularity of retrieval-augmented generation (RAG)

(00:06:03) - Chroma's focus on delivering a horizontally scalable cloud service for vector search and storage

(00:08:07) - Nathan describes his experience building a RAG application for a client profiling use case

(00:10:27) - Anton advises measuring retrieval quality and maximizing relevant information returned

(00:15:05) - Sponsors: Netsuite | Omneky

(00:17:02) - Popular use of open source vs. proprietary embedding models like Anthropic's Ada

(00:19:30) - The importance of keeping the RAG loop in house and not relying solely on external APIs

(00:27:41) - The huge amount of unstructured data that can now be processed by AI

(00:30:40) - Providing a unified interface to structured and unstructured data

(00:33:15) - Much of the data going into Chroma has never been in a database before

(00:38:47) - Categories of organizations adapting to AI: legacy, AI-native, and AI-first

(00:40:55) - Where Chroma is seeing most of its growth right now

(00:46:20) - Interpretability work like Anthropic's circuit evaluation

(00:52:23) - Anton believes new tooling can make latent spaces accessible without AI expertise

(01:06:08) - Scaling constraints between search indexes vs. application databases

(01:09:10) - Potential for time as a dimension in embedding spaces

(01:13:46) - Likelihood of missing results due to representational issues vs. approximate nearest neighbor

(01:15:22) - Automatically handling small data sets without needing elaborate indexing

(01:17:20) - Anton's perspective on whether OpenAI will build its own database

(01:19:43) - Partnering with OpenAI and other labs to increase use of their models

(01:21:19) - Anton's experiments probing GPT's reasoning abilities with Game of Life

(01:25:41) - Closing thoughts on the conversation

This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at erik@turpentine.co.