AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
3min chapter
Jerry Liu - Building LlamaIndex, the Data Framework for LLMs
The MaML Podcast - Medicine & Machine Learning
How LOM Index Decouples State From Compute
The new version of LOM index recently came out congratulations. And I saw there's three key abstractions that more cleanly decoupled state from compute. The indexes are the ways you can like chew up the data and process it in a way that you can use it with a language model. And then the second is the retriever, which here says here fetches the most relevant nodes from an index given a query. And the final one is a query engine that synthesizes response given the query and retrieve nodes might be questioned.
00:00
Transcript
Episode notes
Speaker 2
Yeah, and one of my big questions, I saw on your documentation, you said, there's this new the new version of LOM index recently came out congratulations. And I saw there's three key abstractions, you said we've redesigned LOM index round three key abstractions that more cleanly decoupled state from compute. The first being the index, which I guess that it says here manages the state abstracting way underlying storage and exposing a view over processed data and associated metadata. Is that kind of like the chewed up part we're talking about?
Speaker 1
Yeah, exactly. And so if you think about like the way we think about LOM index, it's you start with like the data, which is like state and part of LOM index is being able to store state. And so there's the raw data itself. And then there's these like indexes on top of the data.
Speaker 2
So that's the first extraction, right? Exactly.
Speaker 1
Yeah. So the indexes are the ways you can like chew up the data and process it in a way that you can use it with a
Speaker 2
language model. And then the second is the retriever, which here says here fetches the most relevant nodes from an index given a query. And the final one is a query engine that synthesizes response given the query and retrieve nodes might be questioned, I feel like maybe this is a secret sauce here, correct me if I'm wrong, but how does the retriever know which nodes to fetch given your question? Yeah.
Speaker 1
So the just as a very basic level, we have this concept of like a retriever, which is just like an abstract class and Python. And the interface that implements is you feed in some query or like a natural language query. And the goal of the retriever is to give you back the relevant nodes from your knowledge corpus. And again, the relevant nodes are just like the relevant documents from your knowledge corpus. And so, you know, we just talked about this, but just a very basic example of what a retriever could consist of is really just using that like embedding based lookup approach that described. So for instance, like getting an embedding of the query, using it to look up the relevant documents from like a vector database and fetching the most relevant documents. I think that's one example. Oh, using like the Euclidean, like which like the minimal distance between the. Yeah, like a similarity Euclidean distance between the embedding of the query with the embedding of the. Okay,
Speaker 2
that makes sense. That's for a true. That's relevant
Speaker 1
documents. And then, you know, there's like other types of retrievers that you could do too, because this interface is so general, there's like a bunch of other retrievers that we support. And happy to talk about that as well. You can also do like retrievers on top of other retrievers. You could like arbitrarily find your own custom retrievers, but that's. Yeah. I think in the framework of like this whole paradigm of like, how do you best like manage the interaction of like a language model with your data, the retrievers a critical piece of this, because yeah, really what's happening when you. Have this interaction is you have like a retriever model that's for treating relevant documents from your data and then working with the language model to give you back the results.
Get the Snipd
podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered
AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any
moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Save any
moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share
& Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
Share
& Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode