Connor Shorten - Research Scientist, Weaviate - ChatGPT, LLMs, Form vs Meaning
Vector Podcast
00:00
The Inverted Index and the Hybrid Search in Bn25
Sorting the inverted index to prioritize documents maybe by clicks would be like the most sensible thing if it's like web pages so to say and you sort the documents. You could probably calculate how much time you have to search and and how much that lets you go into the invert indexYeah super interesting i i think it's very interesting for weevia with the with the hybrid search in the bn25 index because of the cardinality of how many terms you like with the document. The just the size of it inspired me when I was thinking about compression bottleneck from sparse to dense.
Transcript
Play full episode
