Vector databases are purpose-built databases that efficiently manage, store, and update vectors at scale, offering better scalability, efficiency, and access to the latest technologies and algorithms.
Embedded databases, like LancerD and ChromaDB, provide an alternative to client-server architecture, improving data privacy and scalability, while the future of vector databases lies in combining them with graph databases for enhanced retrieval augmented generation models.
Deep dives
Vector databases: Efficiently managing and retrieving vectors at scale
Vector databases are purpose-built databases that efficiently manage, store, and update vectors at scale. These databases retrieve the most similar vectors to a given query, considering the semantics of the query. They are a natural evolution of the NoSQL class of databases and have become more accessible, allowing companies to build powerful search and information retrieval systems. With the combination of vector databases and large language models (LLMs), applications like querying data by natural language and retrieval augmented generation are becoming more achievable. The trade-off between using existing databases with vector capabilities and purpose-built vendors is an important consideration. Purpose-built vendors generally offer better scalability, efficiency, and access to the latest technologies and algorithms.
Embedded databases and the future of vector databases
Embedded databases, like LancerD and ChromaDB, are starting to gain traction in the vector database space. These databases offer an alternative to client-server architecture, moving towards an embedded approach for improved data privacy and scalability. Additionally, the future of vector databases lies in the combination of graph databases and vector databases. Vector databases can add new value to knowledge graphs by enabling factual knowledge retrieval and leveraging natural language query interfaces provided by LLMs. These enhanced retrieval augmented generation models could revolutionize data exploration and discovery in the context of complex and connected data sets.
Trade-offs in vector databases
Vector databases come with various trade-offs that developers should consider. These trade-offs include deciding between purpose-built vendors and existing databases with vector capabilities, choosing between building an external embedding pipeline or using a built-in hosting pipeline, and balancing indexing speed and querying speed. Other trade-offs involve considering the trade-off between recall and latency, choosing between in-memory or on-disk indexes, and evaluating the benefits of sparse versus dense vectors. Hybrid search, which combines full-text search with vector search, and the importance of pre-filtering versus post-filtering to enhance search quality also need to be taken into account.
There’s so much talk (and hype) these days about vector databases. We thought it would be timely and practical to have someone on the show that has been hands on with the various options and actually tried to build applications leveraging vector search. Prashanth Rao is a real practitioner that has spent and huge amount of time exploring the expanding set of vector database offerings. After introducing vector database and giving us a mental model of how they fit in with other datastores, Prashanth digs into the trade offs as related to indices, hosting options, embedding vs. query optimization, and more.
Changelog++ members save 3 minutes on this episode because they made the ads disappear. Join today!
Sponsors:
Fastly – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com
Fly.io – The home of Changelog.com — Deploy your apps and databases close to your users. In minutes you can run your Ruby, Go, Node, Deno, Python, or Elixir app (and databases!) all over the world. No ops required. Learn more at fly.io/changelog and check out the speedrun in their docs.
Typesense – Lightning fast, globally distributed Search-as-a-Service that runs in memory. You literally can’t get any faster!