Open source, on-disk vector search with LanceDB (Practical AI #250)
Dec 19, 2023
auto_awesome
Chang She, Co-founder and CEO of LanceDB, discusses their open source, on-disk, embedded vector search offering. They talk about the unique columnar database structure that enables serverless deployments and drastic savings without performance hits at scale. They also explore the potential applications and benefits of autonomous vehicles and edge computing technology, as well as exciting developments in the practical AI space.
LanceDB's unique columnar database structure enables serverless deployments and significant cost savings at scale.
LanceDB's separation of compute and storage, facilitated by its columnar format and disk-based vector indices, allows for efficient scaling and fast query performance.
Deep dives
Lance DB: An Overview and Evolution
Lance DB started as a company focused on serving computer vision projects, aiming to build better data infrastructure for managing unstructured data. The motivation came from the complexity and challenges experienced when working with multimodal data for AI. They developed a storage layer called Lance Columnar Format to manage tabular and unstructured data more effectively. Eventually, Lance DB introduced a vector index for deduplication and finding relevant samples for training, which led to its recognition as a vector database. It offers ease of use, hyperscalability, cost-effectiveness, and the ability to manage metadata, raw assets, and vectors together. With applications in generative AI, e-commerce, search engines, and computer vision, Lance DB is well-positioned to handle large datasets and support various use cases.
The Power of Separating Compute and Storage
Lance DB's separation of compute and storage is a key factor in its performance and scalability. By storing data on disk and using disk-based vector indices, Lance DB enables efficient scaling and distributed processing. It offers GPU acceleration for indexing and allows users to work with large datasets using commodity hardware. With its stateless kernels and simplified architecture, Lance DB lowers the complexity of the overall stack, making it easier to scale and maintain without coordination and leader election among nodes. This separation of compute and storage is made possible by Lance DB's columnar format and its disk-based vector indices, enabling fast random access and query performance.
Embedded Database for Seamless Integration
Lance DB stands out with its embedded approach, allowing seamless integration into various workflows and programming languages. Alongside Python, Lance DB supports JavaScript and Rust, where the core of the data format and the embedded database lie. The embedded nature of Lance DB eliminates the need for setting up client-server scenarios and provides flexibility with storage options like S3. Whether it's installing through package managers or deploying on edge devices, Lance DB simplifies the integration process. In addition, Lance DB plans to enhance its integration with databases like DuckDB and Polars, aiming to provide a smooth experience where the vector database becomes transparent, allowing users to focus on familiar tools.
Exciting Opportunities in Practical AI
In the next six to twelve months, Lance DB foresees exciting developments in information retrieval tools, particularly in personalized and domain-specific applications. These applications could provide value and democratize deep expertise in various domains like customer success management, documentation, and compliance. Longer-term prospects involve generalized low-code and no-code tools that leverage generative AI for code generation and creative interfaces. Additionally, the potential integration of generative AI with robotics, drones, and edge devices presents fascinating possibilities in autonomous systems. Lance DB is enthusiastic about these advancements and aims to continue delivering great tools to enhance users' experiences in the coming year.
Prashanth Rao mentioned LanceDB as a stand out amongst the many vector DB options in episode #234. Now, Chang She (co-founder and CEO of LanceDB) joins us to talk through the specifics of their open source, on-disk, embedded vector search offering. We talk about how their unique columnar database structure enables serverless deployments and drastic savings (without performance hits) at scale. This one is super practical, so don’t miss it!
Changelog++ members save 1 minute on this episode because they made the ads disappear. Join today!
Sponsors:
Fastly – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com
Fly.io – The home of Changelog.com — Deploy your apps and databases close to your users. In minutes you can run your Ruby, Go, Node, Deno, Python, or Elixir app (and databases!) all over the world. No ops required. Learn more at fly.io/changelog and check out the speedrun in their docs.
Typesense – Lightning fast, globally distributed Search-as-a-Service that runs in memory. You literally can’t get any faster!
