Software Huddle cover image

Software Huddle

SQL Meets Vector Search with Linpeng Tang of MyScale

Apr 2, 2024
01:01:38

Welcome back to an episode where we're talking Vectors, Vector Databases, and AI with Linpeng Tang, CTO and co-founder of MyScale. MyScale is a super interesting technology. They're combining the best of OLAP databases with Vector Search. The project started back in 2019 where they forked ClickHouse and then adapted it to support Vector Storage, Indexing, and Search.

The really unique and cool thing is you get the familiarity and usability of SQL with the power of being able to compare the similarity between unstructured data.

We think this has really fascinating use cases for analytics well beyond what we're seeing with other vector database technology that's mostly restricted to building RAG models for LLMs. Also, because it's built on ClickHouse, MyScale is massively scalable, which is an area that many of the dedicated vector databases actually struggle with.

We cover a lot about how vector databases work, why they decided to build off of ClickHouse, and how they plan to open source the database.


Timestamps

02:29 Introduction

06:22 Value of a Vector Database

12:40 Forking ClickHouse

18:53 Transforming Clickhouse into a SQL vector database

32:08 Data modeling

32:56 What data can be Vectorized

38:37 Indexing

43:35 Achieving Scale

46:35 Bottlenecks

48:41 MyScale vs other dedicated Vector Databases

51:38 Going Open Source

56:04 Closing thoughts

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode