Semantic Search: A Deep Dive Into Vector Databases (with Zain Hasan)
Developer Voices
Efficiency and Flexibility of HNSW Algorithm for Creating and Updating Indexes
Building an index using the HNSW algorithm is more intensive than searching the index, with creation taking hours while searches can be performed quickly. The algorithm's plus point is its ability to support insertion and deletion of data points, allowing for quick updates without reconstructing the entire index. Insertion involves adding data points at the bottom level and letting them probabilistically survive through the hierarchy. The surviving points at the top level are actual documents, not average centroids. HNSW does not average vectors but uses the actual vectors for searching. As the algorithm is fast with a logarithmic complexity, there may not be a significant benefit in cutting off the search hierarchy before reaching the bottom levels.
Remember Everything You Learn from Podcasts
