The Infra Pod Building a serverless full-text + vector search on S3! Chat with Simon from Turbopuffer
Apr 7, 2025
Join a fascinating conversation with a tech CEO who transitioned from scaling Shopify to creating an ultra-fast vector search engine. Discover the challenges posed by traditional databases like Postgres, and how building on object storage is changing the game. The talk dives into TurboPuffer's unique trade-offs, including the delicate balance of consistency and write latency. Plus, get insights on the Bring Your Own Cloud concept and the future of serverless solutions in database management. It's a tech lover's dream!
AI Snips
Chapters
Transcript
Episode notes
Scaling Shopify
- Simon worked at Shopify for almost a decade, scaling their infrastructure.
- He tackled challenges like Kardashian merchandise sales and high-traffic events.
Postgres Challenges
- Postgres, while initially appealing, proved difficult to tune, especially its autovacuum feature.
- This, combined with the ongoing challenge of search, inspired Simon to rethink search engine design.
Readwise Experiment
- At Readwise, Simon experimented with vector search for content recommendations.
- Existing solutions were prohibitively expensive, costing significantly more than their core Postgres database.