The Analytics Engineering Podcast The State of Databases Today (w/ Andy Pavlo)
7 snips
Sep 8, 2023 Andy Pavlo, a professor of databaseology at Carnegie Mellon and founder of OtterTune, talks about the complexity and specialization of database systems, the trend of separating storage and compute, scaling challenges, and the future of the data industry.
AI Snips
Chapters
Transcript
Episode notes
The Golden Era of Databases
- There is a golden era of databases with many specialized systems, more than historically noted.
- The internet and open source platforms like GitHub accelerated this proliferation but sustainability is uncertain.
Graffiti Project Wikipedia Incident
- Andy Pavlo shared a story about his "graffiti project" where he stored data on open MediaWiki sites like Wikipedia.
- The experiment caused a university uproar and led to him being put on probation and teaching computer ethics.
The Power of the Relational Model
- The relational model remains the most flexible and correct way to model almost all data.
- Specialized databases like vector or graph DBs are often unnecessary because relational systems evolve to incorporate needed features.