The Infra Pod Databases are becoming commodity, what's next? Chat with Chris from Materialized View
Mar 19, 2024
Chris Riccomini, ex-Distinguished engineer at Wepay and co-creator of Apache Samza, discusses the evolution of data systems, the rise of Arrow, and the future trends in database systems. Topics include domain-specific databases, object storage for edge systems, developer ergonomics, and the shift towards specialized Postgres-compatible databases.
AI Snips
Chapters
Transcript
Episode notes
LinkedIn's Hadoop Adoption
- Chris Riccomini recounts LinkedIn's struggle to compute "friends of friends" with traditional databases.
- Hadoop was adopted after four other systems failed, providing a visible progress bar and eventual success.
Scalability as Table Stakes
- Scalability is now a basic requirement for data systems, unlike in the early Hadoop days.
- DuckDB promotes a single-query engine approach, challenging the focus on scale.
Vertical Databases
- Chris Riccomini predicts the rise of specialized databases for specific use cases.
- He highlights TigerBeetle (finance) and Niall (SaaS) as examples.