Heroku's glory days & Postgres vs the world, w/ Craig Kerstiens
Sep 24, 2024
auto_awesome
Craig Kerstiens, an expert from Crunchy Data specializing in Postgres, dives into the rise and importance of Postgres, particularly during its glory days at Heroku. He shares insights on scaling databases and the innovative 'data clips' feature that bridges technical and non-technical teams. The conversation touches on the evolution of Postgres and its extension ecosystem, highlighting challenges like database sharding. Craig also discusses future trends and the unique strengths of Postgres compared to competitors like DuckDB.
Craig Kerstiens discusses Heroku's pivotal role in developing Postgres' popularity, emphasizing teamwork and innovative culture during its early adoption.
Postgres distinguishes itself through its extensibility and robust features like transactional DDL, gaining significant traction within the developer community.
The conversation highlights the importance of integrating solutions like Citus for scalability in modern applications, reflecting evolving data management needs.
Deep dives
Craig Kersteins' Journey with Postgres
Craig Kersteins has an extensive background in working with Postgres, spanning over 15 years, which began during his tenure at Heroku. He played a pivotal role in scaling Postgres support at Heroku, managing up to 1.5 million Postgres databases across their customer base. Following his time there, he shifted to Citus Data, dealing with significantly larger databases, and eventually contributed to Microsoft's Azure Postgres. Craig's perspective is shaped by his developer-focused approach to databases, distinguishing him from more traditional database professionals.
The Evolution and Features of Heroku
The early days of Heroku played a significant role in shaping modern application deployment. At Heroku, there was a strong emphasis on teamwork and innovation, with a culture that fostered open discussions and ideation. One key feature that emerged during his period was Data Clips, designed to allow users to share live SQL queries securely, although its adoption among users was limited. Craig reflects on how the concepts of simplicity and user-friendly deployment directly contributed to Heroku's success and its lasting impact on developer communities.
The Power and Popularity of Postgres
Postgres has gained immense popularity, particularly among developer communities, thanks to its extensibility and robustness. The adoption of Postgres by the Rails community, primarily due to Heroku's exclusive support, catalyzed its rise as the preferred database option for many developers. Features such as transactional DDL and a strong focus on data integrity distinguish Postgres from other database systems. The discussions surrounding its permissive licensing and the absence of a central ownership help solidify its position as a trusted choice for modern applications.
Emerging Trends in Data Management
The conversation highlights the growing importance of integrating data management solutions like Citus into the Postgres ecosystem, allowing for better scalability and performance. Citus turns Postgres into a sharded distributed database, enabling horizontal scalability, which is critical for modern applications. Additionally, the introduction of Iceberg as a new data management framework caters to evolving needs in data analytics and warehousing. This evolving landscape signifies a shift towards more sophisticated solutions that combine the familiarity of Postgres with advanced data processing features.
The Future of Databases: Challenges and Opportunities
The future of databases is characterized by its potential and inherent challenges, as the market sees many newcomers but also recognizes that not all will endure. Craig emphasizes the importance of developer experience in modern database solutions, explaining how systems fail without effective user engagement. He draws parallels between the rise of DuckDB and other contemporary databases to the trajectory of Postgres, highlighting their unique approaches to data management. Ultimately, as the landscape evolves, the emphasis remains on providing efficient, user-friendly, and robust solutions that empower developers.
Want to learn more Postgres? Check out my Postgres course: https://masteringpostgres.com.
In this interview, I dive deep with Craig Kerstiens from Crunchy Data into the world of Postgres, covering its rise to prominence, scaling at Heroku, and the power of Postgres extensions. Craig also shares insights on database sharding, the future of Postgres, and why developers love working with it.
Follow Craig: Twitter: https://twitter.com/craigkerstiens Crunchy Data Blog: https://www.crunchydata.com/blog
Follow Aaron: Twitter: https://twitter.com/aarondfrancis LinkedIn: https://www.linkedin.com/in/aarondfrancis Website: https://aaronfrancis.com - find articles, podcasts, courses, and more.
Chapters: 00:00 - Introduction: Welcome to Database School 00:20 - Guest Introduction: Craig Kerstiens and Crunchy Data 01:40 - Craig's Journey from Heroku to Crunchy Data 02:55 - Scaling Postgres at Heroku 04:50 - Mastering Postgres Course Announcement 05:30 - The Importance of Postgres at Heroku 07:50 - The Value of Live SQL with Data Clips 09:25 - Data Clips for Business Intelligence and Real-Time Analytics 11:05 - Heroku’s Unique Company Culture and How Data Clips Came to Be 12:30 - Postgres Extensions and Marketplace 14:00 - Citus: Scaling Postgres for Multi-Tenant Applications 15:40 - The Challenges of Sharding in Databases 18:00 - Managing Large Databases and Sharding Keys with Citus 24:00 - The Evolution of Postgres and Its Growing Popularity 31:00 - Postgres for Developers and the Importance of Extensions 35:00 - Extensions as Proving Grounds for Core Postgres Features 37:50 - Building an Extension Marketplace for Postgres 41:00 - Postgres as a Data Platform and Developer Flexibility 46:00 - Why Developers Love Postgres: Stability, Extensions, and Ownership 51:00 - DuckDB: A Fascinating New Database Approach 53:30 - Crunchy Data: What They Offer and Why It Matters 58:30 - Expanding Postgres with DuckDB for Data Warehousing 01:00:00 - Wrapping Up: Where to Find Craig and Crunchy Data
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
