Delving into the concept of decoupling storage and compute in cloud products like RDS Aurora, Google Cloud AlloyDB, and Neon. Exploring the benefits and challenges of decoupled storage and compute, compatibility in open-source systems like Neon, and the need for thorough research when using Postgres in different providers. Discussing serverless architecture, including scale to zero and decoupled storage and compute. Exploring control and configurability of storage and compute in decoupled systems, and the potential benefits of Aurora's plan management feature for auto scaling and serverless approaches.
Decoupling storage from compute in databases like Postgres can improve performance, scalability, and ease major version upgrades.
Users need to thoroughly evaluate the extent of Postgres compatibility among providers and consider factors like cost, features, and long-term implications.
Deep dives
Understanding the concept of separating storage from compute
This podcast episode delves into the concept of separating storage from compute in the context of databases. The hosts discuss various providers, such as RDS Aurora, Google Cloud AloyDB, and Neon, that offer products decoupling compute from storage. They explore the benefits of this separation, such as improved performance, scalability, and the ability to handle major version upgrades more easily. Furthermore, they discuss the different interpretations of the term 'serverless' and its relevance to this topic, highlighting the importance of understanding the specific features and compatibility of these providers.
Exploring the implications of storage and compute separation
The hosts delve into the practical implications of separating storage from compute in Postgres databases. They discuss the possibilities of using object storage like S3 for storing data, the advantages of bottomless storage, and the potential for more efficient scaling and auto-scaling. They also touch on the concept of database branching and the benefits it offers for cloning and managing databases. However, they raise concerns about the limitations of the current implementations and how the cost structure of serverless models can impact users.
Considerations for Postgres compatibility and Postgres' future
The podcast explores the question of Postgres compatibility among these providers. While they claim to be Postgres compatible, the hosts emphasize the need for users to thoroughly evaluate the extent of compatibility, including the availability of specific Postgres extensions and syntax differences. They discuss the possibilities of certain features like plan management making their way into the Postgres ecosystem, but stress that significant adjustments and contributions from providers or the community would be necessary. They also touch on the commercial and ethical aspects of open source and closed source offerings.
Assessing the benefits and drawbacks for users
The hosts share their opinions on the different providers and their offerings. They mention that while Aurora and Neon have gained popularity, there are factors to consider when choosing these services, such as the cost, availability of features, and the long-term implications of vendor lock-in. They also discuss the potential benefits for startups, such as reduced costs and easier scalability, while acknowledging the need for more mature and stable solutions for larger companies. They conclude that for general users of Postgres without specific challenges, the traditional Postgres setup is likely sufficient and that these new solutions cater to specific use cases and needs.
Nikolay and Michael discuss a listener question — about products that take Postgres and transform it to something that decouples compute from storage (RDS Aurora, GC AlloyDB, Neon etc.) and whether they see something like this landing upstream in the medium term.
Here are some links to some things they mentioned:
What did you like or not like? What should we discuss next time? Let us know via a YouTube comment, on social media, or by commenting on our Google doc!