A distributed database that can withstand a meteor strike
Feb 12, 2025
auto_awesome
Charlie Yang, CTO of OceanBase, dives into his journey from Alibaba to leading a groundbreaking distributed database that supports over a billion monthly users. He discusses the challenges faced by e-commerce giants in database management and how OceanBase emerged as a scalable solution. The conversation highlights performance advantages of native distributed databases over traditional sharding, with insights into transaction-focused databases and their integration with AI applications. Yang emphasizes the vital role of community contributions in shaping effective database solutions.
OceanBase was specifically engineered to tackle the limitations of traditional databases, ensuring high efficiency for Alipay's massive transaction demands.
The database offers seamless disaster recovery and cost savings by enabling real-time analytics, effectively merging OLTP and OLAP functionalities for enhanced data usability.
Deep dives
The Growth of Oceanbase
Oceanbase, a distributed database, was developed to meet the growing needs of Alibaba, particularly for Alipay, which serves over a billion users globally. It faced challenges with traditional transactional databases that relied on sharding, which complicated application maintenance. Instead, Oceanbase was designed from the ground up, allowing it to handle massive transaction volumes efficiently. Over the years, it has proven its capabilities, operating effectively and becoming the sole database utilized by Alipay for various core business functions.
Advantages over Traditional Solutions
One of Oceanbase's key advantages is its ability to provide lossless disaster recovery, ensuring that data is not lost regardless of server or data center failures. It utilizes a column-based consensus protocol to achieve zero recovery point objectives (RPO), meaning recovery happens seamlessly without any data loss. This has not only improved the performance of Alipay's transactional processes but has also significantly reduced database storage costs by about 70% after migrating from MySQL and Oracle. The architecture supports complex transactions while maintaining high availability and consistency across its nodes.
The Future Integration with AI and OLAP
Oceanbase aims to integrate transactional processing with analytical workloads, effectively bridging the gap between Online Transaction Processing (OLTP) and Online Analytical Processing (OLAP). This hybrid transactional and analytical processing (HTAP) model allows users to analyze data in real-time without the inconsistency associated with separate systems. For instance, the successful migration of popular restaurants to Oceanbase has eliminated data lag, leading to immediate accessibility of insights. Going forward, Oceanbase is also expanding its capabilities to support AI applications, incorporating vector storage and processing to streamline complex analytics and AI workloads.