Patterns of Distributed Systems • Unmesh Joshi & James Lewis
May 17, 2024
auto_awesome
Unmesh Joshi and James Lewis discuss patterns of distributed systems, focusing on cloud services, Kafka, Kubernetes, and databases. They describe common problems and solutions in mainstream open-source distributed systems, emphasizing the importance of being prepared for system failures. The podcast explores practical experiences, consistency with Paxos and Raft, system implementations, failure outages, patterns in distributed systems, and industry awareness of distributed systems algorithms.
Understanding the importance of sharding for data integrity in distributed systems like Kafka and Cassandra.
Adopting a patterns approach to design helps in solving common implementation problems in mainstream open-source distributed systems.
Deep dives
Fascination with Distributed Systems
Doshi's deep dive into distributed systems stemmed from practical experience on a telescope project, where the need for coordination among telescope subsystems led to building a distributed system. Using projects like Kafka and Cassandra to understand design decisions, Doshi delved into open-source frameworks, highlighting the balance of theory and code implementation in fully grasping distributed systems.
Understanding Distributed Systems
Distributed systems revolve around maintaining consistent and available data across multiple servers while partitioning the state for scalability. Doshi's focus lies on systems like Kafka, Cassandra, and Amazon S3, emphasizing the need for sharding and ensuring data integrity across distributed systems.
Patterns Approach and Practical Learning
By adopting a patterns approach, Doshi enables a broader audience to grasp concepts that are traditionally core to systems engineers. Recognizing the vital link between patterns and practical code implementations, Doshi's work aims to expose the tools used by a wider range of professionals across various industries.
Enhancing Distributed Systems Knowledge
Encouraging individuals to dive into open-source code bases, learn from implementations, and play with isolated pieces, Doshi advocates for a hands-on approach to gaining a deeper understanding of distributed systems. By navigating actual code and connecting theory to practical implementation, learners can unravel the complexities of distributed systems more effectively.
Unmesh Joshi - Principal Consultant at Thoughtworks & Author of "Patterns of Distributed Systems" James Lewis - Principal Consultant & Technical Director at Thoughtworks
DESCRIPTION A Patterns Approach to Designing Distributed Systems and Solving Common Implementation Problems
More and more enterprises today are dependent on cloud services from providers like AWS, Microsoft Azure, and GCP. They also use products, such as Kafka and Kubernetes, or databases, such as YugabyteDB, Cassandra, MongoDB, and Neo4j, that are distributed by nature. Because these distributed systems are inherently stateful systems, enterprise architects and developers need to be prepared for all the things that can and will go wrong when data is stored on multiple servers--from process crashes to network delays and unsynchronized clocks.
"Patterns of Distributed Systems" describes a set of patterns that have been observed in mainstream open-source distributed systems. Studying the common problems and the solutions that are embodied by the patterns in this guide will give you a better understanding of how these systems work, as well as a solid foundation in distributed system design principles.