The Backend Engineering Show with Hussein Nasser How Apache Kafka got faster by switching ext4 to XFS
Apr 29, 2024
Allegro improved Kafka latency by 80% switching from ext4 to XFS. They traced Kafka protocol and kernel system calls, optimized ext4 before switching. Show explores file system complexities, journaling impact, and challenges of workload increase.
Chapters
Transcript
Episode notes
1 2 3 4 5 6
Introduction
00:00 • 4min
Investigation and Optimization of Apache Kafka's File System for Performance Enhancement
03:39 • 2min
Optimizing Performance in Apache Kafka
05:40 • 13min
Exploration of File System Complexity and Metadata Management
18:38 • 3min
Exploring Journaling in File Systems and Database Optimizations
21:16 • 10min
Challenges of Increasing Workload and Contributions to Kafka
31:24 • 2min