The Backend Engineering Show with Hussein Nasser
How Apache Kafka got faster by switching ext4 to XFS
Apr 29, 2024
Allegro improved Kafka latency by 80% switching from ext4 to XFS. They traced Kafka protocol and kernel system calls, optimized ext4 before switching. Show explores file system complexities, journaling impact, and challenges of workload increase.
33:52
AI Summary
AI Chapters
Episode notes
Podcast summary created with Snipd AI
Quick takeaways
- Allegro improved Kafka's producer tail latency by 82% through switching from ext4 to XFS file system.
- Understanding OS kernel processes is crucial for efficient programming, highlighted by Allegro's optimizations in Kafka.
Deep dives
Allegro Improves Kafka's Producer Rights with File System Switch
Allegro enhanced Kafka's producer rights by 82% through a file system switch. Allegro's blog titled 'Unlocking Kafka's Potential with eBPF' details Allegro's journey to overcome tail latency. They traced Kafka, parsed its TCP binary protocol, and identified file system slowdowns. By transitioning to the XFS file system, Allegro achieved an impressive 82% boost in efficiency.
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
