Exploration of File System Complexity and Metadata Management

https://oscourse.win

Allegro improved their Kafka produce tail latency by over 80% when they switched from ext4 to xfs. What I enjoyed most about this article is the detailed analysis and tweaking the team made to ext4 before considering switching to xfs. This is a classic case of how a good tech blog looks like in my opinion. 

0:00 Intro

0:30 Summary

2:35 How Kafka Works?

5:00 Producers Writes are Slow

7:10 Tracing Kafka Protocol 

12:00 Tracing Kernel System Calls

16:00 Journaled File Systems

21:00 Improving ext4

26:00 Switching to XFS

Blog

https://blog.allegro.tech/2024/03/kafka-performance-analysis.html