

Apache Storm, Disruptor, JCTools and Linearizability
Apr 27, 2025
Francesco Nigro, a software engineer and JCTools contributor, shares his insights on concurrent programming and performance optimization in Java. He discusses the evolution of JCTools and its contributions to efficient data structures, including weight-free queues. The conversation highlights JCTools’ advantages over standard Java collections, the nuances of linearizability, and the differences between lock-free and wait-free algorithms. Additionally, they delve into Apache Storm’s event processing capabilities and the importance of memory management in achieving high throughput.
AI Snips
Chapters
Transcript
Episode notes
History and Contributions of JCTools
- JCTools began as a concurrency utility by Nitsan Wakart and later integrated Cliff Click's non-blocking hashmap when both worked at Azul.
- Francesco Nigro contributed significantly to JCTools after using it to replace Java queues in high-performance stream processing projects.
Apache Storm for In-Memory Processing
- Apache Storm excels for in-memory stream processing without persistence, unlike Kafka which relies heavily on disk.
- Storm suits transient events that don't require durable storage, enabling high-speed event handling in volatile use cases.
Array-Based Queues Reduce Garbage
- JCTools queues minimize garbage creation and optimize memory layout by using array-based ring buffers instead of linked nodes.
- The array approach enables prefetching and avoids pointer indirections, significantly improving throughput and reducing latency.