Life Is But A Stream Ep 16 - Infinite Scale, No Outages: How MarketAxess Modernized Trading Infrastructure With Confluent
Legacy systems don’t have to hold you back. In high-volume markets where milliseconds matter, MarketAxess—an electronic trading platform for institutional investors—needed a way to scale trade data reliably across on-prem systems and the cloud. That journey—from self-managed Apache Kafka® to MSK to Confluent Cloud—reshaped how their engineering teams build, scale, and govern real-time infrastructure.
In this episode, Andy Steinmann, Software Engineering Manager at MarketAxess, breaks down how his team evolved from a monolithic, socket-driven architecture to a streaming-first platform. He shares why Kafka became the foundation of their 2.0 strategy, how Confluent Cloud eliminated operational drag during peak trading windows, and what it takes to modernize without disrupting a mission-critical financial ecosystem.
You’ll learn:
- How MarketAxess scaled real-time trade flows across on-prem and the cloud
- How Confluent Cloud eliminated outages and operational bottlenecks
- How to approach modernization through iterative “2.0” patterns and dark deployments
If you’re ready to explore hybrid connectivity, governance, performance tuning, and the operational realities of scaling market-data workloads, this is the episode for you.
About the Guest:
Andy Steinmann, Software Engineering Manager at MarketAxess, is a seasoned software engineer with 20+ years of experience in web, mobile, and backend technologies delivering proven solutions to thousands of users. Andy has been the technical lead/architect on many teams and participated in implementing and adapting agile methodologies to improve productivity.
Guest Highlights:
“One of the big benefits of the partnership with Confluent has not only been the zero cluster outages—but the bigger side of it was really the access to the experts… Somebody to walk that journey with you as opposed to just starting with a Google search or a ChatGPT question.”
Episode Timestamps:
3:20 – Data Streaming Goodness
20:20 – Beyond the Stream
36:15 – Quick Bytes
43:15 – Joseph’s Top Takeaways
Dive Deeper into Data Streaming:
- EP1—Stream On: Unleashing Innovation With Data Streaming | Life Is But A Stream
- EP2—Processing Without Pause: Continuous Stream Processing and Apache Flink® | Life Is But A Stream
- EP3—The Connective Tissue: Shift Left to Turn Data Chaos to Clarity | Life Is But A Stream
Links & Resources:
- Connect with Joseph: @thedatagiant
- Joseph’s LinkedIn: linkedin.com/in/thedatagiant
- Learn more at Confluent.io
Our Sponsor:
Your data shouldn’t be a problem to manage. It should be your superpower. The Confluent data streaming platform transforms organizations with trustworthy, real-time data that seamlessly spans your entire environment and powers innovation across every use case. Create smarter, deploy faster, and maximize efficiency with a true data streaming platform from the pioneers in data streaming. Learn more at confluent.io.
Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.