Get the app
Vivek Yadav
Engineering manager at Stripe with extensive experience building large-scale systems; discussed designing a Spark-based multi-year regression testing and simulation framework for migrations and what-if analysis.
Best podcasts with Vivek Yadav
Ranked by the Snipd community
8 snips
Nov 26, 2025
• 31min
How to Use Apache Spark to Craft a Multi-Year Data Regression Testing and Simulations Framework
chevron_right
Vivek Yadav, an engineering manager at Stripe, shares his expertise in crafting a multi-year regression testing framework using Apache Spark. He highlights the importance of testing migrations against extensive historical data to avoid user regressions. Spark's parallel processing capabilities allow efficient bulk request replays. Vivek discusses the design of reusable libraries and controlled testing environments, boosting developer confidence while maintaining low costs compared to traditional database methods. He emphasizes the framework's versatility for what-if analyses and projections.
The AI-powered Podcast Player
Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
Get the app