Hard Software Hard Software Episode 11: Dashboard Crashes, Streaming Needs, and Gen AI Pipeline Hype with Ricky Thomas, Co-founder + CTO @ Streamkap
May 29, 2025
In this engaging discussion, Ricky Thomas, Co-founder and CTO of Streamkap, shares his insights on the evolution of data processing from batch to real-time streaming solutions. He emphasizes the critical role of generative AI in enhancing business workflows and the challenges of integrating diverse team dynamics across cultures. Ricky also sheds light on the importance of reliability in AI and the complexities of communication within engineering teams. The conversation ends with a look at the future of data management and the balancing act between batch and streaming architectures.
AI Snips
Chapters
Transcript
Episode notes
Batch ETL Was Necessary Historically
- Batch ETL was necessary when streaming was difficult and costly.
- Now streaming is becoming easier and can serve more use cases beyond traditional ones like fraud detection.
Ricky's Foundational Career Journey
- Ricky's background spans financial trading to e-commerce, highlighting challenges with real-time data and insights.
- His experiences led to founding Avora and later Gravity, focusing on BI, ETL, and anomaly detection.
Streaming Needs Production Reliability
- Streaming infrastructure is still complex and requires more than data engineers to maintain production-grade pipelines.
- Platforms like Streamkap simplify this but production use cases need reliability beyond dashboards.