Dev Interrupted The hidden costs of pre-computing data | Chalk's Elliot Marx
Dec 9, 2025
Elliot Marx, co-founder of Chalk and fintech infrastructure expert, discusses the crucial shift from pre-computed data towards real-time data pipelines for AI and ML. He highlights how real-time compute can significantly reduce costs and latency, and emphasizes the importance of strong problem-solving skills over specific programming language expertise. Elliot advocates for incremental development, empowering engineers, and maintaining close customer relationships to shape product roadmaps. He also touches on the necessity of leadership skills in the AI era.
AI Snips
Chapters
Transcript
Episode notes
Real-Time Pipelines Replace Bulk Precompute
- Pre-computing massive datasets is the old paradigm and fails when inference needs fresh, context-specific data.
- Real-time pipelines fetch and compute only what you need at inference time, reducing waste and enabling new AI use cases.
Compute Only What You Need On Demand
- Save money by computing only the predictions you need at request time instead of computing everything in advance.
- Move expensive vendor or model evaluations to on-demand inference to avoid paying for unused precomputed results.
Low-Latency Needs Favor Traditional Models
- Ultra-low latency workloads still rely on traditional ML models like XGBoost, not LLMs.
- Chalk accelerates Python and SQL data code by converting Python ASTs into vectorized SQL-style execution for real-time speed.