Tech Today with Eric Tarczynski Why AI Needs Fresh Data with Marc Freed-Finnegan
May 30, 2025
Marc Freed-Finnegan, co-founder and CEO of Chalk, explores the critical shift from AI training to real-time inference. He dives into why fresh data is vital for AI infrastructure and how Chalk achieves sub-5ms pipelines for clients like Whatnot and MoneyLion. Marc also discusses challenges in implementing AI in businesses and the need for robust infrastructure. Plus, he shares insights on innovations across industries, from fintech to e-commerce, emphasizing the transformative power of real-time data in enhancing customer experiences.
AI Snips
Chapters
Transcript
Episode notes
Shift From Training to Inference
- AI infrastructure focus is shifting from training to real-time inference.
- Serving fresh data with sub-5ms latency enables instant user decisions improving product experience.
Real-Time Beats Batch Processing
- Batch jobs process millions of users overnight and lack freshness.
- Real-time inference with fresh data enables instant fraud checks and recommendation updates.
Python at Real-Time Scale
- Chalk allows users to write Python code transpiled into fast languages like C++ and Rust.
- This enables running Python at real-time scale with low latency for inference.