The Data Exchange with Ben Lorica The Junior Data Engineer is Now an AI Agent
11 snips
Jan 8, 2026 Matthew Glickman, Co-founder and CEO of Genesis Computing, shares his insights on the rise of AI data agents tailored for data engineers. He discusses the challenges of transitioning from demo to production, emphasizing the importance of reliability. Glickman highlights the potential of automation to alleviate the burdens of data engineers rather than replace them. He also explores capturing institutional knowledge and using AI to document legacy systems. Finally, he examines the implications of job shifts and the need for new AI-related skills in the workforce.
AI Snips
Chapters
Transcript
Episode notes
The Last 10% Is Reliability
- Many enterprises built impressive GPT demos but hit a steep last 10% problem to make them reliable in production.
- Matthew Glickman says non-deterministic models and asymmetric risk make reliability the key obstacle to deployment.
Measure Confidence And Escalate
- Define how to judge task correctness and use model confidence to auto-escalate uncertain results.
- Stop relying on users to review every output and only surface cases with low confidence.
Data Engineers Want Superpowers
- Targeting the data engineer persona unlocks high adoption because they want tools that give time back, not replace them.
- Data engineers welcome automation that frees them for higher-impact work, says Matthew Glickman.