High Signal: Data Science | Career | AI Episode 28: From Context Engineering to AI Agent Harnesses: The New Software Discipline
46 snips
Nov 13, 2025 Lance Martin, a machine learning engineer at LangChain, dives into the evolving landscape of AI engineering. He emphasizes the importance of context engineering and how traditional ML rules are becoming obsolete. The conversation covers why adaptable systems thrive, the architectural advantages of 'agent harnesses,' and the shift towards in-app user feedback for evaluating AI systems. Lance also shares insights on balancing autonomy in agents with human oversight and techniques for managing costs and performance in complex AI tasks.
AI Snips
Chapters
Transcript
Episode notes
Shift To Building On A Model Primitive
- Foundation models shifted ML from training-focused work to higher-level orchestration and engineering.
- Engineers now build on an API-provided computing primitive rather than designing models from scratch.
Start Simple And Instrument Heavily
- Start with the simplest solution: use prompts before workflows and workflows before agents.
- Invest in observability and rigorous evaluation tailored to non-deterministic LLM behavior.
Bitter Lesson Means Embrace Constant Change
- Rich Sutton’s 'bitter lesson' shows general compute-heavy methods outperform hand-crafted structure over time.
- Systems built today must be ready to simplify as models rapidly improve or they'll bottleneck progress.