Forward Deployed, Episode 3: Context Engineering
Forward Deployed
Memory, retrieval and semantic search (RAG)
Lance and Noah consider indexing memories, semantic search, and RAG as practical solutions for scale.
Welcome to episode three of Forward Deployed. Noah and Lance dive deep into context engineering—the art and science of filling context windows with just the right information for agents to take the next action.
Key Topics Covered
- Context engineering: Why Karpathy’s term matters more than prompt engineering for agents
- Three core techniques: Reducing context, isolating context, and offloading context to file systems
- Skills vs MCPs: Why progressive disclosure beats loading everything into context
- Claude Diary: Building agent memory through reflection and evolving CLAUDE.md
- Why subagents should isolate context, not anthropomorphize org charts
- Production patterns from Manus: Compaction, file system offloading, and sandbox architecture
- The convergence of agent primitives: Read, write, reduce, and recombine
- Why treating agents like humans works: Dual-use tools and new coworker onboarding
Timestamps
- 00:10 - What is context engineering and why should we care?
- 00:37 - Karpathy’s definition: Filling context windows with just the right information
- 01:15 - Context engineering vs prompt engineering: Tools bring context too
- 02:27 - Three buckets: Reducing, isolating, and offloading context
- 04:30 - Dynamic context loading: Skills as just-in-time context
- 06:38 - Skills as context offloading: Progressive disclosure of tools
- 09:15 - Sandboxes beyond Manus and Claude Code: Application integration patterns
- 12:12 - Context isolation through subagents: Not org charts
- 14:41 - Using isolation as a feature: Test-driven development with blank slate agents
- 16:33 - Real-world subagent challenges: Shared file systems and context bottlenecks
- 20:47 - Claude Diary: Automatic memory through session reflection
- 22:53 - CLAUDE.md bloat: Reducing unnecessary words like Strunk and White
- 28:06 - Dual-use tools: Why Claude Code uses bash, grep, and human-readable commands
- 31:02 - Skills discovery problems: From manual search to automated finding
- 34:22 - Agent primitives: Read, write, reduce, and creative recombination
- 36:32 - Tool search and action space design: How to find the right function
- 40:50 - Hooks as determinism: Injecting code into agent loops
- 41:41 - Skills everywhere: ChatGPT’s progressive disclosure discovery
- 43:12 - Nano Banana: Text on images unlocks new agentic workflows
Links & References
Core References
* 🔗 Karpathy tweet on context engineering
* 🎙️ Manus webinar on context engineering
* 📄 Cognition blog post on multi-agents
* 📄 Simon Willison on skills and file systems
* 🔗 Anthropic Model Context Protocol announcement
* 🔗 GitHub MCP docs (Anthropic)
Technical Resources
* 🔗 Claude Code hooks documentation
* 🔗 Anthropic frontend design skills example
* 📄 Obra magic skills repository
Tools & Frameworks
* 🔗 Cerebras Systems – Technology
Related Content
- 🎙️ Karpathy on Dwarkesh Podcast - Discussion of AI development principles
- 📚 The Elements of Style by Strunk and White - “Reduce unnecessary words” as principle for CLAUDE.md
- 📄 The Bitter Lesson by Richard Sutton - General methods that leverage computation
About the Hosts
Noah Brier is co-founder of Alephic, an AI consulting company helping brands build custom AI systems. He writes about AI strategy and implementation.
Lance Martin is a founding engineer at LangChain, where he works on developer tools for building AI applications.
Connect with the Hosts
- Noah Brier: LinkedIn | X/Twitter
- Lance Martin: LinkedIn | X/Twitter
* * *
Subscribe for weekly episodes exploring how AI is actually being deployed in the real world.
This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.forwarddeployed.com
The AI-powered Podcast Player
