Intelligent Machines (Audio)

IM 892: Pants-based Benchmarks - GitHub's Agent Revolution

56 snips
Oct 15, 2026
Martin Woodward, GitHub’s developer-relations VP and open-source advocate, explores how AI agents are transforming coding. They discuss agent-generated pull requests overwhelming maintainers, DevOps safeguards, multi-model Copilot workflows, and the emotional impact on professional developers. The conversation also covers personal software, AI memory and privacy, self-replicating models, Character AI’s shaky economics, and Nicolas Cage’s stand against AI waivers.
Ask episode
AI Snips
Chapters
Books
Transcript
Episode notes
ADVICE

Protect Open Source From Agentic Contribution Spam

  • Maintainers should limit or temporarily disable pull requests when inexperienced contributors generate overwhelming low-quality submissions.
  • Automated tests, security scans, and AI review can filter changes before human acceptance.
INSIGHT

Personal Benchmarks Beat Model Leaderboards

  • Leo Laporte’s personal benchmarks test whether models solve his actual workflows, not whether they top standardized leaderboards.
  • Argon automated Slack-to-teleprompter ad transfers and produced a reliable Python script.
ADVICE

Prompt Agents With Goals And Verification

  • Talk to agents conversationally rather than constructing elaborate prompts.
  • State the goal, desired effort, and verification method, then ask the model to repeat its understanding before proceeding.
Get the Snipd Podcast app to discover more snips from this episode
Get the app