The Missing Architecture Behind Autonomous AI with Jacob Buckman

In this episode of Machine Minds, we step beyond today’s transformer-dominated AI landscape and into a deeper conversation about what’s missing on the path to truly autonomous, long-horizon intelligence. Jacob Buckman, co-founder and CEO of Manifest AI, joins Greg to explore why current AI systems struggle with long-term reasoning, persistent memory, and extended task execution—and what it will take to unlock the next paradigm.

Jacob’s journey into AI began early, fueled by science fiction, programming, and a fascination with building systems that could do meaningful work autonomously. From studying and conducting research at Carnegie Mellon to working at Google Brain, he watched deep learning unify once-fragmented AI subfields—vision, language, speech—under a single scalable framework. That unification shaped his conviction that the next breakthrough wouldn’t come from incremental tuning, but from rethinking a fundamental architectural bottleneck.

At Manifest AI, Jacob and his team are tackling what they believe is the missing piece: scalable long-context intelligence. Their work centers on replacing transformer attention with a new family of architectures called retention models, designed to compress and retain relevant information over time—rather than repeatedly replaying massive histories. The goal: AI systems that can reason, learn, and work continuously over hours, days, or longer.

In this conversation, Greg and Jacob explore:

• Jacob’s path from aspiring scientist to AI researcher and founder—and why curiosity plus first principles thinking matter more than trends
• Why today’s large language models excel at short tasks but break down over long horizons
• The core limitation of transformer attention—and why “attention is all you need” may no longer hold
• How retention architectures unify the strengths of transformers and recurrent neural networks
• What it means for an AI system to compress knowledge instead of endlessly appending memory
• Why long-term reasoning, iterative problem solving, and true autonomy require architectural change—not orchestration hacks
• The misconception that agent orchestration can substitute for unified, persistent intelligence
• How long-context models could reshape agents from short-lived “consultants” into persistent, personalized collaborators
• The technical challenge of translating theoretical breakthroughs into high-performance GPU kernels
• Why Manifest AI is open source—and how their work aims to move the entire field forward
• Lessons from unifying AI subfields, the “bitter lesson” of scale, and avoiding ad-hoc solutions that won’t last
• Jacob’s view on cost, intelligence density, and why better architectures will increase—not reduce—investment in AI
• Advice for founders and researchers: focus relentlessly on the single bottleneck that matters most

If you’re building AI systems, researching foundations of intelligence, or trying to understand what comes after today’s models, this episode offers a rare, deeply reasoned look at where the field may be heading—and why architectural simplicity could unlock far more than brute force scale.

Learn more about Manifest AI: https://manifestai.com

Explore the open-source retention models: pip install retention

Connect with Jacob Buckman on LinkedIn: https://www.linkedin.com/in/jacobbuckman

Connect with Greg Toroosian on Linkedin: https://www.linkedin.com/in/gregtoroosian