Jacob Buckman

Jacob Buckman, co-founder and CEO of Manifest AI, dives deep into the world of long-context transformers. He discusses innovative techniques like windowed attention and the revolutionary Power Retention approach, which melds attention and recurrence for astonishing training speeds. Buckman also shares insights on Manifest AI's open-source tools, Vidrial and PowerCoder, and explores the significance of metrics in measuring context utility. Learn about the balance between state and weight for optimal compute architectures and the future potential of context length in AI.

Jacob Buckman, an AI researcher and co-founder of Manifest AI, discusses rethinking AI memory and context. He explores the limitations of current transformers with long inputs, introducing the Power Retention model that offers efficient context management. Jacob explains how this new architecture enhances recall and reduces the need for frequent fine-tuning. He also touches on practical applications, such as retrieval agents and the potential for personal AIs that remember user context over time, while emphasizing the importance of open-sourcing this groundbreaking technology.