Beyond The Prompt - How to use AI in your company

The Future of AI with Illia Polosukhin: The Man Who Put the T in GPT

38 snips

Dec 9, 2025

Illia Polosukhin, co-author of the groundbreaking 'Attention Is All You Need' paper and co-founder of NEAR Protocol, dives into the evolution of AI. He reveals how the shift from recurrent models to transformers was born from practical constraints at Google. Illia discusses the hidden qualities of AI systems, the importance of data provenance, and the future of blockchain in fostering trust. With insights on the empowering nature of personal AI models and the impending agent-to-agent economy, he paints a fascinating picture of AI's trajectory.

Ask episode

AI Snips

Chapters

Books

Transcript

Episode notes

INSIGHT

Early Signals Predicted The Step Change

Illia saw accelerating research signals and benchmark improvements that signaled an imminent step change in AI.
He expected ChatGPT-level breakthroughs years before they became mainstream and left Google to build on that belief.

INSIGHT

Parallel Attention Unlocks Scale

Transformers replace recurrence by processing all words in parallel, unlocking massive GPU utilization and speed.
Each transformer layer adds multi-hop relationships, enabling scalable training on huge text corpora.

ANECDOTE

Paying Students Sparked Near's Crypto Pivot

Near AI began by coordinating global students to label code and struggled to pay them through traditional rails.
That payment friction drove them to explore blockchain as a practical solution for global coordination.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

In this episode, Illia Polosukhin joins Henrik and Jeremy to trace the origins of transformers and how practical constraints inside Google led to a breakthrough that reshaped modern AI. He explains why recurrent models were hitting limits, how parallel attention opened the door to scale, and why he believed a major jump in capability was imminent long before the rest of the world saw it.

The conversation then turns to the risks and responsibilities of today’s AI systems. Illia describes how models can be subtly guided to influence user opinions, why open weights are not the same as truly open models, and how hidden behaviors can be embedded during training. He explains why provenance and verifiable data pipelines matter, especially as AI begins mediating more of the information we rely on.

Later in the episode, Illia outlines how blockchain can support trust, identity, and coordination in a future where AI agents act on our behalf. He shares why information is becoming more valuable than money, how ownership of personal AI models will shape user agency, and why domain expertise becomes significantly more powerful when paired with modern generative tools.

Key Takeaways:

Transformers emerged from practical constraints, not theory
Illia explains that the shift from recurrent networks to attention was driven by speed and parallelization needs at Google, not a desire to invent a new paradigm.
AI’s step change was foreseeable to early builders
Illia expected a ChatGPT level breakthrough several years before it arrived, based on clear research signals and accelerating model performance.
Provenance and trust will define the next phase of AI
As AI systems can be subtly manipulated, Illia argues that verifiable data pipelines and transparent training processes are essential to prevent large scale misinformation.
Ownership and identity matter in an agent driven world
Illia believes individuals will soon rely on AI agents that act autonomously, making it critical that users own their models and that interactions between agents are secured and verified.

https://near.ai – NEAR AI Cloud and Private Chat products are now live, try them here
Illia's X: x.com/ilblackdragon
Illia's Substack: ilblackdragon.substack.com
NEAR X: x.com/nearprotocol

00:00 Intro: AI and Information Control
00:29 Meet Illia Polosukhin: Co-Author of 'Attention is All You Need'
01:03 The Evolution and Impact of AI
13:24 The Birth of Near AI and Blockchain Integration
15:16 Challenges and Innovations in Blockchain and AI
22:17 Privacy and Security in AI Applications
26:58 Exploring Sleeper Agents in AI
29:19 Practical AI Implementation in Teams
30:06 AI's Role in Product Development
31:41 Challenges and Future of AI in Development
36:35 AI and Economic Alignment
41:46 The Future of AI Agents
44:14 Debrief

📜 Read the transcript for this episode: Transcript of The Future Of AI With Illia Polosukhin: The Man Who Put The T In GPT |

For more prompts, tips, and AI tools. Check out our website: https://www.beyondtheprompt.ai/ or follow Jeremy or Henrik on Linkedin:

Henrik: https://www.linkedin.com/in/werdelin
Jeremy: https://www.linkedin.com/in/jeremyutley

Show edited by Emma Cecilie Jensen.