Building GPT-2 in a Spreadsheet — Everything You Wanted to Know About LLMs (But Were Afraid to Ask)

Learn how to demystify large language models by building GPT-2 from scratch — in a spreadsheet. In this episode, MIT engineer Ishan Anand breaks down the inner workings of transformers in a way that’s visual, interactive, and beginner-friendly, yet deeply technical for experienced builders.

What you’ll learn:

• How GPT-2 became the architectural foundation for modern LLMs like ChatGPT, Claude, Gemini, and LLaMA.

• The three major innovations since GPT-2 — mixture of experts, RoPE (rotary position embeddings), and advances in training — and how they changed AI performance.

• A clear explanation of tokenization, attention, and transformer blocks that you can see and manipulate in real time.

• How to implement GPT-2’s core in ~600 lines of code and why that understanding makes you a better AI builder.

• The role of temperature, top-k, and top-p in controlling model behavior — and how RLHF reshaped the LLM landscape.

• Why hands-on experimentation beats theory when learning cutting-edge AI systems.

Ishan Anand is an engineer, MIT alum, and prolific AI tinkerer who built a fully functional GPT-2 inside a spreadsheet — making it one of the most accessible ways to learn how LLMs work. His work bridges deep technical insight with practical learning tools for the AI community.

Key topics covered:

• Step-by-step breakdown of GPT-2 architecture.

• Transformer math and attention mechanics explained visually.

• How modern LLMs evolved from GPT-2’s original design.

• Practical insights for training and fine-tuning models.

• Why understanding the “old” models makes you better at using the new ones.

This episode of AI Tinkerers One-Shot goes deep under the hood with Ishan to show how LLMs really work — and how you can start building your own.

💡 Resources:

• Ishan Anand LinkedIn – https://www.linkedin.com/in/ishananand/

• AI Tinkerers – https://aitinkerers.org

• One-Shot Podcast – https://one-shot.aitinkerers.org/

👍 Like this video if you found it valuable, and subscribe to AI Tinkerers One-Shot for more conversations with innovators building the future of AI!