The Core Architecture in a Transformer

Anton Teaches Packy AI is Not Boring's attempt at making AI more accessible to our audience. It's become increasingly obvious that we're in the Golden Age of AI, so we think it's important to demystify what's going on and how it all actually works.  

Anton Troynikov is the founder of Chroma, former Meta Reality Labs Research Engineer and Roboticist.  

Packy McCormick is the author of the popular tech and buinsess strategy newsletter, Not Boring.  

Anton Teaches Packy AI is exactly what it sounds like -- each video, Anton breaks down AI to a level that Packy (your average above average smart person) can understand.   In Episode 1, Anton and Packy discuss the groundbreaking "Attention is All You Need" research paper which kicked off the entire Transformer generative AI wave.   

LINKS: 

- Attention is All You Need: https://arxiv.org/abs/1706.03762

- The Bitter Lesson: http://www.incompleteideas.net/IncIdeas/BitterLesson.html

- Training Compute-Optimal Large Language Models: https://arxiv.org/abs/2203.15556

- Explain Paper: https://www.explainpaper.com/