S4E2: Google DeepMind’s Dr. Claire Cui on The Next Frontier for Large Language Models
Theory and Practice
The Importance of a Transformer in Natural Language Processing
The transformer architecture really enabled self-supervised learning. Instead of having to label text, we could just try and predict missing words that we blanked out in the text. And with more parameters, you can fit like all kinds of patterns inside. That's why self-super supervised learning is important because it has almost infinite amount of trillions of tokens and data to turn through.
00:00
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
