Lex Fridman Podcast cover image

#459 – DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters

Lex Fridman Podcast

00:00

Importance of Memory in Reasoning

  • Parameter count and mixture of experts models allow for changing active vs. total parameters to embed more data with less flops.
  • However, memory is crucial because of the transformer and attention mechanism, which allow models to understand relationships between words.
Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app