The Gradient: Perspectives on AI cover image

Hattie Zhou: Lottery Tickets and Algorithmic Reasoning in LLMs

The Gradient: Perspectives on AI

00:00

The Inductive Bias Picture Is Really Interesting

The inductive bias picture is really interesting too. There's a great blog post from a while ago on double descent called inductive biases stick around. Yeah, I don't know how to sort of establish that as the actual explanations for what's happening. But I would be curious to see people are able to draw a connection to information bottleneck. So I do wonder if it can explain some of the grokking behavior as well.

Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app