The Gradient: Perspectives on AI cover image

Hattie Zhou: Lottery Tickets and Algorithmic Reasoning in LLMs

The Gradient: Perspectives on AI

00:00

The in-Context Learning Picture

In-context learning involves finding an input, output mapping and then doing algorithmic reasoning. In this case, the model had to be able to generalize mechanistically on a new test example out of the distribution. The fact that we were able to do that with whatever prompt is really interesting. It actually points to the bigger picture that we were hoping to start with this work - which is just this idea of like, can we treat language models as students in school?

Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app