The Gradient: Perspectives on AI cover image

Hattie Zhou: Lottery Tickets and Algorithmic Reasoning in LLMs

The Gradient: Perspectives on AI

CHAPTER

The in-Context Learning Picture

In-context learning involves finding an input, output mapping and then doing algorithmic reasoning. In this case, the model had to be able to generalize mechanistically on a new test example out of the distribution. The fact that we were able to do that with whatever prompt is really interesting. It actually points to the bigger picture that we were hoping to start with this work - which is just this idea of like, can we treat language models as students in school?

00:00
Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner