Episode 20: Hattie Zhou, Mila, on supermasks, iterative learning, and fortuitous forgetting
Generally Intelligent
00:00
Is There a Limit to Context Learning?
I've been really interested in context learning recently, or trying to figure out ways to steer pre-trained models in desirable ways. In context learning is when in the prompt that you give to the model, you provide a couple examples of what you want the model to do. Essentially prompting or what's called few shot learning in some of these language model literature. That's crazy that it works right. I think I certainly didn't expect it to do so well.
Transcript
Play full episode
