[11] Jacob Andreas - Learning from Language
The Thesis Review
00:00
How to Bake Compositional Biases Into Neural Models
I'm really into data augmentation these days as just like a relatively unstructured flexible way to bake compositional biases or inductive biases generally into neural models where we don't know how to do it at the architectural level. I think those kinds of techniques haven't gotten nearly enough attention in NLP as just like engineering solutions to um do this problem but then there's also the deeper question of why is that actually fair game? We want you to generalize to complex sentences involving the word jump or complex web queries that you've never seen before. Can you formalize uh what it is in a way that maybe makes it easier to bake into model structures or at least to just like reason
Transcript
Play full episode
