The Gradient: Perspectives on AI cover image

Kyunghyun Cho: Neural Machine Translation, Language, and Doing Good Science

The Gradient: Perspectives on AI

00:00

Is There a Destructive Interference Pattern in Multitask Learning?

There is always a destructive interference pattern across this task. We want to train one adapter for each of the tasks. And then what we do is that we're going to retrain an adapter fusion layer for each of this task. So it's a two-stage training procedure, which is a bit more cumbersome. But at the same time, completely bypasses the issue of the destructive interference. Again, the same thing with the mix out.

Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app