AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
The Switch Transformer Is a Really Exciting Development
Instead of having one gigundo nerl net where when you feed in something at the bottom, it has to do pachinko through the whole network. Instead, fragment it into what's called a mixture of experts - bunches of little neral networks. So when littlet piece of information comes in, like language token or whatever, it gets routed to the right expert. And that will come. We'll get the pytorch were distributed, you know, a mixture of expert transformers in due time, i'm sure.