
5. Charlie Snell on DALL-E and CLIP
The Inside View
Code Book Vectors and a Vector Quantizing Hack
The DVA is like another discrete auto encoder thing but it's a little bit different than the VQVA. The hack is what's called the gumbell softmax which is sort of a trick for approximating a discrete distribution with a continuous distribution because the problem is you can't really differentiate through sampling from a discrete distribution so they try to approximate it. And yeah if I my end would be dead so now you have like a probability distribution over your code book vector. So sounds a bit hacky but I trusted the networks.
00:00
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.