
Yann LeCun on his Start in Research and Self-Supervised Learning
The Gradient: Perspectives on AI
00:00
The Basic Idea of VCRAG and Borrow Twins
Borrow Twins and VCRAG are two ways to look at mutual information. Borrow Twins uses normalization, while VCRAG does not. The idea is that you want the variance of each of the variables of both of those vectors to be non-zero. It doesn't matter what it is, but it needs to be. So some people have called this dimensional collapse.
Transcript
Play full episode