AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Is There a Difference Between Positive and Negative Pairs?
In contrastive learning, you need positive and negative pairs. And the problem with that is that it doesn't work in high dimension. What I'm much more enthusiastic about these days is non-contrastive methods. We came up with something that we call BALO twins. It's a particular way of maximizing the information content of a vector using some hypotheses.