AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Is It the Contrastive Case?
Self supervised contrastive learning generalized better, not because it capured better features from the tail, from the rare events, but because it captured more diverse features from the mode. And in the mode you have equal diversity because you are still experiencing the same crazy world. Is just that the long tale of actions does not correspond to the long tale of things that happen in the world.