
Episode 01: Kelvin Guu, Google AI, on language models & overlooked research problems
Generally Intelligent
00:00
Getting a Negative Infinity Score From a Human Judge?
The opening i won and then is avoiding that by giving a different training signal where there's no more negative infinities. The truncatin the law, so you don't have negative infinity likes really problematic,. cause your vocabulary is gigantic. Wyhow you have to put so much probability all over the place sohat you don't get infinitely wrong? Yes, i think this reminds me of contrastive losses. If you can pick your negative candidates carefully, in a contrastive loss you can emphasize the thing that was important about the right thing, while also finding the boundary of everything that's invalid.
Transcript
Play full episode