Rohin Shah
TalkRL: The Reinforcement Learning Podcast
00:00
Test Distributions
The challenge of revusness is usually that you don't know what your test distribution is going to look like. We try to deliberately o find situations there, outside the training situation, but where a human would agree that one unambiguously correct answer a and tested in those cases. But even that we it seems very hard to know that. I think, ah, it's still a good idea to be driving these tests and to take failures fairly seriously.
Transcript
Play full episode
