AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
How to Scale a Task Like MNIST?
In order to come up with a task, the only thing you can measure is skill at a task. All tasks are going to involve priors. The trick is to know what they are and to describe that. And then you make sure that this is the same set of priors as what human staff is. That all, you know, each task should be new to the agent passing it. Also should be human interpretable and the sonable so that you can actually have a human pass the same test.