AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Are the Instructions Really Confusional?
Many crowd workers don't really think a lot about the instructions and what the task creators want to achieve. They just quickly look at the examples and do the most similar thing they can come up with. So if the instructions include all kinds of biases, this will just naturally propagate to the behavior of the annotator. But for, you know, tense disambiguation or some of these NLP tasks, surely there's enough quality in the workers to get something out of it. Or am I too optimistic? No, no. It all depends on the nature of like the task.