Is Big Bench Really Going to Be the Driving Force in NLP?

Derek Kite: There's been some contrast between Big Bench and Alan Turing's imitation game, which is sort of the gold standard for determining if a machine bears the property of having artificial general intelligence. Right now in NLP, we train these huge models, right? We train them and they are really great, but they're essentially just snapshots. So I think this is one big challenge that we have to come over. It will be interesting to see how the updating or retraining process becomes more common place.