The Data Exchange Podcast Is a Property of Gradient Flow

Roy Schwartz: Most of my research focused almost exclusively on English because the most of the data sets and benchmarks are in English. There's actually NLP researchers who are working on languages where there's just not that much data, right? Exactly. And the question is how much, I mean, efficiency is super relevant. If you don't have enough data, then you can't pre-train forever.