The Importance of Text in AI Training

The data that it's trained on is 40 plus billion tokens, right? It's just internet text. Some tasks are going to be very solvable, because they're hugely represented in the training set. And some tasks are really niche,. And they probably don't know which ones are niche, because you haven't seen the trainingSet. I say, right me a link expression, and it does it really well. But when I say,right me a linking expression using some other thing, and that isn't in a training set, and it produces me a wrong answer. So we have to think about what kind of questions do we want to completely take a human out of a loop there

Transcript

Play full episode

Transcript

Episode notes

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!

Get the app