AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
The Power of Predicting Words
A neural network is a complicated piece of linear algebra that uses many matrices and weights to produce a, a distribution over tokens. It's simpler to think of it if it is just pretend that a token is a word, right? And this is the form that I think most people can sort of understand intuitively from like their experience with like autocomplete on their phone, right? That you can imagine that there's some process that just like looks over all the text that I've typed so far, and then sees what words are likely to follow them.