AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
The Importance of Predicting the Next Token
muri's visible thoughts project was an attempt to build a data set that would encourage Large language models to think out loud where we could see them by recording humans thinking about out loud. To predict a human talking about their plans you have to predict the human's planning process That means that somewhere in the giant inscrutable vectors of floating point numbers There is the ability to plan because it is predicting a human planning so As much capability as appears in its outputs It's got to have that much capability internally even if it's operating under the handicap of gpt. The cognitive capacity to do the thing you think it can't do is clearly in there somewhere Would would be the thing I would say