The Inside View cover image

Collin Burns On Discovering Latent Knowledge In Language Models Without Supervision

The Inside View

CHAPTER

Using Unsupervised Models to Predict Future Text

I think what our results show is something like suppose suppose futures for human language models actually do into to least speaking think about is this input true or false and they represent that in a way that is analogous to how current language models represent is this inputtrue or false. So if this is represented in a similar way in future models uh then I think our method has a reasonable chance of finding that feature um now I think you would probably need to tweak it for example to make sure it converges to that solution rather than say what a human would say but there are some cases where the model will output true text.

00:00
Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner