LessWrong (Curated & Popular)
“Connecting the Dots: LLMs can Infer & Verbalize Latent Structure from Training Data” by Johannes Treutlein, Owain_Evans
Jun 23, 2024
Researcher Johannes Treutlein and ML expert Owain Evans discuss LLMs' ability to infer latent information for tasks like defining functions and predicting city names without in-context learning. They showcase how LLMs can carry out tasks by leveraging training data without explicit reasoning.
17:56
Episode guests
AI Summary
AI Chapters
Episode notes
Podcast summary created with Snipd AI
Quick takeaways
- LLMs can infer latent information from training data for downstream tasks without in-context learning, demonstrating out-of-context reasoning capabilities.
- Inductive out-of-context reasoning in LLMs raises AI safety concerns due to unmonitored acquisition of sensitive information and potential risks of deception.
Deep dives
Inductive Out-of-Context Reasoning in LLMs
LLMs can infer latent information from training data and utilize it for downstream tasks without in-context learning. Experimental results show that LLMs, fine-tuned on specific data like distances between cities, can deduce latent information such as the identity of unknown cities like Paris. Although effective in some cases, inductive out-of-context reasoning (OOCR) is shown to be unreliable, particularly with smaller LLMs tackling complex structures.
