LessWrong (Curated & Popular) “Condensation” by abramdemski
35 snips
Nov 12, 2025 Dive into a captivating exploration of Sam Eisenstat's condensation theory, which reimagines concept formation by introducing interpretable latent variables. Discover how it compares to Shannon's information theory and algorithmic codes, focusing on minimizing code length for effective data organization. The discussion covers the notebook analogy for retrieval costs and the intriguing distinction between top and bottom latents, shedding light on shared structures versus individual noise. An insightful look at potential research directions rounds off this thought-provoking journey.
AI Snips
Chapters
Books
Transcript
Episode notes
Representation Over Raw Compression
- Condensation optimizes representations for answering queries rather than for maximal compression.
- This yields organized latent variables that form meaningful, translatable concepts across observers.
Data Structure For Query Efficiency
- View condensation as a universal data structure that organizes data into tagged bins for efficient retrieval.
- It prioritizes retrieval cost over storage, producing reusable abstractions across queries.
Information Desk Notebook Example
- Abram uses an information-desk and reference-notebook story to illustrate condensation practically.
- Clerks retrieve all notebooks tagged with requested questions and retrieval cost equals total text retrieved.
