LessWrong (30+ Karma) “Research Reflections” by abramdemski
Nov 4, 2025
Abram Dembski, an AI-safety researcher and writer on LessWrong, shares insights from his decade in AI safety. He discusses a notable paradigm shift experienced at the Agent Foundations conference, igniting actionable conversations. Dembski highlights the surprising connections from his newest paper, merging ideas from finite factored sets with agent boundaries. He evaluates Sam Eisenstadt's work on natural abstraction, comparing modeling approaches and articulating the potential for unified abstraction theories, signaling exciting future developments in research.
AI Snips
Chapters
Transcript
Episode notes
From Divergence To A Concrete Shift
- Abram Dembski describes long-term divergence in AI safety collaborations and how partnerships drift apart over time.
- He recounts a concrete shift at the Agent Foundations conference where a discussion with Scott Garrabrant felt unusually actionable and unifying.
Paper Writing Reveals Unexpected Synthesis
- Writing his Iliad paper let Abram unexpectedly synthesize multiple ideas into a coherent research direction.
- Scott Garrabrant's finite factored sets became urgently interesting and actionable within Abram's work.
Linking Formal Frameworks To Agent Boundaries
- Abram connects Scott Garrabrant's finite factored sets and Cartesian frames to his own work, seeing them as a beginning of calculations he wants to complete.
- He also integrates ideas from Critch on agent boundaries to expand the framework's scope.