LessWrong (30+ Karma)

“In Defense of Goodness” by abramdemski

Nov 20, 2025
The discussion dives into the distinction between goodness and human values, arguing that they are not the same. It explores the concept of goodness as a collective tool for societal coordination. C.S. Lewis's ideas are brought in to illustrate a philosophical dialogue about moral understanding. The podcast also examines how personal values are shaped through experience and reward mechanisms. Finally, it emphasizes that goodness transcends just human values, encompassing concern for future beings as well.
Ask episode
AI Snips
Chapters
Books
Transcript
Episode notes
INSIGHT

Values Are Aggregated Sub-Agents

  • Abram Dembski argues personal values are not a single unified signal but an aggregate of multiple reinforcement systems and social influences.
  • Modeling humans as multiple sub-agents explains conflicting urges and deliberate reward-shaping choices.
ADVICE

Avoid Reward-Hacking Your Values

  • Deliberately avoid certain reinforcers to prevent reward-hacking from reshaping your values.
  • Shape future yumminess by steering experiences, not just following immediate appetites.
INSIGHT

Goodness As Negotiated Aggregate

  • Goodness functions like an aggregated, negotiated norm rather than a mere sum of individual preferences.
  • The word "good" tracks collective policies and becomes an objective compromise by construction.
Get the Snipd Podcast app to discover more snips from this episode
Get the app