“Ethical Design Patterns” by AnnaSalamon
LessWrong (Curated & Popular)
00:00
Designing institutions to stay true to purpose
Anna details ethical design choices—clear mission, incentives, transparency—that make institutions robust and aligned.
Play episode from 09:37
Transcript
Transcript
Episode notes
Related to: Commonsense Good, Creative Good (and my comment); Ethical Injunctions.
Epistemic status: I’m fairly sure “ethics” does useful work in building human structures that work. My current explanations of how are wordy and not maximally coherent; I hope you guys help me with that.
Introduction
It is intractable to write large, good software applications via spaghetti code – but it's comparatively tractable using design patterns (plus coding style, attention to good/bad codesmell, etc.).
I’ll argue it is similarly intractable to have predictably positive effects on large-scale human stuff if you try it via straight consequentialism – but it is comparatively tractable if you use ethical heuristics, which I’ll call “ethical design patterns,” to create situations that are easier to reason about. Many of these heuristics are honed by long tradition (eg “tell the truth”; “be kind”), but sometimes people successfully craft new “ethical design patterns” fitted to a [...]
---
Outline:
(00:31) Introduction
(01:32) Intuitions and ground truth in math, physics, coding
(02:08) We revise our intuitions to match the world. Via deliberate work.
(03:08) We design our built world to be intuitively accessible
(04:22) Intuitions and ground truth in ethics
(04:52) We revise our ethical intuitions to predict which actions we'll be glad of, long-term
(06:27) Ethics helps us build navigable human contexts
(09:30) We use ethical design patterns to create institutions that can stay true to a purpose
(12:17) Ethics as a pattern language for aligning mesaoptimizers
(13:08) Examples: several successfully crafted ethical heuristics, and several gaps
(13:15) Example of a well-crafted ethical heuristic: Don't drink and drive
(14:45) Example of well-crafted ethical heuristic: Earning to give
(15:10) A partial example: YIMBY
(16:24) A historical example of gap in folks' ethical heuristics: Handwashing and childbed fever
(19:46) A contemporary example of inadequate ethical heuristics: Public discussion of group differences
(25:04) Gaps in our current ethical heuristics around AI development
(26:30) Existing progress
(28:30) Where we still need progress
(32:21) Can we just ignore the less-important heuristics, in favor of 'don't die'?
(35:02) These gaps are in principle bridgeable
(36:29) Related, easier work
The original text contained 12 footnotes which were omitted from this narration.
---
First published:
September 30th, 2025
Source:
https://www.lesswrong.com/posts/E9CyhJWBjzoXritRJ/ethical-design-patterns-1
---
Narrated by TYPE III AUDIO.
Epistemic status: I’m fairly sure “ethics” does useful work in building human structures that work. My current explanations of how are wordy and not maximally coherent; I hope you guys help me with that.
Introduction
It is intractable to write large, good software applications via spaghetti code – but it's comparatively tractable using design patterns (plus coding style, attention to good/bad codesmell, etc.).
I’ll argue it is similarly intractable to have predictably positive effects on large-scale human stuff if you try it via straight consequentialism – but it is comparatively tractable if you use ethical heuristics, which I’ll call “ethical design patterns,” to create situations that are easier to reason about. Many of these heuristics are honed by long tradition (eg “tell the truth”; “be kind”), but sometimes people successfully craft new “ethical design patterns” fitted to a [...]
---
Outline:
(00:31) Introduction
(01:32) Intuitions and ground truth in math, physics, coding
(02:08) We revise our intuitions to match the world. Via deliberate work.
(03:08) We design our built world to be intuitively accessible
(04:22) Intuitions and ground truth in ethics
(04:52) We revise our ethical intuitions to predict which actions we'll be glad of, long-term
(06:27) Ethics helps us build navigable human contexts
(09:30) We use ethical design patterns to create institutions that can stay true to a purpose
(12:17) Ethics as a pattern language for aligning mesaoptimizers
(13:08) Examples: several successfully crafted ethical heuristics, and several gaps
(13:15) Example of a well-crafted ethical heuristic: Don't drink and drive
(14:45) Example of well-crafted ethical heuristic: Earning to give
(15:10) A partial example: YIMBY
(16:24) A historical example of gap in folks' ethical heuristics: Handwashing and childbed fever
(19:46) A contemporary example of inadequate ethical heuristics: Public discussion of group differences
(25:04) Gaps in our current ethical heuristics around AI development
(26:30) Existing progress
(28:30) Where we still need progress
(32:21) Can we just ignore the less-important heuristics, in favor of 'don't die'?
(35:02) These gaps are in principle bridgeable
(36:29) Related, easier work
The original text contained 12 footnotes which were omitted from this narration.
---
First published:
September 30th, 2025
Source:
https://www.lesswrong.com/posts/E9CyhJWBjzoXritRJ/ethical-design-patterns-1
---
Narrated by TYPE III AUDIO.
The AI-powered Podcast Player
Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
