LessWrong (Curated & Popular)

“Training AGI in Secret would be Unsafe and Unethical” by Daniel Kokotajlo

6 snips
Apr 21, 2025
The dangers of developing Artificial General Intelligence in secrecy take center stage. It explores the risks of power concentration and the significant loss of control that could ensue. Emphasizing transparency and public engagement, the discussion warns about the creation of misaligned AGI systems. With AGI potentially being trained within this decade, the urgency of addressing these ethical considerations is highlighted. Listeners are encouraged to reconsider their assumptions about the feasibility and ramifications of AGI.
Ask episode
AI Snips
Chapters
Books
Transcript
Episode notes
INSIGHT

AGI Secrecy Risks

  • AGI will likely be kept secret for months after creation within a tight internal circle.
  • Such secrecy increases risks in understanding and controlling AGI capabilities safely.
INSIGHT

Danger of Concentrated Power

  • A tiny elite group will handle critical safety judgments and decisions on AGI alignment.
  • This concentration risks misjudgment and catastrophic outcomes due to limited scrutiny.
INSIGHT

Alignment Failures Impact

  • Two kinds of alignment failures exist: those concealing their failure and those that don't.
  • Even a single hidden failure in a superhuman AGI could be catastrophic.
Get the Snipd Podcast app to discover more snips from this episode
Get the app