Paul Christiano runs the Alignment Research Center, a non-profit research organization whose mission is to align future machine learning systems with human interests. Paul previously ran the language model alignment team at OpenAI, the creators of ChatGPT.  Today, we’re hoping to explore the solution-landscape to the AI Alignment problem, and hoping Paul can guide us on that journey.  ------ ✨ DEBRIEF | Unpacking the episode:  https://www.bankless.com/debrief-paul-christiano    ------ ✨ COLLECTIBLES | Collect this episode:  https://collectibles.bankless.com/mint  ------ ✨ Always wanted to become a Token Analyst? Bankless Citizens get exclusive access to Token Hub. Join Them. https://bankless.cc/TokenHubRSS   ------ In today’s episode, Paul answers many questions, but the overarching ones are:  1) How BIG is the AI Alignment problem?  2) How HARD is the AI Alighment problem? 3) How SOLVABLE is the AI Alignment problem?  Does humanity have a chance? Tune in to hear Paul’s thoughts.  ------ BANKLESS SPONSOR TOOLS:  ⚖️ ARBITRUM | SCALING ETHEREUM https://bankless.cc/Arbitrum  🐙KRAKEN | MOST-TRUSTED CRYPTO EXCHANGE https://bankless.cc/kraken  🦄UNISWAP | ON-CHAIN MARKETPLACE https://bankless.cc/uniswap  👻 PHANTOM | FRIENDLY MULTICHAIN WALLET https://bankless.cc/phantom-waitlist  🦊METAMASK LEARN | HELPFUL WEB3 RESOURCE https://bankless.cc/MetaMask  ------ Topics Covered 0:00 Intro 9:20 Percentage Likelihood of Death by AI 11:24 Timing  19:15 Chimps to Human Jump 21:55 Thoughts on ChatGPT 27:51 LLMs & AGI 32:49 Time to React? 38:29 AI Takeover  41:51 AI Agency  49:35 Loopholes  51:14 Training AIs to Be Honest  58:00 Psychology  59:36 How Solvable Is the AI Alignment Problem? 1:03:48 The Technical Solutions (Scalable Oversight)  1:16:14 Training AIs to be Bad?!  1:18:22 More Solutions 1:21:36 Stabby AIs  1:26:03 Public vs. Private (Lab) AIs 1:28:31 Inside Neural Nets 1:32:11 4th Solution  1:35:00 Manpower & Funding  1:38:15 Pause AI? 1:43:29 Resources & Education on AI Safety  1:46:13 Talent   1:49:00 Paul’s Day Job 1:50:15 Nobel Prize  1:52:35 Treating AIs with Respect  1:53:41 Uptopia Scenario  1:55:50 Closing & Disclaimers  ------ Resources: Alignment Research Center https://www.alignment.org/  Paul Christiano’s Website https://paulfchristiano.com/ai/  ----- Not financial or tax advice. This channel is strictly educational and is not investment advice or a solicitation to buy or sell any assets or to make any financial decisions. This video is not tax advice. Talk to your accountant. Do your own research. Disclosure. From time-to-time I may add links in this newsletter to products I use. I may receive commission if you make a purchase through one of these links. Additionally, the Bankless writers hold crypto assets. See our investment disclosures here: https://www.bankless.com/disclosures 

Paul Christiano, world's leading AI safety researcher, discusses regretting inventing RLHF, modest timelines for AGI development, post-AGI world vision, his research solving alignment as a major discovery, push for responsible scaling policies, preventing an AI coup or bioweapon, and more.

Why would advanced AI systems pose an existential risk, and what would it look like to develop safer systems? In this episode, I interview Paul Christiano about his views of how AI could be so dangerous, what bad AI scenarios could look like, and what he thinks about various techniques to reduce this risk.   Topics we discuss, and timestamps:  - 00:00:38 - How AI may pose an existential threat    - 00:13:36 - AI timelines    - 00:24:49 - Why we might build risky AI    - 00:33:58 - Takeoff speeds    - 00:51:33 - Why AI could have bad motivations    - 00:56:33 - Lessons from our current world    - 01:08:23 - "Superintelligence"  - 01:15:21 - Technical causes of AI x-risk    - 01:19:32 - Intent alignment    - 01:33:52 - Outer and inner alignment    - 01:43:45 - Thoughts on agent foundations  - 01:49:35 - Possible technical solutions to AI x-risk    - 01:49:35 - Imitation learning, inverse reinforcement learning, and ease of evaluation    - 02:00:34 - Paul's favorite outer alignment solutions      - 02:01:20 - Solutions researched by others      - 2:06:13 - Decoupling planning from knowledge    - 02:17:18 - Factored cognition    - 02:25:34 - Possible solutions to inner alignment  - 02:31:56 - About Paul    - 02:31:56 - Paul's research style    - 02:36:36 - Disagreements and uncertainties    - 02:46:08 - Some favorite organizations    - 02:48:21 - Following Paul's work   The transcript: axrp.net/episode/2021/12/02/episode-12-ai-xrisk-paul-christiano.html   Paul's blog posts on AI alignment: ai-alignment.com   Material that we mention:  - Cold Takes - The Most Important Century: cold-takes.com/most-important-century  - Open Philanthropy reports on:    - Modeling the human trajectory: openphilanthropy.org/blog/modeling-human-trajectory    - The computational power of the human brain: openphilanthropy.org/blog/new-report-brain-computation    - AI timelines (draft): alignmentforum.org/posts/KrJfoZzpSDpnrv9va/draft-report-on-ai-timelines    - Whether AI could drive explosive economic growth: openphilanthropy.org/blog/report-advanced-ai-drive-explosive-economic-growth  - Takeoff speeds: sideways-view.com/2018/02/24/takeoff-speeds  - Superintelligence: Paths, Dangers, Strategies: en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies  - Wei Dai on metaphilosophical competence:    - Two neglected problems in human-AI safety: alignmentforum.org/posts/HTgakSs6JpnogD6c2/two-neglected-problems-in-human-ai-safety    - The argument from philosophical difficulty: alignmentforum.org/posts/w6d7XBCegc96kz4n3/the-argument-from-philosophical-difficulty    - Some thoughts on metaphilosophy: alignmentforum.org/posts/EByDsY9S3EDhhfFzC/some-thoughts-on-metaphilosophy  - AI safety via debate: arxiv.org/abs/1805.00899  - Iterated distillation and amplification: ai-alignment.com/iterated-distillation-and-amplification-157debfd1616  - Scalable agent alignment via reward modeling: a research direction: arxiv.org/abs/1811.07871  - Learning the prior: alignmentforum.org/posts/SL9mKhgdmDKXmxwE4/learning-the-prior  - Imitative generalisation (AKA 'learning the prior'): alignmentforum.org/posts/JKj5Krff5oKMb8TjT/imitative-generalisation-aka-learning-the-prior-1  - When is unaligned AI morally valuable?: ai-alignment.com/sympathizing-with-ai-e11a4bf5ef6e

Paul Christiano, an expert in AI, discusses various intriguing topics like the gradual transformation of the world by AI, methods for ensuring AI compliance, granting legal rights to AI systems, and the obsolescence of human labor. He also touches on AI's impact on science research and the timeline for human labor becoming obsolete.

In this enlightening discussion, Paul Christiano, known for popularizing the term 'slow takeoff' in AI development, sheds light on the confusion surrounding AI timelines. He argues that 'slow' should not imply longer calendar periods but rather smoother, continuous progress, which can actually accelerate development. Paul advocates for clearer terminology to enhance communication in the tech community, ensuring that policymakers and innovators can better navigate the complexities of AI advancements.