AI Safety Fundamentals

By Samuel HammondSource: https://www.secondbest.ca/p/ai-and-leviathan-part-iA podcast by BlueDot Impact.Learn more on the AI Safety Fundamentals website.

By Vitalik ButerinEthereum founder Vitalik Buterin describes how democratic, defensive and decentralised technologies could distribute AI's power across society rather than concentrating it, offering a middle path between unchecked technical acceleration and authoritarian control.Source:https://vitalik.eth.limo/general/2025/01/05/dacc2.htmlA podcast by BlueDot Impact.Learn more on the AI Safety Fundamentals website.

By Sella Nevo et al.In this report, RAND researchers identify real-world attack methods that malicious actors could use to steal AI model weights. They propose a five-level security framework that AI companies could implement to defend against different threats, from amateur hackers to nation-state operations.Source:https://www.rand.org/pubs/research_briefs/RBA2849-1.htmlA podcast by BlueDot Impact.Learn more on the AI Safety Fundamentals website.

By Akash Wasil et al.This paper uses scenario planning to show how governments could prepare for AI emergencies. The authors examine three plausible disasters: losing control of AI, AI model theft, and bioweapon creation. They then expose gaps in current preparedness systems, and propose specific government reforms including embedding auditors inside AI companies and creating emergency response units.Source:https://arxiv.org/pdf/2407.17347A podcast by BlueDot Impact.Learn more on the AI Safety Fundamentals website.

By Jamie BernardiJamie Bernardi argues that we can't rely solely on model safeguards to ensure AI safety. Instead, he proposes "AI resilience": building society's capacity to detect misuse, defend against harmful AI applications, and reduce the damage caused when dangerous AI capabilities spread beyond a government or company's control.Source: https://airesilience.substack.com/p/resilience-and-adaptation-to-advanced?utm_source=bluedot-impactA podcast by BlueDot Impact.Learn more on the AI Safety Fundamentals website.

By Leopold AschenbrennerA former OpenAI researcher argues that private AI companies cannot safely develop superintelligence due to security vulnerabilities and competitive pressures that override safety. He argues that a government-led 'AGI Project' is inevitable and necessary to prevent adversaries stealing the AI systems, or losing human control over the technology.Source:https://situational-awareness.ai/the-project/?utm_source=bluedot-impactA podcast by BlueDot Impact.Learn more on the AI Safety Fundamentals website.

By Sarah Hastings-WoodhouseAI Control is a research agenda that aims to prevent misaligned AI systems from causing harm. It is different from AI alignment, which aims to ensure that systems act in the best interests of their users. Put simply, aligned AIs do not want to harm humans, whereas controlled AIs can't harm humans, even if they want to.Source:https://bluedot.org/blog/ai-controlA podcast by BlueDot Impact.Learn more on the AI Safety Fundamentals website.

By Yoshua Bengio et al.This paper argues that building generalist AI agents poses catastrophic risks, from misuse by bad actors to a potential loss of human control. As an alternative, the authors propose “Scientist AI,” a non-agentic system designed to explain the world through theory generation and question-answering rather than acting in it. They suggest this path could accelerate scientific progress, including in AI safety, while avoiding the dangers of agency-driven AI.Source:https://arxiv.org/pdf/2502.15657A podcast by BlueDot Impact.Learn more on the AI Safety Fundamentals website.

By Luke Drago and Rudolf LaineThis section explores how the arrival of AGI could trigger an “intelligence curse,” where automation of all work removes incentives for states and companies to care about ordinary people. It frames the trillion-dollar race toward AGI as not just an economic shift, but a transformation in power dynamics and human relevance.Source:https://intelligence-curse.ai/?utm_source=bluedot-impactA podcast by BlueDot Impact.Learn more on the AI Safety Fundamentals website.

 By Luke Drago and Rudolf LaineThis piece explores key arguments from sections 3 and 4 of The Intelligence Curse, continuing the authors’ analysis of how increasing intelligence can create paradoxical disadvantages, tradeoffs, and coordination challenges.Source:https://intelligence-curse.ai/A podcast by BlueDot Impact.Learn more on the AI Safety Fundamentals website.