AI Safety Newsletter

AI labs like OpenAI and Meta fail to share models with UK's AI Safety Institute, while Google DeepMind complies. Bipartisan AI policy proposals in the US Senate and discussions on military AI in Israel and the US. New online course on AI safety from CAIS. Updates on AI regulation, security, consciousness, and debates.

Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required. This week, we cover: Consolidation in the corporate AI landscape, as smaller startups join forces with larger funders. Several countries have announced new investments in AI, including Singapore, Canada, and Saudi Arabia. Congress's budget for 2024 provides some but not all of the requested funding for AI policy. The White House's 2025 proposal makes more ambitious requests for AI funding. How will AI affect biological weapons risk? We reexamine this question in light of new experiments from RAND, OpenAI, and others. AI Startups Seek Support From Large Financial Backers As AI development demands ever-increasing compute resources, only well-resourced developers can compete at the frontier. In practice, this means that AI startups must either partner with the world's [...] ---Outline:(00:45) AI Startups Seek Support From Large Financial Backers(03:47) National AI Investments(05:16) Federal Spending on AI(08:35) An Updated Assessment of AI and Biorisk(15:35) $250K in Prizes: SafeBench Competition Announcement(16:08) Links--- First published: April 11th, 2024 Source: https://newsletter.safe.ai/p/ai-safety-newsletter-33-reassessing --- Want more? Check out our ML Safety Newsletter for technical safety research. Narrated by TYPE III AUDIO.

Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required. Measuring and Reducing Hazardous Knowledge The recent White House Executive Order on Artificial Intelligence highlights risks of LLMs in facilitating the development of bioweapons, chemical weapons, and cyberweapons. To help measure these dangerous capabilities, CAIS has partnered with Scale AI to create WMDP: the Weapons of Mass Destruction Proxy, an open source benchmark with more than 4,000 multiple choice questions that serve as proxies for hazardous knowledge across biology, chemistry, and cyber. This benchmark not only helps the world understand the relative dual-use capabilities of different LLMs, but it also creates a path forward for model builders to remove harmful information from their models through machine unlearning techniques. Measuring hazardous knowledge in bio, chem, and cyber. Current evaluations of dangerous AI capabilities have [...] ---Outline:(00:03) Measuring and Reducing Hazardous Knowledge(04:35) Language models are getting better at forecasting(07:51) Proposals for Private Regulatory Markets(14:25) Links--- First published: March 7th, 2024 Source: https://newsletter.safe.ai/p/ai-safety-newsletter-32-measuring --- Want more? Check out our ML Safety Newsletter for technical safety research. Narrated by TYPE III AUDIO.

Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required. This week, we’ll discuss: A new proposed AI bill in California which requires frontier AI developers to adopt safety and security protocols, and clarifies that developers bear legal liability if their AI systems cause unreasonable risks or critical harms to public safety. Precedents for AI governance from healthcare and biosecurity. The EU AI Act and job opportunities at their enforcement agency, the AI Office. A New Bill on AI Policy in California Several leading AI companies have public plans for how they’ll invest in safety and security as they develop more dangerous AI systems. A new bill in California's state legislature would codify this practice as a legal requirement, and clarify the legal liability faced by developers [...] ---Outline:(00:33) A New Bill on AI Policy in California(04:38) Precedents for AI Policy: Healthcare and Biosecurity(07:56) Enforcing the EU AI Act(08:55) Links--- First published: February 21st, 2024 Source: https://newsletter.safe.ai/p/aisn-31-a-new-ai-policy-bill-in-california --- Want more? Check out our ML Safety Newsletter for technical safety research. Narrated by TYPE III AUDIO.

Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required. Compute Investments Continue To Grow Pausing AI development has been proposed as a policy for ensuring safety. For example, an open letter last year from the Future of Life Institute called for a six-month pause on training AI systems more powerful than GPT-4. But one interesting fact about frontier AI development is that it comes with natural pauses that can last many months or years. After releasing a frontier model, it takes time for AI developers to construct new compute clusters with larger numbers of more advanced computer chips. The supply of compute is currently unable to keep up with demand, meaning some AI developers cannot buy enough chips for their needs. This explains why OpenAI was reportedly limited by GPUs last year. [...] ---Outline:(00:06) Compute Investments Continue To Grow(03:48) Developments in Military AI(07:19) Japan and Singapore Support AI Safety(08:57) Links--- First published: January 24th, 2024 Source: https://newsletter.safe.ai/p/aisn-30-investments-in-compute-and --- Want more? Check out our ML Safety Newsletter for technical safety research. Narrated by TYPE III AUDIO.

Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required. A Provisional Agreement on the EU AI Act On December 8th, the EU Parliament, Council, and Commission reached a provisional agreement on the EU AI Act. The agreement regulates the deployment of AI in high risk applications such as hiring and credit pricing, and it bans private companies from building and deploying AI for unacceptable applications such as social credit scoring and individualized predictive policing. Despite lobbying by some AI startups against regulation of foundation models, the agreement contains risk assessment and mitigation requirements for all general purpose AI systems. Specific requirements apply to AI systems trained with >1025 FLOP such as Google's Gemini and OpenAI's GPT-4. Minimum basic transparency requirements for all GPAI. The provisional agreement regulates foundation models — using the [...] ---Outline:(00:06) A Provisional Agreement on the EU AI Act(04:55) Questions about Research Standards in AI Safety(06:48) The New York Times sues OpenAI and Microsoft for Copyright Infringement(10:34) Links--- First published: January 4th, 2024 Source: https://newsletter.safe.ai/p/aisn-29-progress-on-the-eu-ai-act --- Want more? Check out our ML Safety Newsletter for technical safety research. Narrated by TYPE III AUDIO.

Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required. This week we’re looking closely at AI legislative efforts in the United States, including: Senator Schumer's AI Insight Forum The Blumenthal-Hawley framework for AI governance Agencies proposed to govern digital platforms State and local laws against AI surveillance The National AI Research Resource (NAIRR) Senator Schumer's AI Insight Forum The CEOs of more than a dozen major AI companies gathered in Washington on Wednesday for a hearing with the Senate. Organized by Democratic Majority Leader Chuck Schumer and a bipartisan group of Senators, this was the first of many hearings in their AI Insight Forum. After the hearing, Senator Schumer said, “I asked everyone in the room, ‘Is government needed to play a role in regulating AI?’ and [...] ---Outline:(00:30) Senator Schumer's AI Insight Forum(01:20) The Blumenthal-Hawley Framework(03:09) Agencies Proposed to Govern Digital Platforms(04:46) Deepfakes and Watermarking Legislation(06:12) State and Local Laws Against AI Surveillance(06:52) National AI Research Resource (NAIRR)(08:18) Links--- First published: September 19th, 2023 Source: https://newsletter.safe.ai/p/the-landscape-of-us-ai-legislation --- Want more? Check out our ML Safety Newsletter for technical safety research. Narrated by TYPE III AUDIO.

As 2023 comes to a close, we want to thank you for your continued support for AI safety. This has been a big year for AI and for the Center for AI Safety. In this special-edition newsletter, we highlight some of our most important projects from the year. Thank you for being part of our community and our work. Center for AI Safety's 2023 Year in Review The Center for AI Safety (CAIS) is on a mission to reduce societal-scale risks from AI. We believe this requires research and regulation. These both need to happen quickly (due to unknown timelines on AI progress) and in tandem (because either one is insufficient on its own). To achieve this, we pursue three pillars of work: research, field-building, and advocacy. Research CAIS conducts both technical and conceptual research on AI safety. We pursue multiple overlapping strategies which can be layered together [...] ---Outline:(00:27) Center for AI Safety's 2023 Year in Review(00:56) Research(03:37) Field-Building(07:35) Advocacy(10:04) Looking Ahead(10:23) Support Our Work--- First published: December 21st, 2023 Source: https://newsletter.safe.ai/p/aisn-28-center-for-ai-safety-2023 --- Want more? Check out our ML Safety Newsletter for technical safety research. Narrated by TYPE III AUDIO.

Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required. Defensive Accelerationism  Vitalik Buterin, the creator of Ethereum, recently wrote an essay on the risks and opportunities of AI and other technologies. He responds to Marc Andreessen's manifesto on techno-optimism and the growth of the effective accelerationism (e/acc) movement, and offers a more nuanced perspective. Technology is often great for humanity, the essay argues, but AI could be an exception to that rule. Rather than giving governments control of AI so they can protect us, Buterin argues that we should build defensive technologies that provide security against catastrophic risks in a decentralized society. Cybersecurity, biosecurity, resilient physical infrastructure, and a robust information ecosystem are some of the technologies Buterin believes we should build to protect ourselves from AI risks. Technology has risks, but [...] ---Outline:(00:06) Defensive Accelerationism(03:55) Retrospective on the OpenAI Board Saga(07:58) Klobuchar and Thune's “light-touch” Senate bill(10:23) Links--- First published: December 7th, 2023 Source: https://newsletter.safe.ai/p/aisn-27-defensive-accelerationism --- Want more? Check out our ML Safety Newsletter for technical safety research. Narrated by TYPE III AUDIO.

Also, Results From the UK Summit, and New Releases From OpenAI and xAI. Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required.This week's key stories include: The UK, US, and Singapore have announced national AI safety institutions. The UK AI Safety Summit concluded with a consensus statement, the creation of an expert panel to study AI risks, and a commitment to meet again in six months. xAI, OpenAI, and a new Chinese startup released new models this week. UK, US, and Singapore Establish National AI Safety InstitutionsBefore regulating a new technology, governments often need time to gather information and consider their policy options. But during that time, the technology may diffuse through society, making it more difficult for governments to intervene. This process, termed the Collingridge Dilemma, is a fundamental challenge in technology policy.But recently [...] ---Outline:(00:36) UK, US, and Singapore Establish National AI Safety Institutions(03:53) UK Summit Ends with Consensus Statement and Future Commitments(05:39) New Models From xAI, OpenAI, and a New Chinese Startup(09:28) Links--- First published: November 15th, 2023 Source: https://newsletter.safe.ai/p/national-institutions-for-ai-safety --- Want more? Check out our ML Safety Newsletter for technical safety research. Narrated by TYPE III AUDIO.