The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

Assessing the Risks of Open AI Models with Sayash Kapoor - #675

Mar 11, 2024

Sayash Kapoor, a Ph.D. student at Princeton University, discusses his research on the societal impact of open foundation models. He highlights the controversies surrounding AI safety and the potential risks of releasing model weights. The conversation delves into critical issues, such as biosecurity concerns linked to language models and the challenges of non-consensual imagery in AI. Kapoor advocates for a unified framework to evaluate these risks, emphasizing the need for transparency and legal protections in AI development.

40:26

Creator website

Episode guests

Sayash Kapoor

AI Summary

Highlights

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

Establishing common ground for assessing AI risks is crucial for mitigating potential harms of open models.

The risks of open model weights, including biosecurity concerns, highlight the need for structured risk assessment frameworks.

Deep dives

The Importance of Open Foundation Models

The podcast episode delves into the societal impact of open foundation models, exploring the motivations behind a recent paper's analysis. By focusing on the risks and benefits of openness in AI, the researchers aimed to create a framework that clarifies misconceptions and fosters constructive debate about the impacts of open models. Emphasizing the necessity for a common ground among experts from various sectors, the paper highlights the significance of addressing risks collectively and mitigating potential harms of open foundation models.

The Value of Openness in AI Research

01:00

Supreme Court says that virtual pornography is protected under the First Amendment

01:01

Intro

4min

Navigating the Open AI Landscape

8min

Evaluating the Impact of Language Models on Biosecurity Risks

2min

Assessing Marginal Risks in Open Foundation Models

3min

Navigating the Risks of Non-Consensual Imagery

9min

Risk Analysis in Open AI Models

14min

Today we’re joined by Sayash Kapoor, a Ph.D. student in the Department of Computer Science at Princeton University. Sayash walks us through his paper: "On the Societal Impact of Open Foundation Models.” We dig into the controversy around AI safety, the risks and benefits of releasing open model weights, and how we can establish common ground for assessing the threats posed by AI. We discuss the application of the framework presented in the paper to specific risks, such as the biosecurity risk of open LLMs, as well as the growing problem of "Non Consensual Intimate Imagery" using open diffusion models.

The complete show notes for this episode can be found at twimlai.com/go/675.

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more