Exploring Harmfulness Monitoring in AI Models

This chapter examines three case studies focused on harmfulness monitoring in AI models, highlighting the use of supervised probes to identify harmful queries and responses. It discusses input space perturbation tactics and the technical intricacies of probes, such as multi-layer perceptrons, to enhance the understanding of their role in detecting harmful outputs.

Transcript

chevron_right

Play full episode

chevron_right

Transcript

Episode notes

In this episode of The Cognitive Revolution, Nathan explores the groundbreaking paper on obfuscated activations with 3 members from the research team - Luke Bailey, Eric Jenner, and Scott Emmons. The team discusses how their work challenges latent-based defenses in AI systems, demonstrating methods to bypass safety mechanisms while maintaining harmful behaviors. Join us for an in-depth technical conversation about AI safety, interpretability, and the ongoing challenge of creating robust defense systems.

Do check out the "Obfuscated Activations Bypass LLM Latent-Space Defenses" paper here: https://obfuscated-activations.github.io/

Help shape our show by taking our quick listener survey at https://bit.ly/TurpentinePulse

SPONSORS:

Oracle Cloud Infrastructure (OCI): Oracle's next-generation cloud platform delivers blazing-fast AI and ML performance with 50% less for compute and 80% less for outbound networking compared to other cloud providers. OCI powers industry leaders like Vodafone and Thomson Reuters with secure infrastructure and application development capabilities. New U.S. customers can get their cloud bill cut in half by switching to OCI before March 31, 2024 at https://oracle.com/cognitive

NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive

Shopify: Dreaming of starting your own business? Shopify makes it easier than ever. With customizable templates, shoppable social media posts, and their new AI sidekick, Shopify Magic, you can focus on creating great products while delegating the rest. Manage everything from shipping to payments in one place. Start your journey with a $1/month trial at https://shopify.com/cognitive and turn your 2025 dreams into reality.

Vanta: Vanta simplifies security and compliance for businesses of all sizes. Automate compliance across 35+ frameworks like SOC 2 and ISO 27001, streamline security workflows, and complete questionnaires up to 5x faster. Trusted by over 9,000 companies, Vanta helps you manage risk and prove security in real time. Get $1,000 off at https://vanta.com/revolution

RECOMMENDED PODCAST:

Check out Modern Relationships where Erik Torenberg interviews tech power couples and leading thinkers to explore how ambitious people actually make partnerships work. This season's guests include: Delian Asparouhov & Nadia Asparouhova, Kristen Berman & Phil Levin, Rob Henderson, and Liv Boeree & Igor Kurganov.

Apple: https://podcasts.apple.com/us/podcast/id1786227593

Spotify: https://open.spotify.com/show/5hJzs0gDg6lRT6r10mdpVg

YouTube: https://www.youtube.com/@ModernRelationshipsPod

CHAPTERS:

(00:00:00) Teaser

(00:00:46) About the Episode

(00:05:11) Latent Space Defenses

(00:08:41) Sleeper Agents

(00:15:06) Three Case Studies (Part 1)

(00:17:02) Sponsors: Oracle Cloud Infrastructure (OCI) | NetSuite

(00:19:42) Three Case Studies (Part 2)

(00:24:09) SQL Generation