The podcast covers a range of interesting topics, including AI Red Teaming and the concept of 'jail-breaking' large language models. They discuss the interplay between language and AI models, as well as the importance of red teaming and mitigating risks in AI. Updates on security for Azure SQL and SQL Server are also provided.
AI Red Teaming now covers a broader scope, including security, responsible AI, bias, and harmful content, and the team works closely with stakeholders throughout the process and assumes both malicious and benign personas.
Inadequate controls or irresponsible usage of AI technology can lead to potential harm, therefore conscious application of AI, addressing biases, and leveraging a diverse team are key considerations to mitigate risks.
Deep dives
Evolution of AI Red Teaming
AI Red Teaming has evolved over the years, starting in 2018 at Microsoft. Initially, engagements were three to four months long and focused on new models. However, with the rise of large language models (LLMs), the team has grown and shifted towards pre-ship testing in two to three weeks. AI Red Teaming now covers a broader scope, including security, responsible AI, bias, and harmful content. The team works closely with stakeholders throughout the process and assumes both malicious and benign personas.
A Day in the Life of AI Red Teaming
In an AI Red Teaming operation, the team creates a test plan and uses various techniques to test the safety and security of AI applications. This includes traditional web app pen testing methods, prompt engineering, and prompt injection. Tools like Pirate may be used to automate testing by sending thousands of prompts to the model. Additionally, the team provides feedback to the wider AI safety and security community.
Jailbreaking Large Language Models
Jailbreaking involves using prompts to bypass the instructions and limitations of a large language model, allowing it to behave outside its intended scope. Jailbreaking techniques have demonstrated the ability to make models produce unintended and potentially harmful outputs. The arms race between identifying jailbreak attempts and creating mitigations continues. Understanding the interplay between language and model weights is crucial, but closed-source models present additional challenges.
AI Security Concerns
AI security concerns revolve around granting models more access to data and actions, increasing the attack surface. Inadequate controls or irresponsible usage of AI technology can lead to potential harm. Incorporating LLMs into technologies where they may not be ready or neglecting traditional security controls pose risks. Conscious application of AI, addressing biases, and leveraging a diverse team are key considerations to mitigate risks.
This is a MUST LISTEN episode for anyone involved in products using AI, or for people who want to learn some of the latest attacks against large language models. Make sure you peruse the exhaustive list of AI security links at The Microsoft Azure Security Podcast (azsecuritypodcast.net),
We cover news about Azure SQL DB, Trusted VMs, NetApp Files, Azure Load Testing and Front Door. Mark covers further details about Zero Trust and the CISO Workshop.
Get the Snipd podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share & Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode