How easy is model poisoning of LLMs?

Segment 1: David Brauchler on AI attacks and stopping them

David Brauchler says AI red teaming has proven that eliminating prompt injection is a lost cause. And many developers inadvertently introduce serious threat vectors into their applications – risks they must later eliminate before they become ingrained across application stacks.

NCC Group's AI security team has surveyed dozens of AI applications, exploited their most common risks, and discovered a set of practical architectural patterns and input validation strategies that completely mitigate natural language injection attacks. David's talk aimed at helping security pros and developers understand how to design/test complex agentic systems and how to model trust flows in agentic environments. He also provided information about what architectural decisions can mitigate prompt injection and other model manipulation risks, even when AI systems are exposed to untrusted sources of data.

More about David's Black Hat talk:

• Video of the talk and accompanying slides: https://www.nccgroup.com/research-blog/when-guardrails-arent-enough-reinventing-agentic-ai-security-with-architectural-controls/
• Talk abstract: https://www.blackhat.com/us-25/briefings/schedule/#when-guardrails-arent-enough-reinventing-agentic-ai-security-with-architectural-controls-46112
• Slide presentation only: https://i.blackhat.com/BH-USA-25/Presentations/USA-25-Brauchler-When-Guardrails-Arent-Enough.pdf

Additional blogs by David about AI security:

• Analyzing Secure AI Architectures: https://www.nccgroup.com/research-blog/analyzing-secure-ai-architectures/
• Analyzing Secure AI Design Principles: https://www.nccgroup.com/research-blog/analyzing-secure-ai-design-principles/
• Analyzing AI Application Threat Models: https://www.nccgroup.com/research-blog/analyzing-ai-application-threat-models/
• Building Security‑First AI Applications: A Best Practices Guide for CISOs: https://www.nccgroup.com/building-security-first-ai-applications-a-best-practices-guide-for-cisos/
• Building Trust by Design for Secure AI Applications: Tips for CISOs: https://www.nccgroup.com/building-trust-by-design-for-secure-ai-applications-tips-for-cisos/
• AI and Cyber Security: New Vulnerabilities CISOs Must Address: https://www.nccgroup.com/ai-and-cyber-security-new-vulnerabilities-cisos-must-address/

Segment 2: Should we replace the CIA triad?

An op-ed on CSO Online made us think - should we consider the CIA triad 'dead' and replace it? We discuss the value and longevity of security frameworks, as well as the author's proposed replacement.

Segment 3: The Weekly Enterprise News

Finally, in the enterprise security news,

1. Slow week for funding, older companies raising via debt financing
2. A useful AI framework from the Cloud Security Alliance
3. two interesting essays, one of which is wrong
4. Folks are out here blasting unencrypted data to and from Satellites, while anyone can sniff and capture it
5. getting hacked during a job interview
6. LLM poisoning is far easier than previously thought
7. F5 got breached
8. Be careful when patching your Jeep ('s software)

All that and more, on this episode of Enterprise Security Weekly.

Visit https://www.securityweekly.com/esw for all the latest episodes!

Show Notes: https://securityweekly.com/esw-429