Measuring Agent Effectiveness and Trust

Guests:

• Alexander Pabst, Deputy Group CISO, Allianz
• Lars Koenig, Global Head of D&R, Allianz

Topics:

• Moving from traditional SIEM to an agentic SOC model, especially in a heavily regulated insurer, is a massive undertaking. What did the collaboration model with your vendor look like?
• Agentic AI introduces a new layer of risk - that of unconstrained or unintended autonomous action. In the context of Allianz, how did you establish the governance framework for the SOC alert triage agents?
• Where did you draw the line between fully automated action and the mandatory "human-in-the-loop" for investigation or response?
• Agentic triage is only as good as the data it analyzes. From your perspective, what were the biggest challenges - and wins - in ensuring the data fidelity, freshness, and completeness in your SIEM to fuel reliable agent decisions?
• We've been talking about SOC automation for years, but this agentic wave feels different. As a deputy CISO, what was your primary, non-negotiable goal for the agent? Was it purely Mean Time to Respond (MTTR) reduction, or was the bigger strategic prize to fundamentally re-skill and uplevel your Tier 2/3 analysts by removing the low-value alert noise?
• As you built this out, were there any surprises along the way that left you shaking your head or laughing at the unexpected AI behaviors?
• We felt a major lack of proof - Anton kept asking for pudding - that any of the agentic SOC vendors we saw at RSA had actually achieved anything beyond hype! When it comes to your org, how are you measuring agent success? What are the key metrics you are using right now?

Resources:

• EP238 Google Lessons for Using AI Agents for Securing Our Enterprise
• EP242 The AI SOC: Is This The Automation We've Been Waiting For?
• EP249 Data First: What Really Makes Your SOC 'AI Ready'?
• EP236 Accelerated SIEM Journey: A SOC Leader's Playbook for Modernization and AI
• "Simple to Ask: Is Your SOC AI Ready? Not Simple to Answer!" blog
• "How Google Does It: Building AI agents for cybersecurity and defense" blog
• Company annual report to look for risk
• "How to Win Friends and Influence People" by Dale Carnegie
• "Will It Make the Boat Go Faster?" book