Detection at Scale

George Werbacher, Head of Security Operations at Live Oak Bank, reviews the practical realities of implementing AI agents in security operations, sharing his journey from exploring tools like Cursor and Claude Code to building custom agents in-house. He also reflects on the challenges of moving from local development to production-ready systems with proper durability and retry logic. The conversation explores how AI is changing the security analyst role from alert analysis to deeper investigation work, why SOAR platforms face significant disruption, and how MCP servers enable natural language interactions across security tools. George offers pragmatic advice on cutting through AI hype, emphasizing that agents augment rather than replace human expertise while dramatically lowering barriers to automation and query language mastery. Through technical insights and leadership perspective, George illuminates how security teams can embrace AI to improve operational efficiency and mean time to detect without inflating budgets, while maintaining the critical human judgment that effective security demands. Topics discussed: Understanding AI's role in augmenting security analysts rather than replacing them, shifting roles toward investigation and threat hunting. Building custom AI agents using Python and exploring frameworks like LangChain to solve specific SecOps use cases. Managing moving agents from local development to production, including retry logic, failbacks, and durability requirements. Implementing MCP servers to enable natural language interactions with security tools, eliminating the need to learn multiple query languages. Navigating AI hype by focusing on solving specific problems and understanding what agents can realistically accomplish. Predicting SOAR platform disruption as agents take over enrichment, orchestration, and response with simpler automation approaches. Removing platform barriers by enabling analysts to use natural language rather than mastering specific tools or query languages. Exploring context management, prompt engineering, and conversation history techniques essential for building effective agentic systems. Adopting tools like Cursor and Claude Code to empower technical security professionals without deep coding backgrounds.  Listen to more episodes:  Apple  Spotify  YouTube Website

Andrew Casazza, AVP of Cyber Security Operations at Ochsner Health, explores how healthcare organizations navigate FDA-approved medical devices running on legacy operating systems, implement AI-powered security tools while maintaining HIPAA compliance, and respond to threats that now move from initial compromise to malicious action in seconds rather than hours.  Andrew gives Jack his insights on building effective security programs in heavily regulated industries, emphasizing the importance of visibility, automation with guardrails, and keeping humans in the loop for critical decisions while leveraging AI to handle the speed and scale of modern threats. Topics discussed: Unique security challenges in healthcare environments where medical devices run on legacy operating systems that cannot be easily updated. Strategies for monitoring and securing systems that cannot have traditional security agents installed due to FDA regulations and medical certification requirements. Leveraging AI and automation in security operations while navigating HIPAA regulations and protecting patient data from external training models. Implementing human-in-the-loop approaches where AI performs initial analysis and triage while escalating critical decisions to human analysts. Understanding the privacy and compliance implications of AI tools that may use customer data for model training and improvement. The dramatic reduction in threat-actor dwell time from hours or days to minutes or seconds. Building effective SOAR automation playbooks to handle repetitive cases and reduce noise while focusing attention on bigger threats. Establishing appropriate guardrails for AI-powered security tools to prevent unintended consequences while enabling automated response capabilities. The importance of being curious and maintaining broad knowledge across multiple domains to become more effective. Listen to more episodes:  Apple  Spotify  YouTube Website

Stephen Gubenia, Head of Detection Engineering for Threat Response for Cisco Meraki, shares his evolution from managing overwhelming alert volumes as a one-person security team to architecting sophisticated automated systems that handle everything from enrichment to containment.  Stephen discusses the organizational changes needed for successful AI adoption, including top-down buy-in and proper training programs that help team members understand AI as a productivity multiplier rather than a job threat.  The conversation also explores Stephen’s practical "crawl, walk, run" methodology for responsibly implementing AI agents, the critical importance of maintaining human oversight through auditable workflows, and how security teams can transition from reactive alert management to strategic agent supervision.  Topics discussed: Evolution from manual security operations to AI-powered agentic workflows that eliminate repetitive tasks and enable strategic focus. Implementation of the "crawl, walk, run" methodology for gradually introducing AI agents with proper human oversight and validation. Building enrichment agents that automatically gather threat intelligence and OSINT data instead of manual investigations. Development of reasoning models that can dynamically triage alerts, run additional queries, and recommend investigation steps. Automated containment workflows that can perform endpoint isolation and other response actions while maintaining appropriate guardrails. Essential foundations including proper logging pipelines, alerting systems, and detection logic required before implementing AI automation. Human-in-the-loop strategies that transition from per-alert review to periodic auditing and agent management oversight. Organizational change management including top-down buy-in, training programs, and addressing fears about AI replacing jobs. Future of detection engineering with AI-assisted rule development, gap analysis, and customized detection libraries. Learning recommendations for cybersecurity professionals to develop AI literacy through reputable sources and consistent daily practice. Listen to more episodes:  Apple  Spotify  YouTube Website

Dave Herrald, Global Head of Cybersecurity GTM at Databricks, tells Jack about transforming security operations through modern data lake architectures and strategic AI implementation. He discusses the practical benefits of separating storage from compute, giving security teams direct control over data retention while maintaining operational flexibility. The conversation explores how organizations can move beyond traditional SIEM limitations by leveraging cost-effective data lake storage with advanced analytics capabilities. They touch on AI agents in security, where Dave advocates for focused agents over broad analyst replacement approaches. He also addresses common concerns about hallucinations, framing them as engineering challenges rather than insurmountable obstacles, and shares real-world examples of successful agent implementations. Topics discussed: Moving from traditional SIEM architectures to modern data lake approaches for cost-effective security analytics and data control. Implementing focused AI agents for specific security tasks like context gathering rather than attempting broad analyst replacement. Leveraging graph analytics for security operations including CMDB visualization, breach scoping, and vulnerability prioritization across enterprise environments. Addressing AI hallucinations through prompt engineering and proper context management rather than avoiding AI implementation entirely. Building detection capabilities using SQL and Python for analytics that provide supersets of traditional SIEM query languages. Creating normalization frameworks using standards like OCSF to enable consistent data analytics across diverse security data sources. Developing career resilience in security through mission-focused thinking, continuous AI learning, and building practical skills. Comparing modern AI agents to traditional SOAR platforms for automation effectiveness and maintenance requirements. Establishing data governance and access controls in security data lakes while maintaining operational flexibility and cost effectiveness. Listen to more episodes:  Apple  Spotify  YouTube Website

Matt Muller, Field CISO at Tines, knows all about revolutionizing security operations through strategic AI integration and intelligent automation. In his conversation with Jack, Matt explores how traditional SOC models create problematic feedback loops where junior analysts make critical decisions while senior practitioners handle escalations, limiting learning and growth opportunities.  Instead, Matt envisions AI-assisted workflows where senior expertise gets encoded into intelligent systems that teach junior team members while they work, transforming security operations from reactive alert-chasing to proactive strategic defense. He also emphasizes communication skills, relationship building, and moving beyond being perceived as the team of no to become strategic enablers. Topics discussed: Evolution from banning ChatGPT to strategic AI integration in security operations, emphasizing augmentation over replacement strategies. Model Context Protocol implementation challenges and the importance of safe-by-default approaches when integrating emerging AI technologies into production. Traditional SOC tier models create problematic feedback loops where junior analysts make critical decisions but lack learning opportunities. AI-assisted workflows can transform security operations by encoding senior expertise into systems that teach while automating routine tasks. Practical approaches to AI adoption including demystification techniques, validation methods, and breaking complex problems into manageable components. Strategic implementation of AI agents in security workflows, particularly for non-deterministic tasks like phishing investigation and alert triage. Importance of maintaining human oversight and guardrails when deploying AI systems in critical security operations and incident response. Communication skills and relationship building as fundamental competencies for security practitioners working with both AI systems and human stakeholders. Safe experimentation with AI technologies through controlled environments and understanding system limitations before production deployment. Listen to more episodes:  Apple  Spotify  YouTube Website

Erik Bloch, VP of Security at Illumio, brings a wealth of experience from transforming security teams at giants like Cisco and Salesforce. He emphasizes the need for solid security foundations—like effective ticketing systems—before jumping to AI tools. Erik critiques traditional security metrics as often misleading and highlights the importance of aligning security with business goals. He also discusses how managed service providers might lead in AI adoption due to their structured processes, pointing out the critical role of data in making informed security decisions.

John Hubbard, Cyber Defense Curriculum Lead at SANS Institute and host of the Blueprint podcast, dives into the future of SOCs in the age of AI. He discusses how AI revolutionizes alert contextualization, enabling better triage decisions by incorporating business context. John highlights the educational gap in teaching both traditional security skills and AI-driven approaches. He also explores the potential of natural language interfaces for complex tasks and shares insights on future-proofing careers in a rapidly evolving tech landscape.

Elliot Colquhoun, VP of Information Security + IT at Airwallex, has developed a cutting-edge AI-driven security program, protecting 1,800 employees with just 9 engineers. He discusses the revolutionary approach of using AI to contextualize security alerts, mimicking top engineer decision-making. Elliot shares his journey from Palantir to fintech, emphasizing a focus on hiring engineers with entrepreneurial skills rather than traditional backgrounds. He also explores navigating global regulatory compliance while maintaining security integrity, highlighting the future of adaptive security solutions.

Jacob DePriest, VP of Security/CISO at 1Password, shares his expertise from the NSA and GitHub. He outlines a fresh framework for assessing security focused on business objectives first. Jacob highlights the importance of integrating generative AI with human intuition in cybersecurity, discussing AI's role in enhancing operations while recognizing its limits. He also details 1Password's transformation from a password manager to a comprehensive security platform and offers valuable leadership tips on building relationships and maintaining work-life balance.

In this episode of Detection at Scale, Matthew Martin, Founder of Two Candlesticks, shares practical approaches for implementing AI in security operations, particularly for smaller companies and those in emerging markets. Matthew explains how AI chatbots can save analysts up to 45 minutes per incident by automating initial information gathering and ticket creation. Matthew’s conversation with Jack explores critical implementation challenges, from organizational politics to data quality issues, and the importance of making AI decisions auditable and explainable.  Matthew emphasizes the essential balance between AI capabilities and human intuition, noting that although AI excels at analyzing data, it lacks understanding of intent. He concludes with valuable advice for security leaders on business alignment, embracing new technologies, and maintaining human connection to prevent burnout. Topics discussed: Implementing AI chatbots in security operations can save analysts approximately 45 minutes per incident through automated information gathering and ticket creation. Political challenges within organizations, particularly around AI ownership and budget allocation, often exceed technical challenges in implementation. Data quality and understanding are foundational requirements before implementing AI in security operations to ensure effective and reliable results. The balance between human intuition and AI capabilities is crucial, as AI excels at data analysis but lacks understanding of intent behind actions. Security teams should prioritize making AI decisions auditable and explainable to ensure transparency and accountability in automated processes. Generative AI lowers barriers for both attackers and defenders, requiring security teams to understand AI capabilities and limitations. In-house data processing and modeling are preferable for sensitive customer data, with clear governance frameworks for privacy and security. Future security operations will likely automate many Tier 1 and Tier 2 functions, allowing analysts to focus on more complex issues. Security leaders must understand their business thoroughly to build controls that align with how the company generates revenue. Technology alone cannot solve burnout issues; leaders must understand their people at a human level to create sustainable efficiency improvements.