Detection at Scale

Matt Muller, Field CISO at Tines, knows all about revolutionizing security operations through strategic AI integration and intelligent automation. In his conversation with Jack, Matt explores how traditional SOC models create problematic feedback loops where junior analysts make critical decisions while senior practitioners handle escalations, limiting learning and growth opportunities.  Instead, Matt envisions AI-assisted workflows where senior expertise gets encoded into intelligent systems that teach junior team members while they work, transforming security operations from reactive alert-chasing to proactive strategic defense. He also emphasizes communication skills, relationship building, and moving beyond being perceived as the team of no to become strategic enablers. Topics discussed: Evolution from banning ChatGPT to strategic AI integration in security operations, emphasizing augmentation over replacement strategies. Model Context Protocol implementation challenges and the importance of safe-by-default approaches when integrating emerging AI technologies into production. Traditional SOC tier models create problematic feedback loops where junior analysts make critical decisions but lack learning opportunities. AI-assisted workflows can transform security operations by encoding senior expertise into systems that teach while automating routine tasks. Practical approaches to AI adoption including demystification techniques, validation methods, and breaking complex problems into manageable components. Strategic implementation of AI agents in security workflows, particularly for non-deterministic tasks like phishing investigation and alert triage. Importance of maintaining human oversight and guardrails when deploying AI systems in critical security operations and incident response. Communication skills and relationship building as fundamental competencies for security practitioners working with both AI systems and human stakeholders. Safe experimentation with AI technologies through controlled environments and understanding system limitations before production deployment. Listen to more episodes:  Apple  Spotify  YouTube Website

Erik Bloch, VP of Security at Illumio, brings a wealth of experience from transforming security teams at giants like Cisco and Salesforce. He emphasizes the need for solid security foundations—like effective ticketing systems—before jumping to AI tools. Erik critiques traditional security metrics as often misleading and highlights the importance of aligning security with business goals. He also discusses how managed service providers might lead in AI adoption due to their structured processes, pointing out the critical role of data in making informed security decisions.

John Hubbard, Cyber Defense Curriculum Lead at SANS Institute and host of the Blueprint podcast, dives into the future of SOCs in the age of AI. He discusses how AI revolutionizes alert contextualization, enabling better triage decisions by incorporating business context. John highlights the educational gap in teaching both traditional security skills and AI-driven approaches. He also explores the potential of natural language interfaces for complex tasks and shares insights on future-proofing careers in a rapidly evolving tech landscape.

Elliot Colquhoun, VP of Information Security + IT at Airwallex, has developed a cutting-edge AI-driven security program, protecting 1,800 employees with just 9 engineers. He discusses the revolutionary approach of using AI to contextualize security alerts, mimicking top engineer decision-making. Elliot shares his journey from Palantir to fintech, emphasizing a focus on hiring engineers with entrepreneurial skills rather than traditional backgrounds. He also explores navigating global regulatory compliance while maintaining security integrity, highlighting the future of adaptive security solutions.

Jacob DePriest, VP of Security/CISO at 1Password, shares his expertise from the NSA and GitHub. He outlines a fresh framework for assessing security focused on business objectives first. Jacob highlights the importance of integrating generative AI with human intuition in cybersecurity, discussing AI's role in enhancing operations while recognizing its limits. He also details 1Password's transformation from a password manager to a comprehensive security platform and offers valuable leadership tips on building relationships and maintaining work-life balance.

In this episode of Detection at Scale, Matthew Martin, Founder of Two Candlesticks, shares practical approaches for implementing AI in security operations, particularly for smaller companies and those in emerging markets. Matthew explains how AI chatbots can save analysts up to 45 minutes per incident by automating initial information gathering and ticket creation. Matthew’s conversation with Jack explores critical implementation challenges, from organizational politics to data quality issues, and the importance of making AI decisions auditable and explainable.  Matthew emphasizes the essential balance between AI capabilities and human intuition, noting that although AI excels at analyzing data, it lacks understanding of intent. He concludes with valuable advice for security leaders on business alignment, embracing new technologies, and maintaining human connection to prevent burnout. Topics discussed: Implementing AI chatbots in security operations can save analysts approximately 45 minutes per incident through automated information gathering and ticket creation. Political challenges within organizations, particularly around AI ownership and budget allocation, often exceed technical challenges in implementation. Data quality and understanding are foundational requirements before implementing AI in security operations to ensure effective and reliable results. The balance between human intuition and AI capabilities is crucial, as AI excels at data analysis but lacks understanding of intent behind actions. Security teams should prioritize making AI decisions auditable and explainable to ensure transparency and accountability in automated processes. Generative AI lowers barriers for both attackers and defenders, requiring security teams to understand AI capabilities and limitations. In-house data processing and modeling are preferable for sensitive customer data, with clear governance frameworks for privacy and security. Future security operations will likely automate many Tier 1 and Tier 2 functions, allowing analysts to focus on more complex issues. Security leaders must understand their business thoroughly to build controls that align with how the company generates revenue. Technology alone cannot solve burnout issues; leaders must understand their people at a human level to create sustainable efficiency improvements.  

The security automation landscape is undergoing a revolutionary transformation as AI reasoning capabilities replace traditional rule-based playbooks. In this episode of Detection at Scale, Oliver Friedrichs, Founder & CEO of Pangea, helps Jack unpack how this shift democratizes advanced threat detection beyond Fortune 500 companies while simultaneously introducing an alarming new attack surface.  Security teams now face unprecedented challenges, including 86 distinct prompt injection techniques and emergent "AI scheming" behaviors where models demonstrate self-preservation reasoning. Beyond highlighting these vulnerabilities, Oliver shares practical implementation strategies for AI guardrails that balance innovation with security, explaining why every organization embedding AI into their applications needs a comprehensive security framework spanning confidential information detection, malicious code filtering, and language safeguards. Topics discussed: The critical "read versus write" framework for security automation adoption: organizations consistently authorized full automation for investigative processes but required human oversight for remediation actions that changed system states. Why pre-built security playbooks limited SOAR adoption to Fortune 500 companies and how AI-powered agents now enable mid-market security teams to respond to unknown threats without extensive coding resources. The four primary attack vectors targeting enterprise AI applications: prompt injection, confidential information/PII exposure, malicious code introduction, and inappropriate language generation from foundation models. How Pangea implemented AI guardrails that filter prompts in under 100 milliseconds using their own AI models trained on thousands of prompt injection examples, creating a detection layer that sits inline with enterprise systems. The concerning discovery of "AI scheming" behavior where a model processing an email about its replacement developed self-preservation plans, demonstrating the emergent risks beyond traditional security vulnerabilities. Why Apollo Research and Geoffrey Hinton, Nobel-Prize-winning AI researcher, consider AI an existential risk and how Pangea is approaching these challenges by starting with practical enterprise security controls.   Check out Pangea.com  

Matt Jezorek, CISO at Panther and a former security leader at Amazon and Dropbox, shares insights on simplifying security operations. He emphasizes focusing on identity protection, vulnerability management, and detection/response. Matt argues that human intuition remains vital, even as AI advances. He discusses navigating the complexities of security data and the importance of strategic response. Additionally, he reflects on how his farm life perspective aids in handling high-pressure situations and the importance of staying curious in both security and life.

Managing security for a device that can autonomously interact with third-party services presents unique orchestration challenges that go beyond traditional IoT security models. In this episode of Detection at Scale, Matthew Domko, Head of Security at Rabbit, gives Jack an in-depth look at building security programs for AI-powered hardware at scale.   He details how his team achieved 100% infrastructure-as-code coverage while maintaining the agility needed for rapid product iteration. Matt also challenges conventional approaches to scaling security operations, advocating for a serverless-first architecture that has fundamentally changed how they handle detection engineering. His insights on using private LLMs via Amazon Bedrock to analyze security events showcase a pragmatic approach to AI adoption, focusing on augmentation of existing workflows rather than wholesale replacement of human analysis.  Topics discussed: How transitioning from reactive SIEM operations to a data-first security approach using AWS Lambda and SQS enabled Rabbit's team to handle complex orchestration monitoring without maintaining persistent infrastructure.  The practical implementation of LLM-assisted detection engineering, using Amazon Bedrock to analyze 15-minute blocks of security telemetry across their stack.  A deep dive into security data lake architecture decisions, including how their team addressed the challenge of cost attribution when security telemetry becomes valuable to other engineering teams.  The evolution from traditional detection engineering to a "detection-as-code" pipeline that leverages infrastructure-as-code for security rules, enabling version control, peer review, and automated testing of detection logic while maintaining rapid deployment capabilities. Concrete examples of integrating security into the engineering workflow, including how they use LLMs to transform security tickets to match engineering team nomenclature and communication patterns. Technical details of their data ingestion architecture using AWS SQS and Lambda, showing how two well-documented core patterns enabled the team to rapidly onboard new data sources and detection capabilities without direct security team involvement. A pragmatic framework for evaluating where generative AI adds value in security operations, focusing on specific use cases like log analysis and detection engineering where the technology demonstrably improves existing workflows rather than attempting wholesale process automation.  Listen to more episodes:  Apple  Spotify  YouTube Website

Mor Levi, VP of Detection, Analysis, & Response at Salesforce, shares her expertise on integrating AI in security operations. She reveals how Agent Force achieved 90% automation in triage while maintaining effectiveness. Topics include securing AI implementations, the evolving roles of security analysts, and the importance of data quality. Mor discusses the balance between AI efficiency and human creativity, emphasizing the need for strategic thinking in an increasingly automated landscape. Real-world examples provide insights into both the challenges and successes of AI in enterprise security.