Cloud Security Podcast

chevron_right

AI-First Vulnerability Management: Should CISOs Build or Buy?

whatshot 12 snips

Dec 4, 2025

Guest

Santiago Castiñeira

Santiago Castiñeira, CTO of Maze and expert in AI-first vulnerability management, dives into the build vs. buy debate for cybersecurity tools. He highlights the complexities of scaling AI systems beyond basic prototypes, emphasizing the need for specialized skills. Santiago discusses the limitations of current frameworks and the importance of creating robust evaluation pipelines. He warns against reliance on Retrieval-Augmented Generation for accurate technical data and envisions future semi-autonomous security agents that could revolutionize vulnerability management.

01:01:30

forum

Ask episode

web_stories

AI Snips

view_agenda

Chapters

auto_awesome

Transcript

info_circle

Episode notes

insights

INSIGHT

Reasoning Replaces Rules

AI-first vulnerability management shifts from rule-based checks to contextual reasoning across assets.
Santiago Castiñeira says reasoning enables more precise prioritization by evaluating impact, likelihood, and exploitability.

volunteer_activism

ADVICE

Don't Ship Scripts As Products

Do not assume a quick script equals a production AI system; build robust data pipelines and provenance first.
Santiago warns to plan for cost, auditability, and long-term maintenance before deploying LLM-driven decisions.

volunteer_activism

ADVICE

Staff ML And Data Skills Early

Hire or borrow software, data, and ML engineers before building AI workflows; security teams rarely have these skills.
Santiago advises explicitly staffing these roles to avoid single-developer bus-factor failures.

Get the Snipd Podcast app to discover more snips from this episode

Defining AI‑First Vulnerability Management

02:08 • 3min

chevron_right

Can You Just Use ChatGPT to Build It?

04:41 • 2min

chevron_right

The Bus Factor and Skill Gaps

07:03 • 2min

chevron_right

Limits of MCP and Human‑Driven Agents

08:41 • 2min

chevron_right

Core Architecture: Data, Execution & Agents

10:28 • 3min

chevron_right

From One‑Off Scripts to Scalable Systems

13:07 • 2min

chevron_right

Real Failures with LangChain/LangGraph Frameworks

14:50 • 2min

chevron_right

Where to Start: Problem, Context, and Skills

16:53 • 3min

chevron_right

Piping Cloud Logs into Agents

19:38 • 2min

chevron_right

Prompt Drift and Consistency Challenges

21:33 • 4min

chevron_right

Minimum Team & CTO Buy‑In For Building

25:35 • 2min

chevron_right

Importance of a Data Lake and Retrieval System

27:07 • 49sec

chevron_right

Model Changes, MLOps and Evals

27:56 • 3min

chevron_right

Measuring Impact: Productivity and Risk Reduction

30:36 • 3min

chevron_right

Surprises from Agent Behavior

33:12 • 3min

chevron_right

Token Costs and Context Volume Tradeoffs

36:07 • 1min

chevron_right

Multi‑Agent Governance and Guardrails

37:35 • 4min

chevron_right

Semi‑Autonomous Security Fleets

41:34 • 4min

chevron_right

Why RAG Fails for Precise Technical Data

45:11 • 2min

chevron_right

Evaluating Vendors: AI‑First vs AI‑Sprinkled

46:53 • 3min

chevron_right

Common Mistakes: Vibe Evals and Cost Ignorance

50:03 • 4min

chevron_right

What an Eval Is and How to Use It

53:50 • 52sec

chevron_right

Responding Faster: Automation Before CVE Publication

54:42 • 2min

chevron_right

Agents Approaching Niche Superintelligence

56:12 • 2min

chevron_right

Personal Notes: Family and Building Maze

58:09 • 2min

chevron_right

How to Contact Maze and Final Offer

Thinking of building your own AI security tool? In this episode, Santiago Castiñeira, CTO of Maze, breaks down the realities of the "Build vs. Buy" debate for AI-first vulnerability management.

While building a prototype script is easy, scaling it into a maintainable, audit-proof system is a massive undertaking requiring specialized skills often missing in security teams. The "RAG drug" relies too heavily on Retrieval-Augmented Generation for precise technical data like version numbers, which often fails .

The conversation gets into the architecture required for a true AI-first system, moving beyond simple chatbots to complex multi-agent workflows that can reason about context and risk . We also cover the critical importance of rigorous "evals" over "vibe checks" to ensure AI reliability, the hidden costs of LLM inference at scale, and why well-crafted agents might soon be indistinguishable from super-intelligence .

Guest Socials -⁠⁠ ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Santiago's Linkedin

If you want to watch videos of this LIVE STREAMED episode and past episodes - Check out our other Cloud Security Social Channels:

If you are interested in AI Cybersecurity, you can check out our sister podcast -⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ AI Security Podcast⁠

Questions asked:

(00:00) Introduction(02:00) Who is Santiago Castiñeira?(02:40) What is "AI-First" Vulnerability Management? (Rules vs. Reasoning)(04:55) The "Build vs. Buy" Debate: Can I Just Use ChatGPT?(07:30) The "Bus Factor" Risk of Internal Tools(08:30) Why MCP (Model Context Protocol) Struggles at Scale(10:15) The Architecture of an AI-First Security System(13:45) The Problem with "Vibe Checks": Why You Need Proper Evals(17:20) Where to Start if You Must Build Internally(19:00) The Hidden Need for Data & Software Engineers in Security Teams(21:50) Managing Prompt Drift and Consistency(27:30) The Challenge of Changing LLM Models (Claude vs. Gemini)(30:20) Rethinking Vulnerability Management Metrics in the AI Era(33:30) Surprises in AI Agent Behavior: "Let's Get Back on Topic"(35:30) The Hidden Cost of AI: Token Usage at Scale(37:15) Multi-Agent Governance: Preventing Rogue Agents(41:15) The Future: Semi-Autonomous Security Fleets(45:30) Why RAG Fails for Precise Technical Data (The "RAG Drug")(47:30) How to Evaluate AI Vendors: Is it AI-First or AI-Sprinkled?(50:20) Common Architectural Mistakes: Vibe Evals & Cost Ignorance(56:00) Unpopular Opinion: Well-Crafted Agents vs. Super Intelligence(58:15) Final Questions: Kids, Argentine Steak, and Closing

Home Top podcasts Popular guests Top books