Max Pollard - Why Hidden System Prompts Are Failing Security Teams

Sep 21, 2026

Ask episode

Chapters

Transcript

Episode notes

S6:E4 - Max Pollard - Why Hidden System Prompts Are Failing Security Teams

Episode Summary

In this episode of Simply Defensive, Josh Mason and Wade Wells sit down with Max Pollard. He is the founder and CEO of CoTool, an AI-native detection and response platform. Max's path ran through quantitative finance, then Okta's customer identity team, then Material Security, where he built and led the forward-deployed engineering team. He left to found CoTool earlier this year. The conversation traces where the idea came from. Security engineers were already turning to tools like ChatGPT and Claude Code for help writing detections. Those tools hit a wall the moment they needed to understand an org's actual logs and schema.

Josh and Wade push Max on the harder questions too. Why do SOAR teams still hesitate to trust automatic remediation, and what has to change before anyone hands that trust to AI? The back half turns into a real debate about control, built around a recent LinkedIn post where Max argued that hidden system prompts are quietly failing the security teams who depend on them. Detection engineers, SOC analysts evaluating AI tools, and anyone tired of agents that ignore instructions will get something out of this one.

What You'll Learn

Max's path from algorithmic trading to Okta, Material Security, and founding CoTool
What CoTool does: better rule-based detections, catching behavior rules that can't be expressed, and full-context response
Why grounding AI in your organization's real logs and schema matters more than a good prompt
How backtesting a detection connects straight back to Max's quant finance background
Why AI got noticeably better at MITRE ATT&CK mapping over the past couple of years
Why SOAR teams still hesitate on automatic remediation, and what AI changes about that calculus
How AI can fast-track junior analysts instead of replacing them
Max's LinkedIn take on hidden system prompts, and why he thinks they hurt security teams
Why CoTool calls itself a harness instead of an agent

Episode Highlights

From Algorithmic Trading to AI Security Max's path ran through quantitative finance, Okta's customer identity team, and Material Security before he left to found CoTool. He draws on that finance background more than you'd expect, especially around backtesting.

What CoTool Actually Does A unified detection and response layer that sits on top of your existing stack, not a SIEM replacement. It helps teams write and maintain better rule-based detections, catch attacker behavior rules can't express, and respond faster with full context across all their security data.

Grounding AI in Your Real Data The gap Max saw in tools like ChatGPT and Cursor: great at generic help, useless the moment they needed to understand an org's actual logs, schema, and normalization choices. CoTool was built to close that gap.

Trusting AI to Act on Its Own Josh and Wade push Max on automatic remediation. If teams don't fully trust a black-and-white SOAR playbook, why would they trust an AI system that can hallucinate? Max walks through why the path to real automation runs through enrichment, false-positive triage, and suggested response first.

The LinkedIn Post That Started This Conversation Max recently argued that bolt-on AI agents with hidden system prompts are a net negative for security teams. He points to Gmail's early email assistant as the cautionary tale: a hidden prompt hardcoded a stiff, overly formal tone that made the tool nearly unusable.

Why CoTool Calls Itself a Harness Instead of shipping a fixed agent, CoTool exposes the same underlying primitive to users and lets them shape it for detection, response, or whatever they need. Max's argument: security changes too fast for hidden, hardcoded behavior to hold up.

Timestamps

(This tag pulls the list straight from the Chapters field below once you paste it in there, so you're not maintaining the same list twice. If you'd rather have a static list here instead, use the block under "Chapters (standalone)" further down.)

Resources Mentioned

CoTool: https://cotool.ai
detections.ai (referenced in the conversation): https://detections.ai
Our earlier episode with Aaron Mog of detections.ai: https://simplydefensive.transistor.fm/s4/3

Connect with Max

CoTool: https://cotool.ai
Email: [email protected]

Connect with Your Hosts

Josh Mason: https://www.linkedin.com/in/joshuacmason/
Wade Wells: https://www.linkedin.com/in/wadingthrulogs/