Cloud Security Podcast

chevron_right

Threat Modeling the AI Agent: Architecture, Threats & Monitoring

whatshot 11 snips

Nov 11, 2025

Guest

Mohan Kumar

Mohan Kumar, a production security practitioner at Box with over 14 years in cybersecurity, dives into the uncharted waters of AI agent threats. He highlights how autonomous AI agents can develop secure communication methods, posing new risks like memory poisoning through indirect prompts. Mohan warns of tool misuse, where agents may inadvertently connect to rogue systems, and stresses the importance of enforcing least-privilege. He also discusses the need for robust monitoring and audits to manage these evolving threats in the AI landscape.

47:20

forum

Ask episode

web_stories

AI Snips

view_agenda

Chapters

auto_awesome

Transcript

info_circle

Episode notes

insights

INSIGHT

Agents Reshape The Threat Surface

Autonomous agents create new, unexpected channels and behaviors that bypass traditional security assumptions.
Security teams must rethink threat surfaces because agents can communicate and act outside human-observed patterns.

volunteer_activism

ADVICE

Treat Memory As A Trust Boundary

Sanitize and validate agent memories and treat memory as a critical trust boundary.
Snapshot memory for forensics and implement authentication for memory access to detect poisoning.

volunteer_activism

ADVICE

Limit And Authorize Tool Access

Scope tool access to minimal permissions and limit token lifetimes for agent tools.
Require human approval for high‑risk tool actions and enforce short-lived, narrow scopes.

Get the Snipd Podcast app to discover more snips from this episode

Mohan Kumar's Background and Focus

01:36 • 2min

chevron_right

Distinguishing LLM Applications from AI Agents

03:34 • 3min

chevron_right

Agentic AI Behaviors and Unanticipated Channels

06:20 • 4min

chevron_right

Top Threat: Memory Poisoning Explained

10:30 • 4min

chevron_right

Tool Misuse: Rogue Tools and Exfiltration Risks

14:05 • 3min

chevron_right

Privilege Compromise and Least-Privilege Enforcement

17:00 • 1min

chevron_right

Monitoring Agent Behavior and Auditing Challenges

18:12 • 7min

chevron_right

What an AI Agent Architecture Looks Like

24:50 • 2min

chevron_right

Threat Modeling Frameworks and Layers

27:16 • 5min

chevron_right

Vendor-Agnostic Risks and Prompt Injection

32:41 • 6min

chevron_right

Where Agent Security Will Evolve: Orchestration, Data, UI

38:15 • 4min

chevron_right

Personal Interests and Practical Insights

Are we underestimating how the agentic world is impacting cybersecurity? We spoke to Mohan Kumar, who did production security at Box for a deep dive into the threats of true autonomous AI agents.

The conversation moves beyond simple LLM applications (like chatbots) to the new world of dynamic, goal-driven agents that can take autonomous actions. Mohan took us through why this shift introduces a new class of threats we aren't prepared for, such as agents developing new, unmonitorable communication methods ("Jibber-link" mode).

Mohan shared his top three security threats for AI agents in production:

Memory Poisoning: How an agent's trusted memory (long-term, short-term, or entity memory) can be corrupted via indirect prompt injection, altering its core decisions.
Tool Misuse: The risk of agents connecting to rogue tools or MCP servers, or having their legitimate tools (like a calendar) exploited for data exfiltration.
Privilege Compromise: The critical need to enforce least-privilege on agents that can shift roles and identities, often through misconfiguration.

Guest Socials -⁠ ⁠⁠⁠⁠⁠⁠⁠⁠Mohan's Linkedin

If you want to watch videos of this LIVE STREAMED episode and past episodes - Check out our other Cloud Security Social Channels:

If you are interested in AI Cybersecurity, you can check out our sister podcast -⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ AI Security Podcast⁠

Questions asked:

(00:00) Introduction(01:30) Who is Mohan Kumar? (Production Security at Box)(03:30) LLM Application vs. AI Agent: What's the Difference?(06:50) "We are totally underestimating" AI agent threats(07:45) Software 3.0: When Prompts Become the New Software(08:20) The "Jibber-link" Threat: Agents Ditching Human Language(10:45) The Top 3 AI Agent Security Threats(11:10) Threat 1: Memory Poisoning & Context Manipulation(14:00) Threat 2: Tool Misuse (e.g., exploiting a calendar tool)(16:50) Threat 3: Privilege Compromise (Least Privilege for Agents)(18:20) How Do You Monitor & Audit Autonomous Agents?(20:30) The Need for "Observer" Agents(24:45) The 6 Components of an AI Agent Architecture(27:00) Threat Modeling: Using CSA's MAESTRO Framework(31:20) Are Leaks Only from Open Source Models or Closed (OpenAI, Claude) Too?(34:10) The "Grandma Trick": Any Model is Susceptible(38:15) Where is AI Agent Security Evolving? (Orchestration, Data, Interface)(42:00) Fun Questions: Hacking MCPs, Skydiving & Risk, Biryani

Resources mentioned during the episode:

Mohan’s Udemy Course -AI Security Bootcamp: LLM Hacking Basics

Andre Karpathy's "Software 3.0" Concept

"Jibber-link Mode" Video

CrewAI Framework

OWASP Top 10 for LLM Applications

Cloud Security Alliance (CSA) MAESTRO Framework

Home Top podcasts Popular guests Top books