Ep140: Architecting Agentic AI Systems - Technical Insights for ISVs with Anyscale, CrewAI and Encrypt AI

Sep 3, 2025

Guest

Joe (CEO, CrewAI)

Join Sahil Agrawal, Co-founder and CEO at Encrypt, Robert Nishihara, Co-founder of AnyScale, and Joe, CEO of CrewAI, as they delve into the intricacies of scaling agentic AI systems. The panel unveils the move from flashy demos to dependable applications, emphasizing the importance of security and interoperability. Discover groundbreaking use cases, like automating IRS forms, and the integration of multimodal data. They also discuss emerging governance frameworks and the critical role of centralized visibility for security in a fragmented AI landscape.

Ask episode

AI Snips

Chapters

Transcript

Episode notes

ANECDOTE

From IRS Forms To Rap Songs

Joe described customers using agents to auto-fill complex IRS forms and to write rap songs for a record label.
He highlighted a shift from flashy demos to dependable, production-ready agents.

INSIGHT

Multimodal Data Unlocks Real Value

Robert emphasized multimodal data (video, audio, screenshots) as a major source of value for agents.
He said agents using diverse data can make more human-like decisions in workflows.

ADVICE

Build Governance Before Scale

Plan governance early because hundreds of agents will create unmanaged enterprise risk.
Build compliance and a code-of-conduct for your agent workforce before scale hits.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

A panel discussion with AI industry leaders revealing how enterprises are scaling AI today, with predictions on coming breakthroughs for AI and the impact on Fortune 500 companies and beyond.

Topics Include:

Three technical leaders discuss production challenges: security, interoperability, and scaling agentic systems
Panelists represent Enkrypt (security), Anyscale (infrastructure), and CrewAI (agent orchestration platforms)
Industry moving from flashy demos to dependable agents with real business outcomes
Breakthrough examples include 70-page IRS form processing and multimodal workflow automation
Multimodal data integration becoming crucial - incorporating video, audio, screenshots into decisions
Less than 10% of future applications expected to be text-only
Companies shifting from experimenting with individual models to deploying agent networks
Need for governance frameworks as enterprises scale to hundreds of agents
Growing software stack complexity requires specialized infrastructure between applications and GPUs
Security teams need centralized visibility across fragmented agent deployments across enterprises
Existing industry regulations apply to AI services - no special AI laws needed
Interoperability standards debate: MCP gaining adoption while A2A seems premature solution
MCP shows higher API reliability than OpenAI tool calling for implementations
Multimodal systems more vulnerable to attacks but value proposition too high ignore
Fortune 500 company automated price operations approval process using 630 brands data
87% of enterprise customers deploy agents in private VPCs or on-premises infrastructure
Specialized AI systems needed to oversee other agents at machine speed scales
Cost optimization through model specialization rather than always using most powerful models
Future learning may happen through context/prompting rather than traditional weight fine-tuning
Predictions include AI meeting moderators and agents working autonomously for hours

Participants:

Robert Nishihara - Co-founder, Anyscale
João Moura - CEO, CrewAI
Sahil Agarwal - Co-Founder & CEO, Enkrypt AI
Jillian D’Arcy - Sr. ISV Sales Leader, Amazon Web Services

Further Links:

Anyscale – Website | LinkedIn | AWS Marketplace
CrewAI - Website | LinkedIn | AWS Marketplace
Enkrypt AI - Website | LinkedIn | AWS Marketplace

See how Amazon Web Services gives you the freedom to migrate, innovate, and scale your software company at https://aws.amazon.com/isv/