Myths: system not model; humans matter

Today's episode is a conversation with Bret Kinsella, recorded while he was in Las Vegas for CES and preparing to step onto the AI stage. Bret brings a rare combination of long-term perspective and hands-on experience.

As General Manager of Fuel iX at TELUS Digital, he operates generative AI systems at a scale most enterprises never see, processing trillions of tokens and delivering measurable business outcomes for global organizations. That vantage point gives him a clear view of both the promise of generative AI and the uncomfortable truths many teams are still avoiding.

Together, we unpack why generative AI breaks so many of the assumptions security teams have relied on for decades. Bret explains why these systems are probabilistic rather than deterministic, and how that single shift creates what he calls an unbounded attack surface.

Users are no longer limited to predefined buttons or workflows, and outputs are no longer constrained to a fixed database. The same prompt can succeed or fail depending on subtle changes, which makes single-pass testing and checkbox compliance dangerously misleading. If you have ever wondered why an AI system feels safe one day and unpredictable the next, this conversation offers a grounded explanation.

We also explore why focusing on the model alone misses the real risk. Bret makes a strong case that the model is only one part of a much larger system shaped by system prompts, connected data sources, tools, and guardrails. Change any one of those elements and behavior shifts. This is why automated, continuous red teaming has become unavoidable.

Bret shares how Telus Digital's Fortify AI attack model uncovered hundreds of vulnerabilities in hours, far beyond what human teams could realistically surface on their own. Yet automation is not the end of the story. The final decisions still depend on people who understand context, trade-offs, and business impact.

Throughout the discussion, we return to a simple but uncomfortable idea. AI safety is not something you bolt on after deployment. It demands a different mindset, broader testing, repeated validation, and ongoing human judgment. For leaders moving from experimentation to real-world deployment, this episode is a clear-eyed look at what responsible progress actually requires.

So, as more organizations rush to deploy agents and autonomous systems in 2026, are we truly prepared for software that learns, adapts, and occasionally surprises us, and what does that mean for how you test and trust AI inside your own business?

Useful Links

• Connect with Bret Kinsella
• Telus Digital Website
• Fuel iX

Thanks to our sponsors, Alcor, for supporting the show.