Debunking AI hype, exploring security risks, and strategies for CTOs in the age of rapid change.
In this episode, Tobi talks with Georg Zoeller, Co-Founder of the Centre for AI Leadership and mercenaries.ai, about the turbulent landscape of AI. Georg, with his background at Meta and deep expertise in AI strategy, cuts through the hype surrounding AI's capabilities and economic impact.
They discuss the 'singularity' we're already in, driven by rapid, open-source AI development, and why this makes future predictions impossible. Georg argues that software engineering is being commoditized due to the vast amount of training data available (Stack Overflow, GitHub), making AI adept at code generation but raising profound security concerns like prompt injection.
Explore:
- Why Georg believes blindly adopting AI early is a 'terrible mistake' for most companies.
- The fundamental security flaws in LLMs (prompt injection) and why they're currently unsolvable for open input spaces.
- The questionable economics of AI: high costs, self-cannibalizing business models, and the reliance on performative fundraising.
- How AI tools impact engineer productivity, shifting the bottleneck to decision-making and validation.
- The geopolitical risks and diminishing trust associated with Big Tech's AI dominance.
- Actionable advice for CTOs: Invest in understanding, focus on governance beyond the tech team, and consider the strategic value of local/open-source alternatives.
TIMESTAMPS:
[00:00:51] Introduction to Georg Zoeller
[00:02:16] Georg's Fascination with Complexity and Computers
[00:03:54] The Commoditization of Software via AI
[00:05:13] Training Data: Software Engineering's "Terrible Mistake"
[00:11:00] The Dilemma of Operating in Exponential Change
[00:12:09] Why Adopting AI Early Can Be a Mistake
[00:16:41] Deep Dive: The Unsolved Security Problem of Prompt Injection
[00:23:15] The Questionable Economics and Business Models of AI
[00:32:23] AI for Information Discovery vs. Decision Making
[00:47:08] Geopolitical Risks and the Growth Limits of Big Tech
[00:55:46] Recommendations for CTOs in 2025
[01:01:08] Looking Back: Advice for Younger Self
QUOTE:
"Software engineering profession made a terrible mistake. We uploaded everything we know onto two websites, Stack Overflow and GitHub… our entire profession, our entire knowledge… they're all uploaded… And when it's in the training data… the technology gets really good at creating results." - Georg Zoeller
