Navigating Client Needs and AI Challenges

First, there was vibe coding. Now, get ready for "vibe entrepreneurship." 

Andrew McNamara, Director of Applied Machine Learning at Shopify, joins us to explain how his team is making this new era of business a reality. He shares the vision behind Shopify Sidekick, an AI co-founder designed to empower merchants by acting as their on-demand e-commerce expert. Drawing on his 16-year journey building AI assistants, Andrew reveals what it truly takes to create an AI tool that customers can trust with their livelihood.

He shares a critical insight from his applied research background: the hardest and most important part of building production-ready AI isn't the model, but the evaluation ("eval") system. Andrew breaks down Shopify's innovative approach of using an LLM-as-a-judge to measure how well they're fulfilling user goals, not just executing features. This conversation is a definitive guide to creating high-trust AI systems and offers a powerful glimpse into the future of commerce.

Check out:

• Register now: Closing the AI gap: Exceeding executive expectations for AI productivity

Follow the hosts:

• Follow Ben
• Follow Andrew

Follow today's guest(s):

• Follow Andrew on X
• Follow Andrew on LinkedIn
• Andrew's Talk at ICML
• Learn more about Shopify Sidekick
• Follow Fatima on LinkedIn

Referenced in today's show:

• The Economics of Software Innovation
• “AI First” and the Bus Factor of 0
• GitLab 18.3 released with Duo Agent Platform in Visual Studio (Beta) and Embedded views

Support the show:

• Subscribe to our Substack
• Leave us a review
• Subscribe on YouTube
• Follow us on Twitter or LinkedIn

Offers:

• Learn about Continuous Merge with gitStream
• Get your DORA Metrics free forever