Shopify’s Sidekick Just Leveled Up: AI That Runs Your Store (Proactively)

Dec 25, 2025

Andrew McNamara, Shopify's Director of Applied Machine Learning, shares insights from his 15-year journey in AI, emphasizing the evolution from basic assistants to sophisticated LLMs. He discusses how Sidekick has transformed into a proactive AI co-founder that democratizes data for merchants. Andrew introduces Sidekick Pulse, highlighting its ability to uncover critical business insights, and explains SimGym, which allows merchants to A/B test with AI shoppers without risking live traffic. The conversation touches on 'vibe entrepreneurship' and the exciting future of AI in e-commerce.

Ask episode

AI Snips

Chapters

Transcript

Episode notes

INSIGHT

LLMs Are A Powerful Evolution, Not A Rewrite

LLMs perform similar intent and slot-filling tasks as earlier assistants but with far greater capability today.
Andrew McNamara says past assistant experience directly shaped what Shopify built with Sidekick.

INSIGHT

Natural-Language Analytics Democratizes Data

Embedding natural-language analytics into admin democratises data for merchants who can't write queries.
McNamara says Sidekick makes complex multi-step insights accessible and actionable for small teams.

ADVICE

Use LLMs To Scale Product Evaluation

Use an "LLM as a judge" to scale evaluation by training models to match product experts' judgments.
Validate the judge with statistical rigor before automating feature prioritization and deployment checks.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

How Shopify Is Building The Future of AI Commerce with Andrew McNamaraShopify's Director of Applied ML, Andrew McNamara, reveals how the new Sidekick Pulse and SimGym features are revolutionizing e-commerce for merchants of all sizes. With 15 years of experience building AI assistants—dating back to pre-Siri days—Andrew breaks down how Shopify is using "LLM as a judge" to ensure quality and why "vibe entrepreneurship" is the future of business.Andrew explains how Sidekick has evolved from a simple chatbot into a proactive "AI Co-founder" that can democratize data for small business owners. He shares behind-the-scenes details on Sidekick Pulse, which performs deep research to surface actionable insights (like finding shipping errors that cost sales), and Simgym, a powerful simulator that uses AI shoppers to A/B test store changes before they go live.We also dive into the technical side of how Shopify evaluates these models using statistical rigor and why the ability for merchants to build admin apps with a single prompt is a game-changer for productivity.---Topics Covered- The 15-year evolution of AI assistants from rule-based systems to LLMs- How Sidekick Pulse proactively finds and fixes business-critical errors- Simgym: Using AI shoppers to simulate A/B tests without risking live traffic- The "LLM as a Judge" framework Shopify uses for product quality control- "Vibe Entrepreneurship" and reducing technical barriers for founders- Building custom Admin Apps in seconds using natural language prompts- Real-world examples from Andrew's own maple syrup store- The difference between general chatbots (Copilot) and specialized agents (Sidekick)- How "App Gen" allows merchants to create custom workflows instantlyEpisode Timestamps00:03 - Introduction to Andrew McNamara and his 15-year history with AI01:01 - Comparing early AI assistants (BlackBerry/Samsung) to modern LLMs06:45 - How Sidekick democratizes data analytics for small merchants11:25 - Deep Dive: Sidekick Pulse and proactive business research15:43 - Using "LLM as a Judge" to replace human evaluation at scale18:41 - Generating custom Admin Apps with a single prompt ("App Gen")21:58 - Andrew's personal experience running a Shopify store25:31 - The concept of "Vibe Entrepreneurship" as a North Star26:30 - Using AI to edit online store themes and layouts in real-time37:39 - Simgym: Simulating buyer behavior to predict experiment results42:06 - Why simulation is critical for both small and large enterprise merchants48:04 - The culture of technical depth and passion at Shopify51:34 - Why Andrew has dedicated his entire career to building assistants---## About Andrew McNamaraAndrew McNamara is the Director of Applied Machine Learning at Shopify, where he leads the development of Sidekick and other intelligent merchant features. Previously the Director of the Montreal Research Lab at Microsoft and a key contributor to Bing Chat (Copilot), Andrew has over 15 years of experience building and deploying AI assistants at scale.Shopify is the leading global commerce company, providing trusted tools to start, grow, market, and manage a retail business of any size. It powers millions of businesses in more than 175 countries and offers a unified platform for physical and digital commerce.Resources Mentioned- Shopify Sidekick (AI Commerce Assistant)- Sidekick Pulse (Proactive Research Agent)- Simgym (AI Shopper Simulator)- Microsoft Copilot & Bing Chat- "LLM as a Judge" Evaluation Framework---Partner Links- Book Enterprise Training — https://www.upscaile.com/- Subscribe to our free newsletter — https://www.theaireport.ai/subscribe-theaireport-youtube