The Unofficial Shopify Podcast Native A/B Testing Hits Shopify: What You Need to Know
Dec 22, 2025
Aaron Glazer, Product Lead at Shopify, shares insights on two groundbreaking features for merchants: Rollouts and SimGym. Rollouts allows for native split testing directly in the Shopify admin, empowering merchants with data-driven decisions. SimGym employs AI to simulate shopper behavior, identifying usability issues before they impact real customers. Glazer emphasizes the importance of bold testing and offers practical advice on how to leverage these tools for effective store optimization.
AI Snips
Chapters
Transcript
Episode notes
Native Split Testing Inside Shopify
- Rollouts brings native split testing into Shopify's theme editor so merchants can test storefront variations without external tools.
- It measures impact directly in Shopify admin so decisions are data-driven and faster.
Schedule Theme Changes For Sales
- Use Rollouts from the theme editor to test your original theme versus edits and schedule changes at specific times.
- Schedule launches and automatic rollbacks for sales to avoid manual midnight publishing.
Use Built-In Results To Trust Tests
- Rely on the built-in analytics and Rollouts results pages to view statistical significance and avoid guessing.
- Let Shopify present significance in an accessible way so you can act confidently on test outcomes.