Experiment Nation: The Podcast

Learn how to get executive buy-in for your CRO program with Jonny Longden, who shares the importance of having a team with diverse skills in optimization. Discover the key ways experimentation can go wrong and why companies use CRO. Explore the significance of strategic thinking in experimentation for business growth.

Experimenters obsess about many A/B Testing Statistics concepts - but not all of them are worth worrying about. Statsig's Timothy Chan shares his thoughts about test isolation, interaction effects and confidence levels - some of his thoughts may surprise you. #experimentation #product #statistics

Client Relationship Tips and secrets behind a thriving relationship between a CRO agency and the client - anybody can learn how to get started in CRO but it takes years of experience to know how to run a CRO agency and how to scale while keeping the same standards of quality. In my presentation, I will walk you through what you need to know to build a strong, successful relationship with your clients and with your team. You will learn: - how to set expectations with your client from Day 1 - how to leverage operating procedures and automate your work - how to communicate with clients and get stakeholder buy-in every time Chapters --- 0:00 - Introduction 1:52 - The testing Masterfile 3:37 - The testing sheet template

10 lessons that helped John Ekman in Launching a CRO agency. John learned many lessons when launching his agency - he wants to share them with you.

After decades of acquisition marketing, companies are waking up to the fact that optimising the entire lifecycle of the customer is the way to go. That happens due to rising acquisition costs, cookie deprecation, siloes, company mentalities, etc. In my session, I will share the insights that I got during my 12 years journey, transitioning from CRO - website optimization to increase the conversion rate to CVO - experimenting towards improving the CLV. About the speaker: I am the CEO & Founder at Omniconvert, a solution that helps eCommerce companies all over the world become customer-centric and thrive in a cookie-less future. I am also the founder of Omniconvert CVO Academy where he helps marketers become certified in CVO through his teaching of the importance of customer lifetime value. On top of this, he is writing, blogging, podcasting, and advising for brands. I have a genuine calling to help others and share the shortcuts I discover on my journey. Chapters --- 0:00 - Introduction 3:59 - Experimentation fatigue 4:53 - A/B Testing is finding the one size that fits all 5:29 - The dominant numbers in eCommerce 8:50 - Aha moment 10:07 - RFM customer segmentation 12:50 - The soulmates 14:12 - Qualitative and Quantitative 17:23 - The CVO Methodology 20:06 - The conversion rate can be a misleading metric in eCommerce 21:53 - Customer value optimization 22:20 - The 4 types of CVO campaigns 24:53 - 4 reasons why the CRO experts are in pole position to nail CVO

Message-Market fit is the idea that a message, whether it be a product, service or idea, needs to resonate with a specific target market in order to be successful. It involves finding the right message that effectively communicates the value proposition of a product or service to a particular audience, in a way that resonates with them. This process involves identifying the unique needs and preferences of the target audience, understanding their pain points and motivations, and crafting a message that speaks directly to them. The message should be clear, concise, and compelling, and should effectively communicate the benefits of the product or service. Achieving message-market fit requires a deep understanding of the target audience and ongoing feedback and iteration to refine the message until it resonates with the market. When a message effectively resonates with a target market, it can lead to increased engagement, loyalty, and ultimately, growth. It is important to note that message-market fit is not a one-time event, but rather an ongoing process that requires continual testing, refinement, and adaptation to changing market conditions. Businesses that prioritize achieving message-market fit are more likely to create products or services that are successful in the long term. About Daphne Tideman: Teaching D2C Startups how to grow faster | 10x the subscriber base of a startup in 18 months | Worked with 200+ brands | Eco-friendly and wellness brands specialist | Author Linkedin: https://www.linkedin.com/in/daphnetideman/#marketing #growth #messaging #experimentation

A/B testing plays a critical role in decision-making - but so does AB Testing Statistics Mistakes. In data-driven companies, A/B tests not only prove positive impacts of product features (treatments) but also provide evidence to safely roll out neutral features that are not expected to move business metrics. When rolling out a neutral feature, we need to test if the treatment is non-inferior to control, and conventional A/B testing is not suitable in this case. In my talk, I present when conventional A/B testing would fail, what the non-inferiority test is, and how is non-inferiority test better when testing treatment is not worse than control. We've published this topic here https://bit.ly/3vmFsYp About the speaker: Max is a senior data scientist at Wish where he focuses on experimentation (A/B testing) and machine learning. He has been improving the A/B testing platform at Wish on various fronts, including infrastructure, statistical testing, usability, etc. His passion is to empower data-driven decision-making through the rigorous use of data. Max earned his Ph.D. in Statistical Informatics from the University of Arizona.

Exploring the impact of filtering out bad testing ideas, collaboration in CRO for valuable insights, challenges of statistics in behavioral science, differentiating between exploratory and confirmative experiments, applying consumer psychology to CRO, optimizing user behavior through behavioral science experiments, and the ethical side of digital experimentation.

Being an Experimentation leader requires a different set of skills. Vistaprint's Cristina Pujol walks through the lessons she's learned. 0:00 - Introduction 1:03 - How Vistaprint organizes for Experimentation 2:22 - The Ambassador role 4:19 - Experimentation process for domains 8:18 - Let’s take a real example 11:36 - Ideas to share learnings and celebrate success #leadership #program #experimentation

Does exactly what it says on the tin. #research #conversion #optimization