A/B testing plays a critical role in decision-making - but so does AB Testing Statistics Mistakes. In data-driven companies, A/B tests not only prove positive impacts of product features (treatments) but also provide evidence to safely roll out neutral features that are not expected to move business metrics. When rolling out a neutral feature, we need to test if the treatment is non-inferior to control, and conventional A/B testing is not suitable in this case. In my talk, I present when conventional A/B testing would fail, what the non-inferiority test is, and how is non-inferiority test better when testing treatment is not worse than control. We've published this topic here https://bit.ly/3vmFsYp About the speaker: Max is a senior data scientist at Wish where he focuses on experimentation (A/B testing) and machine learning. He has been improving the A/B testing platform at Wish on various fronts, including infrastructure, statistical testing, usability, etc. His passion is to empower data-driven decision-making through the rigorous use of data. Max earned his Ph.D. in Statistical Informatics from the University of Arizona.

Exploring the impact of filtering out bad testing ideas, collaboration in CRO for valuable insights, challenges of statistics in behavioral science, differentiating between exploratory and confirmative experiments, applying consumer psychology to CRO, optimizing user behavior through behavioral science experiments, and the ethical side of digital experimentation.

Being an Experimentation leader requires a different set of skills. Vistaprint's Cristina Pujol walks through the lessons she's learned. 0:00 - Introduction 1:03 - How Vistaprint organizes for Experimentation 2:22 - The Ambassador role 4:19 - Experimentation process for domains 8:18 - Let’s take a real example 11:36 - Ideas to share learnings and celebrate success #leadership #program #experimentation

Does exactly what it says on the tin. #research #conversion #optimization

Our industry can be damning of those doing things ""imperfectly"" - but it seems not to draw the distinction between those doing things imperfectly through ignorance (who can be educated) and those doing things imperfectly through lack of a better option (who need to be helped to show the value to decision-makers).   This attitude has a negative impact on new entrants to the industry and sets completely the wrong tone for the next generation. The greater the participation in the industry, the stronger it will become - the wider that participation, the more ideas (and the more diverse those ideas) will come through. And that is only a good thing for us all longer-term.  About Matt: Matt Scaysbrook,  Founder & Director of Optimisation at WeTeachCRO Matt has been optimising online experiences across B2B & B2C industries for over 10 years. He has a great passion for guiding & educating businesses on the value of Conversion Rate Optimisation. He places the understanding of human behaviour at the heart of all of his work, knowing that that breaks all problems down to the lowest common denominator – us. Clients include Nando’s, GoDaddy and Sage.  Chapters --- 0:00 - Introduction 0:55 - The quality of our community is a function of quantity and diversity 2:53 - The purist 8:09 - The pragmatist 12:58 - So what? 16:36 - We need the best of both

How to be a good test coder with Jeroen Wiersma

In this session, I want to cover the most strategic actions that a new Experimentation Strategist (or Program Manager) can execute during their first three months in the role to increase trust among the stakeholders and clients.  I will share this based on my experience working with clients such as Miro, Procore, The Bouqs, and Gympass, among others.  About the speaker Carlos has over nine years of experience working in online marketing and mass media. As an agency founder, he has led web development and conversion optimization projects for more than 80 brands in multiple regions and verticals. It has provided him with broad expertise in conversion optimization, user research, ecommerce strategy, digital psychology, and data analysis that let him contribute to driving innovation that delivers more profits.  As an Experimentation Strategist (former Program Manager) at Speero, he has been actively involved in running the Experimentation Programs of fast-growth companies such as Miro, Procore, The Bouqs, and Gympass, among others.

You don't have to A/B test everything. Be smart by having guiding principles.  #conversionrateoptimization #experimentation #ecommerce

Many optimizers are good at using data to craft hypotheses. Great optimizers, however, build data-driven segments, analyze by business goals (not just test KPIs), and importantly, propose how to take action on campaign results.  About the speaker Alexandra leads Product Marketing at Kameleoon. She loves storytelling as a way of demonstrating a product’s values and benefits to customers.  Chapters --- 0:00 - Introduction 3:41 - Great experimenters consolidate their customer data for precise and persistent segmentation 6:40 - Great experimenters analyze by meaningful business goals and draw actionable conclusions 8:05 - Great experimenters take action on their experimentation results 10:29 - Coming to full circle

Having mature conversations with immature markets, the problem with the word, “Hypothesis”, the importance of HiPPOs with Nima Yassini  #experimentation #conversionrateoptimization #growth