Experiment Nation: The Podcast

Teams are horrible at this! They have no idea it should happen, and if they do, they're usually lost and the approach is terrible. When you break it down, it's really not difficult. This shouldn't be a black box anymore.   I will cover the following: (1) What each of the test planning aspects are and how to do them including MDE and duration estimates, (2) CXL's pre-test calculator unless our new Optimizely one is live by the time I record, (3) how to turn pre-test calculations into a testing roadmap, and (4) how to determine when a business can't test or when a swimlane isn't viable so teams can stop wasting time.  Chapters --- 0:00 - Introduction 1:50 - Imagine you’re with your team building an experimentation program 2:30 - Common struggles 6:26 - What is pre-test planning? 6:56 - Primarily…MDE and Duration estimates 26:28 - Roadmapping 28:38 - Should we test everywhere? Should we test at all? 31:00 - Example scenario 32:20 - Wrap up

Skeptical first-time buyers need to be given the utmost priority in order to make them feel comfortable and secure with their purchase. This may include providing detailed product or service information, offering multiple payment options, providing customer service support, and offering warranties or guarantees for their purchase. Rishi shares his thoughts on how to win over this critical customer segment.

Stop testing randomly. Have a plan.

CRO is a process of optimizing a website or landing page to maximize the percentage of visitors to convert into paying customers. It involves testing different elements of the website to determine what works best for increasing conversions. As long as businesses value increasing their conversion rate, CRO will remain an important part of online marketing and web design - but it may not exist as a profession one day. Our new podcast host, Richard Joe, asks Ton about the culture of Experimentation, and how we CRO will die...one day.

There’s a lot of talk about how to find things to test, how to analyze tests and the importance of iterating on tests. This week, I was thinking about how I could add something a little different to the conversation and decided to share my thoughts about when to run a test - particularly when building products.

Curiosity needs to be a driver in every experimenter’s toolkit. What is going on in the head of my customer? What is really their job-to-be-done? When buying flowers, we typically think of the standard occasions: Mother’s Day, Valentine’s Day, new baby & sympathy. But if we were to structure our ecommerce website on those occasions alone, we may be addressing the majority, but we’d be missing out on the real motivation behind the purchase as well as other reasons people buy flowers.  By deconstructing the customer’s journey of buying flowers, we discovered a piece of data that was sitting in front of us the entire time… the card message. These wonderfully written messages were our customers literally telling us the reason behind their purchase. It drastically changed how we approached our ecommerce website, now centered around the gifter’s journey.

Even if you are the best Experimenter at your company, unless you are able to nurture a culture of Experimentation - you’ll end up constantly fighting uphill battles. Here are a few quick pointers on how to be a better Experimentation leader.

Pineapple on pizza. If you ever want to have a heated debate with a group of people, ask them if pineapple belongs on pizza. Now let’s replace pizza with CRO and pineapple with “best practices” (or whatever topic CROs are angry about on LinkedIn these days).

Watch this session on Youtube: https://youtu.be/JqT95k7eG_8

About this series: Rommil Santiago, the founder of Experiment Nation shares his most recent thoughts about the industry and the craft of Experimentation in less than 5 minutes.