Getting the Most Out of Your Tests

Testing and experimenting is one of the best ways to find what works best for your business. This week we are talking about the benefit of experiments and some of my tips for how to do this on your own in your business, as not every experiment requires hiring a consultant to come in and run a big study.

If you have been listening to the podcast for a while you probably remember me mentioning a few times that I have a research paper coming “soon” based on a project I did on behalf of the Filene Research Institute and a credit union in Portland, Oregon. I am so excited that I have approved the (potentially) final draft and that should be published any day now! 

There will be a dedicated episode talking about the research in two weeks, but I wanted to give some tips about experimenting first, because it truly is so important for every organization to test things. Next week is an episode on the physical representation of concepts, which is the only main concept in the study I have not yet covered on the podcast. Then on September 13th, I’ll share all the details and findings of my research study.

Show Notes:

• [05:08] Experimenting is important for any organization. You have probably done experiments and not even realized it.
• [05:30] Good experiments need to be narrow and focused, because if you test too many things at once you won't know what contributed to the result.
• [07:09] I share a story about how I needed to continue to narrow down a research paper topic. 
• [08:16] The study found that advertisements for low cognition products were twice as likely to be standardized as high cognition products. Ads using pictures were more than twice as likely to be standardized as those using text.
• [09:08] There were so many variables and items cross referenced just for a study that looked at one month of magazine ads.
• [09:59] The study I ended up with, which felt incredibly small to me at the time, was actually a huge undertaking - it was a true experiment.
• [10:16] When there is a lot weighing on the outcome of the experiment, it's a good idea to bring in experts. There are also tests you can do on your own fairly easily, which can still have a great impact on your business.
• [10:41] You can be more agile and adapt quickly with small tests.
• [10:56] The three things to keep in mind when setting up experiments are to be thoughtful, keep it small, and test as often as you can.
• [11:14] Keeping it small allows you to do the test on your own and understand what contributed to the results you are seeing.
• [11:55] To determine what is best, separate everything into multiple mini-tests.
• [12:53] Make one small change and track what the results are, so you'll be learning every step of the way.
• [14:44] One of the studies I share the most often is the one with the end cap displays for Snickers bars. This used anchoring and adjustment and found when they said “buy 18 for your freezer” there was a 38% increase in sales.
• [15:36] Behavioral economics shows us that hunches about what customers will do are often wrong, because they are based on logic, not the rules of the subconscious brain. This is why everything needs to be tracked.
• [16:01] Some other things you could test would be how your ads (or emails or direct mailers or website pages) do when you change a number frame. You can also do tests on blog post headers, or copy on social media posts, and images you use on ads.
• [17:04] The second important way to focus your attention is to be thoughtful. Being thoughtful means looking outside of what you always do or what you “know” to be true.
• [17:45] Behavioral economics teaches that humans do not always act “rationally” or with much forethought. Take the time to plan before you jump into a test, or start testing absolutely everything.
• [18:45] Instead of testing everything, just test the right things. Know the problem you are trying to solve and narrow your focus.
• [19:15] Anything can be worth testing, but everything can be a waste of time if you don’t have a clear focus and goal.
• [19:36] If your company is about driving value, then all your tests should be about creating more value for your customers.
• [20:16] Focus on items that are driving revenue and value to your company.
• [21:01] The results of one test will not necessarily hold true in every situation or for every business. This is called generalizability or being generalizable - while it matters in most academic studies, it isn’t as important if you are testing for your own business (because if the results don’t apply to your competitor...who cares?)
• [22:05] It's also important to know whether the data you are collecting is qualitative or quantitative. Conversations with people are qualitative, number of clicks are quantitative.
• [24:00] If you know you will want to dig into demographics and other details, you probably need to build that into your data pull up front. Think and talk through what you actually want to know.
• [24:35] My final tip is to test early and often.
• [25:24] Small tests let you act quickly, so the more you test, the more you learn. No results still tells you something important.
• [26:25] Findings tell you what attracts attention or what matters to your customers, but non-findings tell you what they don’t pay attention to or care about. Remember to keep it small, be thoughtful, and test often.

Remember - The Brainy Business now offers courses!

Thanks for listening. Don’t forget to subscribe on Apple Podcasts or Android. If you like what you heard, please leave a review on iTunes and share what you liked about the show. 

Links and Resources:

• Filene
• The CUInsight Experience podcast: Melina Palmer – Seeking answers (#34)
• Episode 4. Questions or Answers
• Behavioral Economics and Business
• 048 How Applying Behavioral Economics Can Benefit Your Business, With Melina Palmer of Brainy Business Podcast
• Bam Success Summit Conference
• Your Sales Maven
• Association for Consumer Research
• Episode 60. Surprise and Delight
• Episode 11. Behavioral Economics Foundations: Anchoring and Adjustment
• 1 Word That Increased Sales by 38%
• Episode 16. Behavioral Economics Foundations: Framing
• Qualitative vs Quantitative