Don't Be Afraid of Your First A/B Test ft. Kelly Wortham
Sep 11, 2024
auto_awesome
Kelly Wortham, an A/B testing expert, and Shiva Manjunath, a conversion rate optimization guru, dive into the world of experimentation. They tackle the fears surrounding your first A/B test and share real experiences from the CRO community. Learn what to prioritize and what to ignore when starting out, and explore simple tests that anyone can run. The discussion even heats up around the questionable use of ROI in evaluating test results, highlighting a more nuanced approach to A/B testing. Get ready to embrace the fun of experimenting!
Emphasizing that A/B tests don't need to be perfect, simple adjustments can build confidence and yield actionable insights.
Establishing supportive networks and a culture of experimentation reduces anxiety and fosters a continuous improvement mindset in testing.
Deep dives
The Value of A/B Testing in Business Success
A/B testing offers significant financial benefits, as highlighted by a successful example where a simple subject line test for an email resulted in substantial profits for Circuit City. The evolving tools and methodologies have made the process less labor-intensive since these early days of using Excel spreadsheets, making it accessible to more businesses. Emphasizing the importance of cross-functional collaboration, the conversation underscores that A/B tests can lead to major wins, but they also require proper coordination with other teams to avoid fallout. Ultimately, the pursuit of effective testing not only contributes to immediate financial gains but satisfies the organization's long-term strategic goals.
Overcoming the Fear of A/B Testing
Many professionals experience anxiety when considering their first A/B test due to fear of failure, political dynamics in the workplace, and a lack of clear ownership over the process. The emphasis on needing to produce perfect tests can paralyze individuals, causing them to hesitate rather than move forward with experimentation. By framing A/B testing as a series of incremental learning opportunities, individuals can mitigate anxiety and focus on achieving actionable insights rather than perfection. Additionally, establishing supportive networks, such as testing communities, can provide encouragement and share experiences that help alleviate fears.
First-Time A/B Test Ideas
The discourse highlights that first A/B tests do not need to be perfect or sophisticated; simple changes, like testing email subject lines or altering web copy, can be effective starting points. Examples of successful tests shared by community members illustrate diverse applications, from minor adjustments in landing pages to removing form fields to streamline user experience. Such straightforward tests build confidence and allow individuals to gather valuable data that can inform future experimentation. Emphasizing that not every hypothesis must be comprehensive, the conversation promotes a mindset of exploration over judgment.
Value of Continuous Improvement in Testing
Developing a culture of testing entails not only executing A/B tests but also improving the processes surrounding them over time. Leaders are encouraged to implement transparent methodologies to track results, understand failures, and ensure that learnings from each test are documented for future refinement. By creating a robust optimization program alongside running tests, teams will cultivate a more effective structure that leads to growing success rates and better insights. Ultimately, the focus remains on maintaining momentum towards continual improvement and fostering an environment where experimental learning thrives.
Terrified of running your first A/B test? We've all been there. While it may be super intimidating... you CAN do it! And we're here to help! Shiva Manjunath and Kelly "The TLC" Wortham. recorded an episode to help you reduce that anxiety of running your first test.
We got into:
- Asking the CRO community their first A/B tests
- Things to not care about in your first A/B test
- Some easy, and quick, A/B tests for you to run as your first
We also got into some interesting debate around using ROI calculations to calculate revenue for A/B tests in the "Shit You Need to Know" section...
Timestamps:
00:00 Start of Episode
1:42 Kelly's First Ever A/B Test
3:51 The CRO Community's First A/B Tests
9:34 CROs are Addicts... For Test Results
15:00 Shiva Gives Bad Advice for CROs
20:53 Things Which Are Important for Your First A/B Test (With Some Ideas to Run Your First Test)
35:07 Shit You Need to Know: Calculating ROI for Tests Results (this one got a little spicy... ;)
Go follow Kelly Wortham on LinkedIn (https://www.linkedin.com/in/kellywortham/), and go sign up for the TLC community - it's free, and SUPER fun. (https://testandlearn.community/). And go check out Experimentation Island, a conference on experimentation in February 2024 (https://experimentationisland.com/)
Also make sure you guys are following MJ Smith on LinkedIn - link to her post here: https://tinyurl.com/MJ-FromAtoB
If you have listener questions, submit them at https://tinyurl.com/askfromatob for a chance to be featured too!
Get the Snipd podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share & Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode