DataFramed #289 How I Nearly Got Fired For Running An A/B Test with Vanessa Larco, Former Partner at New Enterprise Associates
19 snips
Mar 6, 2025 
In this engaging discussion, Vanessa Larco, a former partner at NEA with an impressive background in product leadership from giants like Microsoft and Disney, shares her insights on A/B testing. She explores the challenges of balancing short-term gains with long-term user experience, particularly in gaming. Vanessa also highlights the pitfalls of misaligned metrics and the need for clear communication and cross-team collaboration. Her anecdotes underline the importance of understanding user behavior and leveraging data to drive meaningful changes in product development.
AI Snips
Chapters
Transcript
Episode notes
Playdom's A/B Testing Culture
- Vanessa Larco's first A/B testing experience was at Playdom during the social gaming boom.
- They prioritized data-driven decisions, A/B testing everything from game themes to mechanics.
Unpredictable User Preferences
- A/B testing revealed user preferences that the team wouldn't have predicted.
- The Gnome Town theme unexpectedly won, highlighting the data's power.
A/B Testing Limitations
- A/B testing has limitations, especially regarding long-term strategy.
- It excels in optimizing existing elements, but not for developing entirely new products.