133: Simon Heaton: Buffer’s Director of Growth Marketing on agile sprints, holdout testing and why a CRM or GA4 isn't in their tech stack

What’s up everyone, today we have the pleasure of sitting down with Simon Heaton, Director of Growth Marketing at Buffer.

Summary: Simon helps us explore Buffer's martech journey, highlighting their shift from traditional tools to a product-led approach driven by data and server-side analytics. We unpack their use of Customer.io for automation and hold out testing, Redash for data insights, and their agile sprint model that fosters continuous innovation. Discover how Buffer's small team thrives with efficient, data-driven strategies.

About Simon

• Simon started his career in the agency world at Banfield in Ottawa, Canada
• He later moved over to Shopify where he would spend nearly 7 years, first as a content Marketing Manager and later as the Senior Growth Lead, Acquisition
• Simon’s also worn a part-time teaching hat for over 5 years, he was an Instructor with Telfer School of Management at UofO as well as a Professor at Algonquin College
• He’s a startup mentor for founders that are part of the Singapore-based equity fund at Antler
• Today Simon is Director of Growth Marketing at Buffer, the world-renowned social media management platform

Buffer’s Marketing Tech Stack and Why it Doesn’t Include a CRM

Buffer’s marketing strategy is unique. They don’t use a traditional CRM like HubSpot or Salesforce. Simon explains that Buffer is a product-led company without a dedicated sales team. This means they don't need typical CRM functionalities like lead routing and scoring. Instead, Buffer relies heavily on data and product analytics to drive their marketing efforts.

The core of Buffer’s operations is their data warehouse, with Segment acting as their Customer Data Platform (CDP). This setup allows Buffer to integrate various tools and centralize crucial information. Mixpanel, their product analytics tool, is pivotal in this system. It gathers both product usage and marketing data, providing a comprehensive view of user interactions.

Simon highlights the importance of server-side tracking and integrating data from diverse sources such as AdWords, Customer.io, and Pendo. This integration helps Buffer understand the user lifecycle and measure the impact of marketing efforts beyond basic website metrics.

Tools like Customer.io are also essential for Buffer. It manages most user communications, making it a critical component of their stack. The combination of Mixpanel, Customer.io, and other integrated tools ensures that Buffer can seamlessly track and analyze user behavior.

Key takeaway: Not all B2B companies need a CRM or a sales team. A product-led approach, using robust data and product analytics tools, can effectively drive your marketing efforts and provide comprehensive insights into user behavior.

The Power of a Visual and Intuitive Automation Flow Interface

Simon loves working in a smaller team like Buffer, where he can get hands-on with their tools daily. He highlights how Buffer uses Customer.io for their marketing automation, a tool he's familiar with from his previous experience at Shopify. Unlike Shopify, which eventually switched to Salesforce Marketing Cloud for more enterprise-level needs, Buffer continues to thrive with Customer.io.

Buffer relies on Customer.io to manage email marketing, push notifications for mobile apps, and various communication programs. Simon appreciates how the tool handles both marketing and transactional communications, offering a unified view of user interactions. This integration ensures consistency in messages, whether they're marketing emails or product notifications.

Simon praises Customer.io's user-friendly interface, especially the journey mapping functionality and the WYSIWYG editor, which make it accessible for non-technical team members. Despite its ease of use, the platform also boasts deep technical capabilities, allowing for extensive customization through HTML and API integrations. This flexibility has been crucial for Buffer's needs.

The integration with Segment, Buffer's Customer Data Platform (CDP), is particularly valuable. Simon emphasizes that having all data in Segment and seamlessly integrating it with Customer.io enables precise data handling. This setup ensures accurate and timely data flow, essential for personalized and effective marketing automation workflows.

Key takeaway: Even as a small team, you can effectively manage complex marketing automation needs by choosing user-friendly tools like Customer.io that offer both simplicity and deep customization. This approach allows your non-technical team members to contribute meaningfully while ensuring your technical needs are met, enhancing overall efficiency and personalization in your communications.

Experimentation and Holdout Testing at Buffer

Experimentation is a cornerstone of Buffer’s approach, and Simon is particularly enthusiastic about the capabilities provided by Customer.io. He explains that the platform's holdout testing functionality is essential for validating new programs and comparing campaign performance. Unlike some tools, Customer.io counts a delivery for the holdout group, simplifying the tracking process over time.

The integration with Segment and Mixpanel is a game-changer for Buffer. This setup allows them to surface Customer.io data in Mixpanel, creating unique reports and dashboards to support their experiments. Tracking differences in behavior between groups becomes straightforward, thanks to the detailed delivery events logged for both test and holdout groups. This level of detail ensures that Buffer can effectively measure the impact of their campaigns.

Simon also highlights the ease of A/B testing within Customer.io. Whether at the message level or within workflows, the platform’s randomization logic allows for extensive testing. Buffer can run tests on content, sequencing, and other variables, ensuring they continually optimize their marketing efforts. The ability to branch workflows and test different variants simultaneously is particularly valuable, enabling ongoing experimentation.

Key takeaway: Leverage holdout testing and detailed event tracking within your marketing automation tools to gain deeper insights into your campaign effectiveness. This approach allows you to validate new programs, compare performance, and optimize your strategies based on precise, data-driven insights.

Testing Journeys and Templating Language with QA Draft Mode

Simon praises Customer.io's QA draft mode, a feature he finds invaluable for Buffer’s marketing automation. This functionality allows the team to build complex workflows, trigger off specific data points, and test the entire process in a production environment without actually sending emails. It’s a unique capability that Simon has not found in other tools, making it a standout feature of Customer.io.

Simon highlights how QA draft mode lets them see real users qualifying for different branches of the workflow while emails remain in draft. This means they can verify that users are correctly segmented and the emails look as intended, all without prematurely sending any messages. This testing phase is crucial for catching errors that might not be evident during initial previews.

Buffer has used this feature for several initiatives, such as new onboarding iterations and product notifications. Given the high frequency and volume of these emails, ensuring everything works perfectly before going live is essential. Simon appreciates that once the testing phase is complete, it only takes a click to start sending the validated emails to users.

This capability saves time and reduces the risk of errors in live campaigns. It allows Buffer to maintain high st...