Finding Market Fit: Marketing Leaders in Tech

Measurement capabilities of world class advertisers, incrementality testing, media mix models, and when to use attribution | Andrew Covato (Growth by Science, Snap, Netflix)

Nov 21, 2023

Andrew Covato, an experienced Ad and MarTech professional, discusses the varying measurement methods of advertisers, the use of persistent holdouts, national media approach, and effective measurement methods. Topics include the history of advertising, evolution of measurement capabilities, running tests for marketing efficacy, challenges in brand and national campaigns, understanding look back periods, and the importance of curiosity in marketing.

49:32

Creator website

Episode guests

Andrew Covato

AI Summary

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

Implementing persistent holdouts is recommended for ongoing assessment of marketing efficacy and understanding true ROI of campaigns.

Assessing the impact of brand campaigns and non-targetable channels requires a nuanced approach, considering all campaigns as performance campaigns, employing marketing mix modeling (MMM), and analyzing pre and post impacts.

Deep dives

Focus on Incrementality Testing for Marketing Efficacy

Implementing persistent holdouts is recommended as an ongoing assessment of the overall marketing efficacy program. This approach helps determine whether the marketing efforts are delivering incremental results or not. By setting a constant holdout and analyzing the data, businesses can understand the true ROI of their campaigns. While it might not be feasible for all companies, having a regular cadence of performing tests, whether quarterly or monthly, is encouraged. A regular check-in on the marketing program helps identify areas for improvement and optimization. It is important to keep the methodology simple, specific, and focused on the most important lower-funnel key performance indicators (KPIs) like sales, revenue, or profitability. Recycling holdouts at a quarterly or annual frequency is recommended, considering market dynamics and the life cycle of the product or service. While implementing persistent holdouts can be more challenging for brand campaigns and non-targetable channels such as linear TV or podcasts, approaches like marketing mix modeling (MMM) can provide insights into incrementality. Overall, the goal is to strive for measurement methodologies that derive the causal impact of ads, understand the true efficacy of the marketing program, and continuously improve.

Introduction

2min

Evolution of Measurement Capabilities in Advertising

28min

Frequency of Running Tests for Marketing Efficacy

9min

Challenges and Approaches to Brand and National Campaigns, Affiliates, and Podcast Ads

6min

Understanding Look Back Periods, Incrementality, and Durations of Impact in Campaigns

3min

Importance of Curiosity in Marketing and Availability for Discussion

2min

Andrew has over fifteen years of Ad and MarTech experience at eBay, Google, Meta, Netflix, Snap, and fintech startups. He has tackled hard challenges in product, strategy, growth, go-to-market, and partnerships across many verticals and geographies. He is highly strategic, and has experience building and applying buy- and sell-side ad technology at scale. He has been advising companies from start-ups to Fortune 500 enterprises since 2012.

---

We talk about:

Varying measurement methods and when to lean in on one or two of them
How the most sophisticated advertisers use a persistent hold out
The best way to approach media that has to run nationally
The differences between measuring advertising and channel effectiveness and optimization impact
The variables needed to structure effective measurement methods

---

Where to find Andrew:

Andrew's LinkedIn Profile: https://www.linkedin.com/in/andrewcovato/

Growth by Science: https://www.growthbyscience.com/

---

We reference:

https://www.measured.com/blog/geo-testing-series-test-design/

https://www.measured.com/blog/geo-testing-series-part-2-test-management/

https://www.measured.com/blog/geo-testing-series-part-3-analytics-and-reporting/

---

Where to find Patrick:

Patrick's LinkedIn Profile: https://www.linkedin.com/in/pcmoran/

---

(2:07) History of advertising and contextualizing the ideal measurement stack

(7:40) Options for measurement - Geo testing, using media mix models, time series analysis, and newer versions of multi-touch attribution

(12:55) Using a measurement tax effectively to value advertising and the consequences of using attribution

(15:16) The most effective ways to impact advertising using creatives, reach, and frequency and not playing around with bidding

(16:50) Using upper funnel metrics to inform optimization approaches

(18:54) Situations where attribution is applicable

(21:46) Budget approach required to run these tests and committing to the process to detect signal

(25:44) Approaching marketing level impact first and then measuring channel impact

(30:26) Using a constant holdout, when it works and when it doesn't work

(33:39) Simplicity and specificity when setting this up

(37:28) The three main variables on structuring a persistent hold out

(39:39) Measuring brand advertising

(45:41) Perspective on using varying look back periods

(47:56) Where to find Andrew!

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.

Finding Market Fit: Marketing Leaders in Tech

Measurement capabilities of world class advertisers, incrementality testing, media mix models, and when to use attribution | Andrew Covato (Growth by Science, Snap, Netflix)

Episode guests

Podcast summary created with Snipd AI

Quick takeaways

Deep dives

Focus on Incrementality Testing for Marketing Efficacy

Operationalizing Persistent Holdouts

Analyzing Brand Campaigns and Non-Targetable Channels

Remember Everything You Learn from Podcasts