174: Joshua Kanter: A 4-time CMO on the case against data democratization

What’s up everyone, today we have the pleasure of sitting down with Joshua Kanter, Co-Founder & Chief Data & Analytics Officer at ConvertML.

Summary: Joshua spent the earliest parts of his career buried in SQL, only to watch companies hand out dashboards and call it strategy. Teams skim charts to confirm hunches while ignoring what the data actually says. He believes access means nothing without translation. You need people who can turn vague business prompts into clear, interpretable answers. He built ConvertML to guide those decisions. GenAI only raises the stakes. Without structure and fluency, it becomes easier to sound confident and still be completely wrong. That risk scales fast.

About Joshua

Joshua started in data analytics at First Manhattan Consulting, then co-founded two ventures; Mindswift, focused on marketing experimentation, and Novantas, a consulting firm for financial services. From there, he rose to Associate Principal at McKinsey, where he helped companies make real decisions with messy data and imperfect information. Then he crossed into operating roles, leading marketing at Caesars Entertainment as SVP of Marketing, where budgets were wild.

After Caesars, he became a 3-time CMO (basically 4-time); at PetSmart, International Cruise & Excursions, and Encora. Each time walking into a different industry with new problems. He now co-leads ConvertML, where he’s focused on making machine learning and measurement actually usable for the people in the trenches.

Data Democratization Is Breaking More Than It’s Fixing

Data democratization has become one of those phrases people repeat without thinking. It shows up in mission statements and vendor decks, pitched like some moral imperative. Give everyone access to data, the story goes, and decision-making will become magically enlightened. But Joshua has seen what actually happens when this ideal collides with reality: chaos, confusion, and a lot of people confidently misreading the same spreadsheet in five different ways.

Joshua isn’t your typical out of the weeds CMO, he’s lived in the guts of enterprise data for 25 years. His first job out of college was grinding SQL for 16 hours a day. He’s been inside consulting rooms, behind marketing dashboards, and at the head of data science teams. Over and over, he’s seen the same pattern: leaders throwing raw dashboards at people who have no training in how to interpret them, then wondering why decisions keep going sideways.

There are several unspoken assumptions built into the data democratization pitch. People assume the data is clean. That it’s structured in a meaningful way. That it answers the right questions. Most importantly, they assume people can actually read it. Not just glance at a chart and nod along, but dig into the nuance, understand the context, question what’s missing, and resist the temptation to cherry-pick for whatever narrative they already had in mind.

“People bring their own hypotheses and they’re just looking for the data to confirm what they already believe.”

Joshua has watched this play out inside Fortune 500 boardrooms and small startup teams alike. People interpret the same report with totally different takeaways. Sometimes they miss what’s obvious. Other times they read too far into something that doesn’t mean anything. They rarely stop to ask what data is not present or whether it even makes sense to draw a conclusion at all.

Giving everyone access to data is great and all… but only works when people have the skills to use it responsibly. That means more than teaching Excel shortcut keys. It requires real investment in data literacy, mentorship from technical leads, and repeated, structured practice. Otherwise, what you end up with is a very expensive system that quietly fuels bias and bad decisions and just work for the sake of work.

Key takeaway: Widespread access to dashboards does not make your company data-informed. People need to know how to interpret what they see, challenge their assumptions, and recognize when data is incomplete or misleading. Before scaling access, invest in skills. Make data literacy a requirement. That way you can prevent costly misreads and costly data-driven decision-making.

How Confirmation Bias Corrupts Marketing Decisions at Scale

Executives love to say they are “data-driven.” What they usually mean is “data-selective.” Joshua has seen the same story on repeat. Someone asks for a report. They already have an answer in mind. They skim the results, cherry-pick what supports their view, and ignore everything else. It is not just sloppy thinking. It’s organizational malpractice that scales fast when left unchecked.

To prevent that, someone needs to sit between business questions and raw data. Joshua calls for trained data translators; people who know how to turn vague executive prompts into structured queries. These translators understand the data architecture, the metrics that matter, and the business logic beneath the request. They return with a real answer, not just a number in bold font, but a sentence that says: “Here’s what we found. Here’s what the data does not cover. Here’s the confidence range. Here’s the nuance.”

“You want someone who can say, ‘The data supports this conclusion, but only under these conditions.’ That’s what makes the difference.”

Joshua has dealt with both extremes. There are instinct-heavy leaders who just want validation. There are also data purists who cannot move until the spreadsheet glows with statistical significance. At a $7 billion retailer, he once saw a merchandising exec demand 9,000 survey responses; just so he could slice and dice every subgroup imaginable later. That was not rigor. It was decision paralysis wearing a lab coat.

The answer is to build maturity around data use. That means investing in operators who can navigate ambiguity, reason through incomplete information, and explain caveats clearly. Data has power, but only when paired with skill. You need fluency, not dashboards. You need interpretation and above all, you need to train teams to ask better questions before they start fishing for answers.

Key takeaway: Every marketing org needs a data translation layer; real humans who understand the business problem, the structure of the data, and how to bridge the two with integrity. That way you can protect against confirmation bias, bring discipline to decision-making, and stop wasting time on reports that just echo someone's hunch. Build that capability into your operations. It is the only way to scale sound judgment.

You’re Thinking About Statistical Significance Completely Wrong

Too many marketers treat statistical significance like a ritual. Hit the 95 percent confidence threshold and it's seen as divine truth. Miss it, and the whole test gets tossed in the trash. Joshua has zero patience for that kind of checkbox math. It turns experimentation into a binary trap, where nuance gets crushed under false certainty and anything under 0.05 is labeled a failure. That mindset is lazy, expensive, and wildly limiting.

95% statistical significance does not mean your result matters. It just means your result is probably not random, assuming your test is designed well and your assumptions hold up. Even then, you can be wrong 1 out of every 20 times, which no one seems to talk about in those Monday growth meetings. Joshua’s real concern is how this thinking cuts off all the good stuff that lives in the grey zone; tests that come in at 90 percent confidence, show a consistent directional lift, and still get ignored because someone only trusts green checkmarks.

“People believe that if it doesn’t hit statistical significance, the result isn’t meaningful. That’s false. And danger...