Sam McNerney on Surveys & Story

Sam McNerney is a survey designer and researcher in Brooklyn. We met ages ago because, to me, he was an odd quant guy with a qualitative soul. (And, it turns out, he’d written some articles I’d referred to over and over about embodied cognition.) He’s designed surveys for Fortune 500 companies like Walmart, P&G, and Citibank, conducted over 400 studies with 100k+ respondents, published in Scientific American and TechCrunch, and teaches consumer behavior at CUNY’s BIC program.

Sam’s Writings

A Brief Guide to Embodied Cognition: Why You Are Not Your Brain

The Science of Asking What People Want

Bias the Participant: Designing Surveys That Elicit a Deeper Emotional Response

I’m not sure if you know this, but I start all of my conversations with the same question. I borrowed it from a friend of mine who helps people tell their stories. I use it because it's a big, beautiful question—but because it's so big and beautiful, I tend to over-explain it, like I’m doing now. Before I ask it, I want you to know that you're in complete control, and you can answer—or not answer—in any way you want. The question is: Where do you come from?

I was expecting something like that. It's a good, full-circle kind of question. It reminds me of a distinction I picked up early on when I started learning how to do market research—the difference between "Where are you from?" and "Where do you currently reside?" Or "Where's home?" versus "What's your address?" One is about things that can be tracked or scraped, and the other isn’t.

So, let me take a shot at answering in a way that captures the stuff that can’t really be tracked. I was essentially born and raised outside of Minneapolis. Born just north of Chicago, but I’m from Minnesota.

My mind always defaults to that when people ask a question like this. I live in Brooklyn now, as you know, so mentioning Minnesota feels even more important—as if to signal, “Hey, I’m not one of these freak New Yorkers. I’m from over there.” That’s probably the pithiest way to answer such a meaty question.

What was it like growing up in Minneapolis? What do I remember? It was a really great place to grow up. A quintessential suburban upbringing, but with the twist that Minnesota winters are long, cold, and snowy. Unlike the East Coast—especially upstate New York or Buffalo—the winters in Minnesota are sunnier, or at least they feel sunnier. I went to school in upstate New York, so I’ve kind of run the A/B test on winters.

Yeah, it only took one iteration to wrap up the experiment. There are a lot of outdoor activities: cross-country skiing, skating, ice fishing. A lot of people spend time outside—that’s the big difference. We were lucky. We had a pond in our backyard that froze over in the winter, so we spent a lot of time skating. I was outside a lot, which was just really good.

Another thing worth mentioning: I have three older brothers. They were quite a bit older than me—the oldest is about 12 or 13 years older. So, unlike a lot of my friends, I feel like I had a firmer grasp on where all of this was headed.

And by “all of this,” I mean life itself. Like, oh yeah—you go to college, then you move to a city, and you get a job. But yeah, Minnesota—I’m a huge homer. I cheer for all the sports teams. I loved it. And I miss it. You don’t move to Brooklyn for the environment or the outdoors.

Why do you move to Brooklyn?

I think—this might be romanticizing a bit—but it’s kind of like what JFK said about going to the moon: you do it because it's hard. That strikes me. I'm married and have two kids, five and two, so it's definitely difficult. The amount of space you get per dollar is terrible. And then there are just all the annoying things about living in a city. I'm fairly neurotic—I get annoyed by small noises and things like that pretty easily.

But there's an upside to all of that annoying stuff. You can't be passive or settle in or get too comfortable. You have to stay creative, stay productive. You're forced to come up with good ideas in a way you wouldn't be if you were in the suburbs, where your money goes further and life might be more comfortable.

That’s the romantic answer. The short answer is I graduated college, followed a girl—now my wife. All my friends moved to New York, so socially, it just felt like the natural thing to do.

I want to go back—when you were a kid, do you remember what you wanted to be when you grew up?

For sure. From about age four or five through twelve or thirteen, I really liked sports. I grew up playing hockey. I was always curious about the world, so while I don’t think I ever said I wanted to be a scientist, I loved learning about cool stuff—about Earth, nature, that kind of thing.

Then, around age fourteen or fifteen, I became a little more inward. I liked reading a lot more. I ended up majoring in philosophy. I loved learning about the history of science, the history of ideas. That interest gradually pushed me into the work I do now. So to me, there's a kind of fault line—obviously marked by biological changes—around twelve or thirteen. Nothing too interesting before that, but things started taking shape after.

You're in Brooklyn now—tell me a bit about the work you do. What keeps you busy these days?

To riff off the last part, when I got into philosophy, I also got really into the behavioral sciences. Not in an academic sense—I didn’t go to grad school—but I just enjoyed learning about judgment and decision-making. My timing was lucky, because the field was really starting to be popularized in a way that even people outside the space were noticing.

I’m talking about books like Predictably Irrational by Dan Ariely and others. That interest pushed me into advertising, but specifically into roles that focused on research and insights. I ended up spending five or six years at Publicis in that kind of capacity.

Something interesting happened during that time. I entered the field with a strong behavioral science worldview—this idea that people don’t have fixed preferences, that they’re prone to decision-making errors they’re not even aware of. You’re not supposed to ask people what they like, because they don’t really know. The metaphor I liked was that the mind is like a press secretary, and the brain is the Oval Office—what the press secretary says isn’t necessarily trustworthy.

But then, ironically, I taught myself how to do surveys—to add value to existing clients and help with new business. Which, according to that worldview, was kind of a sin, since surveys usually involve asking people what they like. What I found, though, was that the behavioral science perspective was a bit exaggerated. And the so-called replication crisis kind of confirmed that.

You may not be attuned to this, but over the years, a lot of the major findings in judgment and decision-making—and in other parts of cognitive psychology and moral psychology—haven’t been replicable. Researchers haven’t been able to replicate them. In hindsight, it’s sort of obvious why: the incentive was to land a big TED Talk or a book deal. So the incentive was to spin these so-called counterintuitive narratives that were interesting and surprising to readers or audience members.

But what I’ve found is that people will share the information you’re seeking through a survey—if you ask a good question. And by the way, we can get into what that means. I don’t mean a philosophically deep question, like something out of the Platonic dialogues. I just mean a clear question, without jargon, that’s easy to read. Your opening actually illustrated this nicely.

I want to return to your reaction to my opening question—and the way it resonated with something you’ve learned about survey design or behavioral psychology. Is that what you're saying?

Yeah. What I’m saying is that I discovered people do have access to their preferences. They have concrete opinions they can share, but it’s up to the surveyor to extract those responses by asking simple, clear, pithy questions.

So an example—I think I might’ve shared this with you—I was helping a client, a brand, with their post-purchase surveys. These are the surveys you get after you buy something. And one of the questions was phrased as: “What were the primary benefits you were seeking with your purchase?”

An easy edit is to just change that to: “Why’d you buy?” There are two reasons to make that change. One, it's easier for people to read, so they’re less likely to skip it and more likely to stick with the survey. Two, you’ll get better data. “Primary benefits” is already a mouthful, and typically, people just have one reason they buy something—the reason they bought it.

So when you ask a question like that, you’re expecting people to do all this mental work—as if they’re supposed to distinguish between primary and secondary reasons, retrieve them, and then match them to your list of response options. But if you ask, “Why’d you buy?”—everyone gets that.

What makes that more effective? I feel like there’s so much baked into the story you just told between those two questions. It’s really about what makes a good question.

But part of me wants to go back to the replication crisis and this idea of how well people can answer questions to begin with. There was a moment when we were all kind of in love with the idea that you shouldn’t ever talk to your customers. I can’t tell you how many stupid think pieces I had to read with headlines telling people not to ask or listen to their customers—giving tons of reasons not to engage in meaningful conversations. But you’re saying you learned something different.

And it’s funny—I’m rambling at this point—but it occurred to me, as I was prepping for this and digging through old links, that I don’t know if you and I ever talked about this: you wrote something for Scientific American on embodied cognition. I remember what you wrote. I bookmarked it and kept it. You were an early source for me on that topic, especially as I was figuring out—almost post-rationally—why I was so drawn to free association and projective techniques. Why I felt like I had found a more imaginative way to access good data from people—because it was kind of doing an end run around the press secretary. Do you know what I mean? So I thought it was funny that you had written about that.

So I guess what I’m asking is: where are we now? Where are you now in terms of how you think about what people can answer? And why is a shift into something like “Why did you buy that?” valid and more effective when we’ve all been told that surveys should have a form—that you’re supposed to ask clear, rational, formal questions?

Yeah. So the short answer is—well, actually, let me go back to the embodied cognition piece. I didn’t know you’d read that, by the way. It’s been years since I’ve thought about it.

The short answer is: when people have a concrete, accessible preference, just ask about that. There are certainly times when you shouldn’t trust what people say—and that’s when they don’t care about the topic you’re asking about. In that case, you shouldn’t trust their feedback.

As for embodied cognition—I haven’t thought about that in years—but let me try to steelman that worldview. And by the way, if anyone listening is interested, Jesse Singal has a really great book on this called The Quick Fix. He does a great job explaining what happened—the incentives, the oversimplifications, all of it. The embodied cognition stuff is really interesting. One thing I remember from an experiment—and I might get the details wrong—but the insight, I think, was that they had two groups of people evaluate the importance of something.

The manipulation was that the clipboard used to fill out the questionnaire, for one of the groups, was a lot heavier. And they found that people in that group evaluated the item—which I can’t remember—as being more important. The conclusion was that the weight of the clipboard was used to infer importance.

You see examples of this in the real world, like how the really premium American Express cards are heavier. That metaphor is very real. Like in Back to the Future, when Marty McFly would always describe things as being “heavy”—like, “Oh, this is so heavy.” And Doc Brown would say, “Is there something wrong with gravity in the future? Why do you keep saying that?” He didn’t get the metaphor.

The insights into metaphors are really interesting, too. There's a famous book called Metaphors We Live By—lots of great insights in there. But the point is, when you read about all these results from the embodied cognition literature, you get the idea that we're just not in control—that we’re strangers to ourselves. That we're being manipulated by the environment, not necessarily in a way that hurts us, but in a way that's outside our awareness. The embodiment stuff is especially interesting because it challenges the idea that you are your brain and that the brain is separate from the world.

Like I said, the full story is told in Jesse Singal's book The Quick Fix. A lot of these insights weren’t replicated—across embodied cognition, judgment and decision-making, moral psychology, and so on. So around the same time, this shift happened. You asked about this shift. What I realized is that it doesn’t make sense to say surveys are unhelpful or unreliable. That’s like saying pencils are unhelpful or unreliable, or blaming a bad novel on the pencil instead of the writer.

In other words, you have to learn how to use the tool—use it in a way that works with our nature, or the nature of the mind. That’s a little grandiose, but you get the idea. Obviously, people are going to b******t if you ask them a question about shampoo or soap. Sure, there are some people who really care about that stuff, but most people don’t. So what do you expect?

You have to focus on asking about things that are real to people. And by real, I mean things they can easily reflect on, draw from, and share—whatever is in the contents of their consciousness. A big part of enabling that is just removing jargon. Being clear and pithy.

Now, I can pause here, but another direction this goes is that a lot of the quantitative market research world almost takes pride in adding jargon. They sort of disguise that in the form of good methodological practices—being rigorous in question design—and sure, there’s value in that. But you can still ask methodologically sound questions—no double-barreled questions or whatever—without all the jargon and b******t.

There are two villains in this story: behavioral science and the quant establishment. Both, I think, miss the obvious model for doing surveys and market research, which is just to be pithy, clear, and ask questions people can actually answer.

How would you describe the best practices of the quantitative research field?

Another good thing to read on this, if people are interested, is the origin of surveys and polls—namely, what George Gallup did in the 1930s. He became famous for correctly forecasting the 1936 U.S. presidential election, much like Nate Silver in 2008 and the years after. And he did it by applying what was then a relatively new science: sampling—specifically, quota-based sampling.

He figured out that you can’t just ask newspaper or magazine subscribers who they’re going to vote for. You have to get a sample that matches the base population—the actual population of voters.

That’s how he got famous, but he spent much of his career in market research. He was actually partners with David Ogilvy. When Ogilvy came over from England, one of his first jobs was with Gallup as a researcher. Ogilvy has a few lines about this—something like, “I’m a researcher first and a copywriter second,” or “A good copywriter should be a researcher.” When I learned about that, I realized he was drawing on his own experience there.

The point is, Gallup was very, very thorough. He would test questions for months to make sure people understood what was being asked. He’d get real detailed—like trying to figure out if “Prohibited” versus “Not Allowed” made a difference. He essentially birthed what we now call the quantitative market research world.

He was meticulous in the way that Gordon Ramsay is meticulous—or David Blaine. He wasn’t just following a handbook. He was trying to optimize the consumer experience, so to speak. It really was about the person being interviewed—their experience. Yes, accuracy was a big part of it, but he’s got this great line where he says something like, “I care less about whether a question is leading or advising someone than whether it's intelligible.”

So there's a real craft to this. And I mean craft in the sense that a chef or a magician has a craft.

Over the years, with the arrival of the telephone—and then the internet—designing surveys got really cheap. Now you can go on SurveyMonkey, whip together a survey, and use a vendor to buy a panel of 200 people to answer it. So the cost of screwing things up is a lot lower.

But what happens is there's now a flood of just... crap. Questionnaires and surveys everywhere. People pay lip service to the rules Gallup and his contemporaries developed, but I don’t think they understand what those people really did—which was to become masters of their craft.

So that’s kind of how I would describe the quant world now.

And now, it’s a lot more about whether you’ve got experience using Excel or R or another data analysis platform. There’s a kind of filtering mechanism where, if you’re someone from the humanities or see yourself as a creative person, you’re less likely to use those tools. And then you’re less likely to think of yourself as a “quant.” And so, you don’t get hired into those roles. So over time, that world—the quant world—sort of filtered out people like Gallup, ironically.

And you're describing Gallup as a humanities guy—not belaboring things, but focused on the craft of questions.

Yeah. I mean, to be clear, what he did for the science of polling—I wouldn’t put that in the humanities category at all. But I don’t think that’s the reason we still know him. It is a big part of it—he was on the cover of Time magazine in the ’30s for forecasting that election. But his influence, I think, came from the extent to which he turned surveying into a craft.

In what way did he do that? I mean, his legacy is all about the polling side, which makes him sound like a quantitative genius. But what was it in particular that he brought? How did his creativity—or his qualitative instincts, his attention to intelligibility—show up?

He put a huge premium on the user experience—what we would now call user experience. And this was no joke. He would hire hundreds of people to go out into the streets of America with clipboards, find people in specific demographics, and ask them to answer questions.

It was painstaking work. And he would always pretest all his surveys. So, for the first few batches of any survey, the interviewer was trained to notice if people looked confused or seemed to struggle with a question. He would literally spend months making sure each question was clear and intelligible—his word.

Now, jumping to today—another source of inspiration for me is Don Norman’s The Design of Everyday Things. You're probably familiar with his basic thesis, which is that objects afford certain actions. A monkey bar affords being gripped and hung from. A bar on a door affords being pushed.

It’s up to the designer to make interactions with objects intuitive and easy—by using what he calls “signifiers” or “signals” that help people know what to do. That idea had a strong influence on the field of user experience—like how digital forms are designed, how people check out on Amazon, or register for something online. And obviously, in product design too—like your iPhone.

The reason it was inspiring to me was really about the extent to which it encouraged people to think about making online forms intuitive and easy. A survey is an online form. I’m not going around the country with clipboards asking people questions. So, yeah—in other words, to optimize for ease and make a survey intuitive is to optimize for better insights.

And it also just saves you time and money, because your survey will be shorter and more to the point. That’s another big piece of this—worth mentioning aside from just approaching it as a craft, not merely a hard science.

What do you love about the work? Where’s the joy in it for you?

Getting things right. When I’m designing a survey for a client, I’ll field a draft to, say, 20 people. Then I look at the results—you can kind of tell what’s working and what’s not. I’ll also just take the survey myself several times to spot what’s confusing. I’ll go through those early drafts six, seven, eight, nine times. And usually, if things go well, you land on something better. It just works.

It’s similar to what a good user experience person does. They start with an early version of a digital interface—a form or a checkout page—and realize people aren’t completing it. Drop-off rates are high. So they make a series of changes, and eventually the drop-off rates go down to zero. And they know: “Okay, we fixed this interface.” That feeling—getting it right—is rewarding.

There are other parts too. Once you have the data, there’s the analysis, creating a narrative from it—or often, in my case, helping a client support their narrative. We can definitely talk about that. But yeah, optimizing a survey for ease and intuitiveness is deeply satisfying.

Let’s talk about questions. You’ve clearly thought a lot about this. I mean, I think we met years ago—probably through LinkedIn—and I remember having coffee with you. I connected with you as, in my mind, a very qualitative person working in quantitative surveys. That’s a reductive way of saying it, but I’m a qualitative person too, and everything you’ve described points to that sensibility. As a qualitative researcher, I probably think more than the average bear about the words I use in a question. And you clearly do too. So, what is a good question?

I hope this resonates with you—but to me, a good question is a spotlight, not a floodlight.

A floodlight question is like: “What’s your favorite toothpaste?” or “What do you think of the United States’ position on tariffs and foreign alliances?” Or even, “How would you evaluate the last six months of your life?” It’s like... I don’t know! I’m just trying to get through the end of the day.

So, another example—I was thinking about this for my newsletter this week. Imagine you’re doing a survey for a cold and flu over-the-counter brand. I riffed on something similar about a year ago.

You could ask: “Have you experienced any sleep disruptions that you would attribute to a respiratory tract infection?”

Or you could just ask: “Did a cold or flu keep you up last night?”

That second one—that’s the spotlight. It points to a very specific thing: a time, a place, a situation. Last night. Cold or flu. Did it keep you up? Yes or no. The phrase “Did you experience any sleep disruptions…” is just—it’s vague, abstract, clinical. It doesn’t land. Those words are like... what even is that?

What is one trying to achieve with that kind of language? There’s something happening there, right? The first question in your example—the more jargon-heavy one. What is that jargon meant to accomplish? People use it because they love it or because they feel like they’re doing the right thing. It’s meant to do something. What does it do?

Yeah, absolutely. And it speaks to the example you gave. I mean, the formal question—and I totally appreciate this—the primary purpose, or one of the primary purposes, of that kind of language is to signal credibility to coworkers or within the organization. You're performing as a “professional” person.

And I think there’s another layer to it. I always think about when I started as a qualitative researcher—in the focus group setting, you’re called the moderator, and there’s this weird expectation of objectivity and distance. There's this “view from nowhere” that's associated with science—like you’re suddenly more scientific if you’ve extracted any kind of humanity from the process. That kind of language is intentionally neutral or “objective,” but in a way that just makes it... I mean, I guess it’s also speaking to the idea of the mask—but not the consumer’s mask.

It invites the consumer to wear one. Like, “Oh, I know what I’m supposed to do here. I’m supposed to follow this script. This is the dance we do in surveys. We pretend we’re all very, very rational all the time about everything we do. And these are the words I’m supposed to use to describe my body and my experience.” It’s so alienated from actual experience. It’s amazing.

And this is why I’ve always been intrigued by your work. I think of you as being a bit outside that world. But do you feel at home in the survey world? Do you feel welcomed by it? Or have things changed? Or am I exaggerating the extent to which quant is kind of inhuman?

I don’t think you’re exaggerating—not just because of the historical context I mentioned earlier. In the quant world, there’s really very little middle ground between standardized questions that scale and what you might call “custom” questions.

What I mean by questions that scale are the ones like, “Would you recommend our product or service to a friend or family member?” “Do you recognize this brand?” “How satisfied are you?” You can apply those anywhere. The problem is, they aren’t intuitive. But the upside is that you don’t have to reinvent the wheel every time.

A custom question—by which I mean what I’ve been talking about: something intuitive, specific, pithy, tailored to what the client actually wants to learn—that takes a lot more work. So I think it’s always going to be this way. There’s a trade-off between those two approaches.

I feel most at home, I think, in advertising. That’s kind of where I came up. When I was at Publicis, I spent almost all my time with strategists and planners, and I learned a lot from them. I was always trying to impress them—and the copywriters.

Ideally, I’d deliver research results, and they’d take one of my survey questions and use it as a headline in their deck. Or a copywriter would say, “Oh, we don’t actually need to do any work—that question is the copy.” And they wouldn’t even need to see the results—they’d just get it.

So I think I feel closest to that world—creatives and strategists within agencies. That’s mostly who I work with these days. But yeah, I always feel a little bit like a fish out of water.

Yeah. And I’m exaggerating a bit for effect, just so everybody knows I’m not totally ignorant to the value of quant. I do get excited about quant. I hype it up over qual sometimes—but I know they’re both valuable. That’s my diplomatic caveat.

Are there other practices that bother you, or that you’ve made a point to address in your work? Like, how do you talk about what you do? You've told me before, but when someone asks, “Sam, what do you do?”—what do you say?

A lot of my clients are agencies—mid-sized ones. The people I work with are usually heads of planning, heads of strategy, CMOs, CSOs. One of the things that really annoys them, in terms of quant solutions—and it annoys me too—is when they hire a vendor, get the results back, and it’s in 30 slides that are totally unusable.

So they have to redesign all the slides and force-fit them into their deck. It takes a ton of time.

They also have access to all these social listening tools, subscriptions, big databases—which can be helpful—but they don’t really help you understand the shopper, or what that experience is actually like.

That’s the world I feel most comfortable in. And that’s how I describe what I do: I provide quant solutions to mid-sized agencies. But the difference is, I try to correct for all those annoying things they’ve come to expect from quant vendors and subscriptions.

And not just remove the pain points—but make the work more fun. More interesting. As I said before, when you really get a survey right, it feels good. Because it’s not just that you’ve optimized it for the respondent—you’ve also gotten the strategy and thinking right.I think it signals thoroughness—internally, within an organization, or to another employee. It’s certainly not optimized for the person answering the question. But I don’t think you’d ever get fired for asking something like, “Did you experience any sleep disruptions that you would attribute to a respiratory tract infection?”

Right. It’s like—look how thorough I am. Look at all these long words.

Exactly. And yeah, there are some pockets where this isn’t acceptable, but for the most part, it’s totally fine. I mean, just go look at SurveyMonkey’s templates, or Typeform, or any of those platforms. What will you find? You’ll find language that you won’t hear anywhere else except in a corporate survey. I mean, the question “Would you recommend this to a friend or family member?”—it’s a fine question. It has value.

But it’s also a bit of a floodlight question. People don’t often recommend things to friends and family. One way I’ve tried to improve that is by asking: “If a friend or family member mentioned this in conversation, would you recommend our service or product?”

And another thing that could be improved is the response options. Usually, it’s just 0 through 10. I prefer three options:

* Yes, I’d recommend.

* No, I wouldn’t recommend.

* Yes, I’d recommend, but I’d mention a few things I don’t like or that are annoying.

And the times I’ve used that, most people select the third one. There’s something psychological going on there. It gives people space to say, “Hey, I like you... but.” It’s like arguing with a friend or a spouse: “Okay, I like you—but...”

So again, that model I rely on: is this a flashlight? Meaning—are you asking about something specific that people can easily grasp? “Oh yeah, I know what you’re asking.” If you can do that, their answer comes easily. They don’t have to think. They just answer.

How do you mean?

What I mean is: I’m not trying to get at hidden motives or deep-seated unconscious sentiments. I’m not interested in getting psychological in that way. That strikes me as really hard—and not that valuable. What I want to know is what’s on the surface—but still concealed. Another model for this—and I was just watching Squid Game—there’s a scene that kind of hit me as a good metaphor.

There’s a young woman working at a carnival, wearing this giant cartoon costume—like a huge Mickey Mouse-style character. Then you follow her into the break room, and she takes off the headpiece, and she’s drenched in sweat. Earlier, there had been a scene with all these happy kids. So as a parent, I thought—great scene. Like, I have to put on a smile every day, and underneath, I’m drenched.

But as a researcher, I saw something else—it captured the fundamental difference between the public and private self. And by private, I don’t mean your id or subconscious. I just mean things you carry with you all the time—but don’t show. So to ask the person under the mask, so to speak—to get an answer from the actual person—you simply need to speak to that person. And they will respond if they feel spoken to.

Again, take the question: “Did you experience any sleep disruptions that you would attribute to a respiratory tract infection?” vs. “Did a cold or flu keep you up last night?” The latter speaks more directly to the person under the mask. And it’s not psychologically deep or philosophical in a Platonic dialogue sense. It’s just a simple, clear question that speaks to them.

And from there, it’s easier to ask a good follow-up. Like: “What did you do about it?”

Because in this case, if you’re the brand, you don’t want to just treat the symptom—you want to understand what the person is going through. So yeah, the mask—that scene was a decent model. I know the image of the mask has been used in literature and film a million times, so I’m not pretending it’s original. But it is useful.

Yeah, absolutely. And it speaks to the example you gave. I mean, the formal question—and I totally appreciate this—the primary purpose, or one of the primary purposes, of that kind of language is to signal credibility to coworkers or within the organization. You're performing as a “professional” person.

And I think there’s another layer to it. I always think about when I started as a qualitative researcher—in the focus group setting, you’re called the moderator, and there’s this weird expectation of objectivity and distance. There's this “view from nowhere” that's associated with science—like you’re suddenly more scientific if you’ve extracted any kind of humanity from the process. That kind of language is intentionally neutral or “objective,” but in a way that just makes it... I mean, I guess it’s also speaking to the idea of the mask—but not the consumer’s mask.

It invites the consumer to wear one. Like, “Oh, I know what I’m supposed to do here. I’m supposed to follow this script. This is the dance we do in surveys. We pretend we’re all very, very rational all the time about everything we do. And these are the words I’m supposed to use to describe my body and my experience.” It’s so alienated from actual experience. It’s amazing.

And this is why I’ve always been intrigued by your work. I think of you as being a bit outside that world. But do you feel at home in the survey world? Do you feel welcomed by it? Or have things changed? Or am I exaggerating the extent to which quant is kind of inhuman?

I don’t think you’re exaggerating—not just because of the historical context I mentioned earlier. In the quant world, there’s really very little middle ground between standardized questions that scale and what you might call “custom” questions.

What I mean by questions that scale are the ones like, “Would you recommend our product or service to a friend or family member?” “Do you recognize this brand?” “How satisfied are you?” You can apply those anywhere. The problem is, they aren’t intuitive. But the upside is that you don’t have to reinvent the wheel every time.

A custom question—by which I mean what I’ve been talking about: something intuitive, specific, pithy, tailored to what the client actually wants to learn—that takes a lot more work. So I think it’s always going to be this way. There’s a trade-off between those two approaches.

I feel most at home, I think, in advertising. That’s kind of where I came up. When I was at Publicis, I spent almost all my time with strategists and planners, and I learned a lot from them. I was always trying to impress them—and the copywriters.

Ideally, I’d deliver research results, and they’d take one of my survey questions and use it as a headline in their deck. Or a copywriter would say, “Oh, we don’t actually need to do any work—that question is the copy.” And they wouldn’t even need to see the results—they’d just get it.

So I think I feel closest to that world—creatives and strategists within agencies. That’s mostly who I work with these days. But yeah, I always feel a little bit like a fish out of water.

Yeah. And I’m exaggerating a bit for effect, just so everybody knows I’m not totally ignorant to the value of quant. I do get excited about quant. I hype it up over qual sometimes—but I know they’re both valuable. That’s my diplomatic caveat. Are there other practices that bother you, or that you’ve made a point to address in your work? Like, how do you talk about what you do? You've told me before, but when someone asks, “Sam, what do you do?”—what do you say?

A lot of my clients are agencies—mid-sized ones. The people I work with are usually heads of planning, heads of strategy, CMOs, CSOs. One of the things that really annoys them, in terms of quant solutions—and it annoys me too—is when they hire a vendor, get the results back, and it’s in 30 slides that are totally unusable. So they have to redesign all the slides and force-fit them into their deck. It takes a ton of time.

They also have access to all these social listening tools, subscriptions, big databases—which can be helpful—but they don’t really help you understand the shopper, or what that experience is actually like.

That’s the world I feel most comfortable in. And that’s how I describe what I do: I provide quant solutions to mid-sized agencies. But the difference is, I try to correct for all those annoying things they’ve come to expect from quant vendors and subscriptions. And not just remove the pain points—but make the work more fun. More interesting. As I said before, when you really get a survey right, it feels good. Because it’s not just that you’ve optimized it for the respondent—you’ve also gotten the strategy and thinking right.

So yeah, I probably could’ve answered that in a more to-the-point, sales-letter-y kind of way.

Well, let’s try it again. I’m curious—because I feel like we’ve had this conversation before—how do you describe it? What’s the version?

Yeah. The short version is, I’d say: I provide quant solutions to agencies. A lot of these agencies—another pain point or annoying thing for them—is that they need, and often benefit from, quant solutions. But they don’t want to hire a full-time employee. And they definitely don’t need a whole department of insights people.

So it’s a good fit. A good market-service fit. I don’t need to be hired full-time. The projects themselves are usually short. If I’m helping with a new business pitch, that might be just a few days. If it’s part of a larger initiative, it might last a month or two, maybe more. But yeah—I’d just say I provide quant solutions to agencies and brands, mostly agencies.

What I was fishing for, I think, is the survey design part of it. You bring a level of creativity—and you’ve done work for me—creativity to the design of the survey experience that transforms it. It still gives a quantitative output, sure, but it’s a very different kind of survey.

Yeah. So when I’m establishing relationships with these people, I don’t talk about any of this—the details or the theory—like we’re getting into here. I like this stuff, but most people don’t care. And they shouldn’t. They’re just trying to add value for their clients or win new business.

So the thing I talk about is what I just mentioned: I can provide quant solutions that aren’t annoying. You don’t have to deal with templated surveys that aren’t insightful, or expensive subscriptions. You don’t need to hire me full time. I won’t be on your payroll. You’ll get the insights you need.

And then—if people are interested (and some of them really are—there are a lot of interesting people out there)—then we can have a conversation about survey design, strategy, consumers, shoppers, all of it. But Lord, I would never lead with theory or my personal philosophy of survey design when I’m just starting a relationship. My assumption is always that they wouldn’t care.

That’s probably right.

Awesome.

Well, Sam, thank you so much. I appreciate you sharing your time with me.

Yeah—great questions. And again, the way you opened this was great. Short questions that are easy for people to grab onto are almost always better than long ones that are hard to hold. And off we went. So yeah—thanks for the good questions.

Get full access to THAT BUSINESS OF MEANING at thatbusinessofmeaning.substack.com/subscribe