Erika Hall on Fear & Ignorance

Erika Hall is a designer, author, and consultant. She is the Co-Founder and Director of Strategy at Mule Design Studio in San Francisco and author of the influential books Just Enough Research and Conversational Design. Her work centers on evidence-based design, organizational learning, and ethics in digital practice.

Research Questions are Not Interview Questions:

So I start all these conversations with the same question, which I’ve borrowed from a friend of mine. She helps people tell their story. I stole it because it’s such a big, beautiful question—but because it’s so big, I kind of over-explain it the way that I’m doing now.

So before I ask it, I want you to know that you are in complete control, and you can answer any way that you want to. That’s my general way I approach life. And the question is: Where do you come from?

That’s a fantastic question.

Yeah, I’ll answer that on several levels, because I think they’re all important to where I am now. And the origin point is I come from Los Angeles. And it’s a two-parter.

I come from across the street from the airport until we got eminent domain, and then the Valley. So if you’ve heard of Valley Girls—I was there. I was a child when that song was blowing up.

But those parts of being in Los Angeles, and then really being in the Valley in the ’80s—that’s a cultural context. And then the next most important origin is I got the heck out of L.A. and went back East for school, where I studied philosophy. So I come from L.A., I took a tour through New England, and I’m back in the Bay Area. So my perspective is very Californian and very question-asking. I don’t have a traditional design or research background. I come from philosophy, with a dash of studying abroad in Moscow. And all of those things—I’m finding, and the reason I’m answering this question like that—is every part of that is so wildly relevant to what I do and how I am now. Those are kind of the key ingredients to that.

So having grown up in the suburbs outside of Rochester, L.A., California—the Valley—I mean, these are mythical, mythical places. That you were there, growing up—what was it like? What can you say about growing up in that?

I mean, the really salient thing to say is: it’s well documented. Because I really felt like I was growing up in a place and time that all the movies were being made about. So it’s like, what was it like? I went to a prom in the same ballroom that Pretty in Pink had been filmed in. So it was like, I really felt like—if you want to feel like, “Oh, we’re the center of the cultural universe”—in Los Angeles at that time, that’s sort of the feeling.

And yeah, so if you watch Terminator 2, that aqueduct is right by my house. That’s sort of the fun part of it—how much was happening there then that was culturally important. Like we had KROQ, which is an amazing radio station.

So I felt like all of the best new music—I was listening to it. And then, yeah, it was really funny because I went to school back East, and to people back there, it was mythical. I came from this mythical place, and they would ask me questions about it, like, “Does everybody really talk like that?”

And I think part of it—one of the reasons I left—was I needed finishing school to get rid of my strong Valley accent. Our lawyer actually spent a lot of time in Southern California, and we had a podcast, and one of the podcast reviews was, “Their California accents are so strong.” So if I’m talking to someone who’s from the same place, or if I go back there, the accent comes back.

And the other question I got was about whether I was worried about getting shot on the freeway, because that was a thing that was happening.

And I’m like, well, yeah, I worry about being among all those cars and everything. And so, yeah, it was like that in a lot of ways. I feel that Frank Zappa—that song—is an ethnographic document, really, a linguistic situation.

But I went to the Galleria. I went to the beach. There was a section of my yearbook devoted to the large hair. People had shoulder pads. I hated Reagan. I don’t—I don’t know.

So yes, I’d say the one thing is the movie Valley Girl with Nick Cage, which I love—I love him so much—the thing that’s most wrong with that movie is that he’s supposed to be punk, and he was in no way punk. Because it was about this girl from the Valley, this affluent suburb. I went to public school, and a bunch of my friends drove BMWs. I was not from the BMW part of the Valley, but it was wild.

And people were really self-aware, you know? Because I think children and teens always know more than adults give them credit for. And we were really clear on what was going on in the world and in politics and everything—even before the internet.

So yeah. I drove once I got a car when I was 16. I drove a lot and really was like, yeah, if you watch those movies—and there was Booksmart, I think, is a recent movie—that was still the vibe in Los Angeles.

So yeah. It’s incredibly well documented, I think, just because the movie industry was there.

What—do you have a memory of what you wanted to be as a child? Like, what did you want to be when you grew up?

I wanted—well, there were a few different things. I wanted to be an architect for a while. And I had an Etch A Sketch, and I would actually draw floor plans on my Etch A Sketch.

And then a couple of things took me off track. I even applied—one of the schools I applied to was a school of architecture. So I got in, and I could have done that.

But I was all over the place. I was like, maybe I’ll do psychobiology, maybe I’ll do architecture—we’ll see what I’ll do. And then I ended up going to a liberal arts school, which was perfect.

But then I took a look at the built environment of Los Angeles, and I’m like, oh, we don’t need more architects. And then one of my teachers made us read The Fountainhead, I think in a prophylactic manner—like, you have to read this to be inoculated against these terrible ideas. And I read that, and it angered me so much that I was just like, I don’t want to be part of this.

Also, you read about the profession, and it’s super competitive and super misogynistic and all of that. But I also didn’t realize until much later that I grew up surrounded by Eames stuff, right? Because being in Los Angeles—we had Mathematica, which I think is still in Boston, which is this amazing, kinetic, sculptural, experiential exhibit of the principles of mathematics. And that was my early childhood. So if you haven’t seen it, look it up—it’s at a science museum in Boston, I think still. There were a few different instances of this. And at the Museum of Science and Industry, they had the Eames explaining math.

And I think the sort of Eames was in the air—all of that mid-century modern stuff was in the air. And that was a big part of coming from Los Angeles, too.

And so I think the fact that I ended up doing a lot of information architecture—I was like, oh, this is sort of similar. And buildings are great. And I have friends who are architects who have gone to architecture school. It’s just—I am not a patient person. That’s also why the movie business—even though I grew up in Los Angeles and love the movies, everything about that—I never wanted to be in front of the camera.

But seeing that process, I have so much admiration for filmmakers. But wow, the patience of putting something like that together is beyond me. So I’m happy about the internet.

I’m curious about—you went East to a liberal arts school. And, you know, I’m from the East, I went to a liberal arts school, so probably the question of what it’s like to be from California came to mind. But what did you make of the people in the Northeast? Who did you find at these liberal arts colleges, as somebody from the Valley?

It was a lot of—I couldn’t afford to visit. So I just kind of dropped in, like, oh, I guess I’m doing this now. I just flew out, you know, September of my freshman year, and I was like, what is going on here?

Because I didn’t understand a lot of the things people were saying to me. It was class-coded. I was like, why are you—like, I told a story, and someone asked, “Why are you so hyped on Nantucket? Do you come from a whaling family?” And like, you summer places? Does that mean that where you live sucks part of the year?

There were all these—there were all these codes and ways of being that I was like, really? Why? Why are you like that?

And I found out about private beaches, and I was just horrified. Because a private beach is illegal in California. As a person who is not even a resident, but just as a human being, you have a right to coastal access in California. And if you have a beach property, you have to let people—there’s a number of feet. I mean, this sounds like maybe a minor thing, but I think it’s a hugely important difference in how you think about the land.

California is not perfect, but it’s like, you can’t just block people off from access to the ocean. And I feel like I learned about all the private clubs and ways of excluding people. And California being a place where people just end up.

The unfortunate part is we haven’t built enough housing for all the people who end up here. But just the space and the light.

I thought people were fascinating, and a lot of the things sort of didn’t make sense to me. Like, it was fun—like seasons. I’m like, oh, seasons are cool.

But I noticed that I was friends with people from New York, from Maine, and from California, mostly. There were states where I’d meet somebody and we’d get along, and they’d be from one of those states. It was a great experience.

I had a friend who was in the dorm next to me freshman year. He was from Hawaii, and he was the only person I knew who was even more like a fish out of water than I was—just because it dropped below 60, and he was bundled up in his coat like, “Why did I do this?” And we were both there like, “Why did we decide this was a good idea when we were 16 or whatever?”

A lot of it was tough because it was just a different way of being. It was a small town instead of Los Angeles. A lot of it was really hard. I’m glad I did it. I learned a lot. I got a great education from doing it. But yeah, I was just like, huh, East Coast people.

It was really important to me, even when I was young, because I had a lot of autonomy over where I was going to college. I was the first person in my family to go to college, so it was my whole project. I thought I would stay and go to a UC—for many reasons—but then I got a scholarship, and I was like, “Okay, we’re doing it.”

It’s wild, the things that just happen in life. I’m just like, why am I here? This is wacky.

So, catch us up. Where are you now, and what’s the work that you do?

Where am I now? I’m in San Francisco. After college, I settled into the Bay Area like a tick, and I’ve been in San Francisco for a really long time. I co-founded Mule Design with my partner, Mike Monteiro, a really long time ago.

What I’m doing now—it’s consulting, really. A lot of it is practice development. And what that means is, we’ve done so much work with so many different organizations. Now that so many design teams are in-house, the best we can offer is helping the people. Because if you go to work in-house, you don’t have that kind of cross-training from an agency, and you don’t know what the job is supposed to be like. And you’re in that reporting structure.

The best we can do for both the practitioners and the organizations is bring that outside perspective. When you see the same things in ten different organizations and you’ve had to wrestle with them—things around decision-making, getting the work done, or knowing what questions to ask—it gives you useful insight.

A lot of what I do are research workshops now, because everybody sort of does design research wrong. We also do communication strategy and sometimes get more into the actual hands-on work—straightening out your information hierarchy, doing actual branding, things like that. But mostly it’s just taking all the expertise we’ve had from twenty years of design work in all these circumstances and providing that to people—providing that expertise that’s hard to get in the current situation.

When did you first discover that you could make a living doing this? Were you a designer first or a design researcher?

Neither. First, I was on the technical side. I was coding and stuff like that because that’s what I was interested in.

The other part of my origin story is, when I got my first computer a long time ago, I wanted to learn to program. And the key origin story part is, I really wanted a computer.

Where did that come from? I hung out at Radio Shack a lot—more than a lot of little girls in the Valley. I don’t know. But I asked for a computer because I wanted to learn to program.

And they got me a game system. They got me an Atari. And I was mad.

So that’s—if you want the key to my whole way of being—it’s that I wanted to learn to program, and you got me video games? Man, the Lisa Simpson energy was just strong with me. And I’m like, fine, fine, I’ll play a lot of Atari.

Here’s your lightly edited version—cleaned up for clarity, grammar, and flow, but preserving your original voice and meaning, with no added words or labels:

And I actually, at one point, did work for a company that was doing video game-based community and stuff like that. So it worked out.

I started off coding—front-end stuff like HTML, Perl, JavaScript, all that. That’s where I started, because I started working at a publishing company. It was the earlier days of the web, so there was a lot of fluidity in the role. And when you’re a liberal arts person, you’re like, I’ll do whatever. So I started there.

Then I ended up working for a consultancy, an agency, and I was like, this I like. Short attention span—I liked going in, helping people solve a problem, getting out, working on a team of people. That was all really fun. And just because I can do things, I ended up doing project management, content strategy, information architecture, and stuff like that.

Then we started an agency, and I was doing all sorts of things. I worked with a researcher—who’s still a friend—at my first agency, and that’s how I was mentored in design research. But the only reason I sort of specialize in it is because people were approaching it wrong. We kept having arguments with clients about, “Can’t we just do the design part without the research part?”

I was so tired of having the conversation that I wrote a book about it—because there was no book. I had to do the thing where you write the book you wish existed. There was nothing accessible for people, and I was like, this is bananas. People aren’t asking questions—they just want to make things. And I felt there was a lack of focus because so many people just wanted to make things.

And I’m like, well, I like asking questions, so I’ll just kind of work on that part of it.

My experience is—I’m assuming you’re talking about Just Enough Research, is that the book you’re referencing? Your writings across the board have always struck me as so welcome—and kind of alone, really. To your point, I don’t encounter a lot of people articulating the in-the-weeds principles of what’s research and what’s not, other than you. So maybe this is just a way of saying thank you so much for doing that work.

But I wonder—maybe the follow-up question is: what do you make of research today? I know I’ve had my own fixation on how these weird labels—UX, CX—feel like machine-like acronyms for what’s really a human interface. So maybe tell us: when you say “design research,” what do you actually mean? And what are the mistakes people make when they talk about users and all that?

Oh, boy.

Yeah, the reason I talk about research—and talk about design research, not UX research, not user research—is that design research is the investigations you do, or the things you learn, in order to make better decisions. To make intentional changes in the world. Because design is fundamentally about intentionally intervening in systems and making artifacts under conditions of uncertainty, right? That’s the whole design-versus-craft—or all those arguments people get into. The key is, you’re trying to do something in a new way.

You’re figuring it out as you go. And design research is the stuff you investigate or learn so you have a better chance of success and reduce your risk. That includes things like: you want to understand the people you’re designing for, because you’re fundamentally making choices on their behalf.

But it also includes: what will it take for this thing to succeed? Whether or not it’s a for-profit business—what are the conditions that will sustain it? You have to understand who else is solving this problem, because that’s a huge mistake people make—like, “Oh, we’re going to do this thing that’s already solved,” or “nobody wants it,” right?

You have to understand your organization, who you’re working with, your capabilities. You have to look at the history and say, “How has this been tried before?” And you have to know how to talk about it—the brand, all of that. All of these pieces were part of what we did at that first agency at the end of the ’90s. It was very holistic.

Then, for reasons, it got reduced—limited to, “Oh, we’re doing user research,” and not thinking about these other things. And “user experience” became the label people used instead of “design,” for reasons.

I think the biggest mistake people make is carving up the way of understanding the world. It’s like that parable about the blind men and the elephant. Organizations codify and reify that. They’re like, “Okay, we’re definitely distributing the elephant throughout our organization.” That makes no sense to me.

If you’re making decisions about bringing something new—maybe consequential, maybe not—into the world, don’t you need to understand all the parts of it? And organizations do not do that. That’s why I’ve really focused on that piece of it. Because making all the other pieces—people are really hyped on those. “I made a beautiful, tangible artifact!” Cool. It doesn’t matter if it doesn’t do the thing you want it to do, or doesn’t fit into the world, or if it’s based on false assumptions.

Because I’ve always liked asking questions—that’s just my thing. And I’m happy doing that. I’m happy helping other people do that. And that’s where I’ve ended up focusing my work.

Yeah.

How would you say it’s changed over time—from the first to second edition? It’s been a while, right? How are organizations today trying to understand the world?

Watching this happen has been fascinating.

The way it’s changed—again, looking back at what I was doing with my colleague in ’99 or whatever—we were doing all the things. The tools weren’t as available, and I don’t know that that’s necessarily a bad thing. There are a lot of platforms now that promise “insights at scale” and all that crap. But tools only help if you already have a solid practice. I think people are substituting.

So, I would say the biggest change I’ve seen in terms of practice—this might not be linear—but the biggest change is: once the concept of design research, or user research, or user experience research, percolated into organizations as “Oh, this is a thing we need to do,” they started doing it, but they don’t really want to do it.

What I mean is: because of the incentive structure organizations are working in—which is typically to maximize shareholder value, maximize investor value—when things are highly financialized, reality doesn’t matter as much. It’s all about telling a story to the market, telling a story to investors.

The thing that’s changed structurally is that the economy has gotten more financialized—in large part because the internet enabled that. It allowed the abstraction and securitization of everything. So many shenanigans are enabled by the internet, and that fed back into everything.

If everything is just a story you’re telling to investors, reality just gets in the way. Because if you’re talking about creating something for someone to use in the world—you know, like we have a really good coffee maker that we bought on the recommendation of a friend, and it seemed expensive, but he said, “Oh, this will last forever…”

Here’s your cleaned-up, lightly edited transcript. I’ve preserved your tone and wording while improving flow, punctuation, and sentence structure for easier reading. Nothing added, just refined:

And I’ve had the same coffee maker for, like, what, ten years now? It’s a really good coffee maker. And that was designed to be really high quality because they were selling it to people in exchange for money.

There are very few things now that are just sold in exchange for money. And when things aren’t sold in exchange for money, then it’s like—what are the factors in the decision? And quality is not really a factor. In fact, there’s a news story now saying that people aren’t upgrading their phones enough, and so we’re all going to tank the economy, right?

So the whole economy is based on not creating things that really work in the real world. It’s based on all these financial shenanigans. And that’s what made it tough for research.

There’s a lot of conversation about, “Oh, we just have to prove business value.” But the fundamental issue is that business value doesn’t come from making better quality things for users or customers. Business value comes from telling a story to the market.

And when the business value is based on those sorts of fictions and relationships—and getting market power and shenanigans—research has less value to the business. Often it’s really inconvenient to know things about the world that interfere with your story. So that’s part of it for a lot of these businesses that care about scaling and telling stories to investors or whatever.

There are still—though they don’t get a lot of press—organizations that do things that are real, right? People still make coffee makers. People make devices and things like that. So, if an organization makes things that are real, and the real world matters to their success, then research still matters.

Then the problem is the tools.

So many organizations have created software tools, and so much of the information about how to do good research comes from the makers of these tools. Some of the tools are fine, but—

What kind of tools are we talking about?

Survey platforms, testing platforms, analytics platforms. They put all this marketing money out there, and so if you’re just looking up “how do I learn things?” what you’ll get is: subscribe to our giant, expensive enterprise platform, and that’ll give you what you need.

That feeds into a common practice—organizations buy a tool set. We’re seeing it now with so-called AI. Like, “If I buy the tool set, it promises benefits.” And once you’ve made the investment, you make everyone use the tool. Then there’s a lot of skepticism for things that don’t have a cost associated with them—which is the stuff I advocate for.

Like, “What if you talked to people?” There’s no marketing budget behind, “What if you listen to people or just look at the world?” And that’s why I do what I do. That’s why there’s that gap.

I have a book that costs $25, and that’s fine. I have a workshop that’s not that much if you’re buying an individual ticket—or even if you’re bringing me into your company, it’s still not that much. It’s a tiny amount of money to say, “What if you just talk to people?”

Meanwhile, these software companies are making huge promises and charging huge amounts. And because of how the human mind works, people value what they pay more for.

Often, bringing it back to our consulting practice, the greatest service we provide is charging money to organizations to get them to listen to the people they already hired.

I mean, I identify with a lot of that. You brought me back—I’ve been in those conversations. I guess it’s a beautiful articulation of... I mean, I’m always interested in the argument for qualitative. What is the argument for qualitative in that system? How do you make the case to talk to people?

Well, the argument for qualitative is—you can’t. Like, this is also something I work on—this “versus” battle between qualitative and quantitative, because it makes no sense. You cannot measure what you don’t understand.

You need both. But you need qualitative work first because you have to say, “Hey, what things exist in the world?” Once you determine that—phenomena, patterns of behavior, physical objects, ways of being, concepts—then you can say how much, how often, when.

But it’s easier to develop and charge money for systems that aggregate a lot of quantitative data. So there’s all this focus there.

You could read James C. Scott’s book, Seeing Like a State—which I recommend constantly. He was writing about governments, but these organizations now are quasi-governmental. I mean, they’re larger than nation-state economies. Their decisions are more consequential.

If you think of people who are active Facebook users as citizens of Facebook—that’s larger than any country, really. And these organizations are moving money and creating individuals—the billionaires—who have amounts of power and influence beyond anything we’ve seen in the history of the world.

That’s the focus on quantitative. Also, you can make numbers say anything. Ten years ago—before it was “AI,” when it was “big data”—we had these giant “data lakes,” and the promise was, “If we have this data, we’ll make great decisions.”

I had a whole talk based on that. It’s the same thing: the surface promise is that you’ll have insight, but really, you’ll have so much data that you can pull from it to support whatever you want to do.

That’s why making the case for qualitative is tricky. Because if you have someone in a position of power who’s just looking for support for what they already want to do—that’s why qualitative gets in the way. And that’s why quantitative is so exciting.

Also, everything’s about scale, scale, scale. Which—cool—except if you’re scaling the wrong thing. I’d say scale is more often a bad thing. Up to a point, maybe it’s good, but wow—we need to unscale some things.

So the issue isn’t one or the other. The issue is: you have to understand what people are doing before you say how much.

Here’s your lightly edited transcript—same approach: cleaned up for flow and clarity, but your tone and content remain unchanged. No added words, just polishing.

And then it’s feedback—you need to—they go together, right? But again, I’ve talked to so many people who are in these versus situations, especially when quant is one team and qual is another team. That makes no sense to me whatsoever.

But it depends on the business. I talked to someone recently who’s in a sort of lead-gen kind of business, and it really is just a little machine for generating a transaction fee off something. All they’ve got to do is keep that little machine running. So they don’t really need to do qual, because it’s like a little machine. So it depends on the business.

But yeah, you really do need to understand the actual things in the world.

What do you love about the work? Like, where’s the joy in it for you?

Joy in it for me? People are so interesting. I mean, the real joy is that if you come at the world with kind of a research mindset, nothing is wasted. Right?

If you’re in a really annoying situation—and I tell people this all the time when I’m working with them—if you’re being frustrated by something, or if you’re dealing with a product that makes no sense, or something that’s good, anything you’re interacting with that’s interesting to you, and you’re like, huh… If you stop and go, “Why is it like that?” and you follow that...

Despite everything—the degradation of websites and internet tools and all of that—you can still find information really, really quickly. If you’re curious about anything, about any term, any concept, any physical thing, and you follow it back, you could, within ten minutes, find out why something’s like that. And that’s really interesting.

Because people get so focused on the future—designers and technologists and entrepreneurs focus on the future—and ignore the past. And that’s a real mistake. Because whatever you’re doing, you’re intervening in a world that exists. And it’s worth looking at what’s persisted—why are things like they are?

The fun for me is that people are really interesting. And it’s fun—like, it’s now fun—because we’ve been doing this for so long that I’ll be in a situation that used to be super itchy and uncomfortable for me, like there’s a conflict or something’s gone wrong, and a client’s upset. And I’m like, oh, I know how to deal with this.

So there’s the part where experience makes things more fun, because you’re not like, oh my god, I’m in an uncomfortable situation.

But there’s also the, like—I want to help. Fundamentally, I am a problem solver at my core. We joke about this all the time: when you recognize that you’re a consultant in your heart, and you see a problem, and you’ve got to stop yourself.

The question we talk about internally—if we’re dealing with somebody we know personally who has a problem—is: don’t offer help that wasn’t asked for.

That’s the thing. If you’re a problem solver, if you’re a consultant, it’s like, “Oh, let me help you with that.” But it’s like—no. If they’re paying you to help them, then help them. But don’t try to solve people’s problems if they didn’t ask.

So it’s satisfying when I actually help—that too.

I love the way you use the word practice. I’m curious about that. And maybe within this: what kind of practice do you recommend, or try to help teams build or develop? What are the things you see them struggle with—what are the problems you see over and over again?

That’s a good question. Because, wow—it’s like the same five problems. And this is what I love about that.

Now, when I do the workshops—the public ones where people can just go and buy an individual seat—I get people from different countries. Just last week, I had people from Northern Europe, Kenya, and all over America, all talking about the exact same problems.

The struggle for teams, one of the big ones, is how the organization they work in sees the value of research. A lot of times, people were hired to do a job nobody actually wants them to do. But they’re told, “You just have to prove your value.” And it’s like, why should somebody have to prove their value? They went through some heinous hiring process that probably took a year. They have a job—and then their job becomes justifying their job. That’s garbage.

Right? Because it’s like, wait—you hired me. I didn’t suggest my job title. You’re like, “We have this role. We hired you. We’re paying you to do a thing with a job description.” And then the organization turns around and says, “Justify why your job exists.”

And I’m just like, no. Do not participate in that. Don’t be on the defensive. Look at the organization and ask, “Why am I really here?”

Because the bad news I have for a lot of designers and researchers is: they were hired as part of a growth story. They were never hired to create the kind of business value they were told they were hired to create. They were hired to say, “Look, we have a giant research team! A robust design team!”—to ignore. Right?

Then they’re just handed instructions of things to build. And the strategy is shifting all the time, because it’s just reacting to competitors or to the market.

If design is fundamentally doing something intentional—and trying to do it well—you bring these poor practitioners and experts into an environment, and they’re like, “Is it me?”

The worst part is, I see people making themselves insane being like, “Obviously I’m doing something wrong.” And it’s like—no. The first step is looking around and asking: What is the organization actually incentivizing, and why? How much can you change what it’s incentivizing?

And if you can’t? Then it’s like—relax. Stop trying so hard to justify your job. You can’t. There’s a little serenity prayer in there: “Oh, this is just how it’s going to be.” Okay.

But if you’re in an organization where the decision-making is broken, the first things to change are collaboration and decision-making. If you don’t have a collaborative environment...

I’ve worked with organizations where I’ve talked to people in large enterprises with a whole building full of researchers. And they’re off doing their research, generating reports—and the organization does what it’s going to do anyway.

Sometimes I get asked, “How do I get stakeholders to pay attention to the research? How do I present better?” And it’s like—if they didn’t care at the start, there’s nothing you can do to make them care.

So the actual practice change—once you have an organization that aligns on goals and has a reality-based business—is getting people to actually talk to each other and resolve the territory battles. Then, you get everyone asking questions together.

The biggest practice shift is moving away from tools and away from activities to: What do we actually need to know?

That’s the big first step. It’s often internal research first: “What are we trying to accomplish?” “Why is that our goal?” “What do we already know?” That sort of level-setting around what we actually agree on.

Only then can you start to work on the research part of the practice—where you say, “Let’s all ask questions together.”

This is the part everyone skips over.

A lot of the value I bring is helping people understand what it means to ask a question—how to ask a good one, and how to know when you’re done asking it. Everything else is taken care of. There are tons of tools and 10,000 books, but everybody skips over the “What are we asking?”

They skip right to: “Let’s run a survey,” “Let’s do interviews,” whatever.

And it’s like—why, though? What do you need to know?

Then they end up with results from the research and they’re like, “We don’t know what to do with these.” All the problems show up at the end: “We think we learned something, but we don’t know,” or “It’s getting ignored.” And all that money and time gets wasted.

You have to start by agreeing on your goals and where you need more information. Then: when do you need to make a decision by?

Once you have those things—“We need to decide in two weeks,” “Here are our goals,” “Here’s what we don’t know”—everything gets straightforward. Then you can fit your research into your schedule.

Because objections about time and budget are really just people not wanting new information. So you can’t argue against time and budget objections with time and budget answers.

In preparing for our conversation, I was reminded of one of the first things I saw you write—on LinkedIn. The title was Research questions are not interview questions. And it was like a chorus of angels.

Because I’ve so often been trapped in those conversations where the expectation is: just ask people to answer my question. Like, “Let’s just ask them to solve our problem for us.”

I didn’t always feel armed with a good response. But you just talked about educating people about what a question is. So maybe—what is a good question? And how do you help people understand what can be asked and how?

It took me a long time to realize the confusion between interview questions and research questions. Again, this is something you talk about because it’s an intellectual exercise. It’s not something you buy a tool to do. So there’s less information about it out there.

It’s often associated with the more academic side. So it’s just not a thing people are ever taught. You really just have to start with: What are your questions? Get them all out there.

People are afraid of asking questions. Then, you separate out—once you see all the things you might need to know—where your risk of failure is. That’s how you get to the real research question.

There are questions you have, and there are questions that are good research questions—questions you can turn into a little project.

So if you’re with all the people who will be making decisions based on the information, you have to get all the questions out. That can be really scary, especially for people higher up in an organization, who have to project confidence. That’s often the biggest barrier to research: “I have to look like I know what I’m doing.”

But really—you have to admit ignorance in order to learn anything. If you can’t say “I don’t know,” then you can never learn.

Once you have a sense of everything you need to know, you can sort through them:

These are questions we can answer easily. Maybe it’s in analytics: how many people bought our product last year? You don’t need a research project—someone can just pull the data.

Then there’s a question like: How are recent college graduates looking for jobs?Say you’re building a service to help them. You need to know what they’re doing now, in the real world.

If your question can’t be answered with existing data, that’s a signal to do research. That’s a practical question you can turn into a project and go out and explore.

We’ve got maybe almost no time left, but I want to hear you dish on surveys. You’re very critical of them—and articulate about it. What do we need to worry about when we think about surveys?

The reason I fight surveys is that it’s a real tool. It’s a genuine research tool. It’s an advanced tool.

The problem is, they’re so easy to make. It’s so easy to create a tool that lets you run a survey, and so easy to get garbage data. And there’s nothing about running a survey that lets you know the data is garbage.

Other methods help you course correct. If you’re doing interviews, you’ll notice if you’re not talking to the right person. You’ll hear when your question is confusing. If you’re testing something, you’ll see when the prototype isn’t working.

But with a survey—you might get answers, and you have no way of knowing if the sample was skewed, the questions were bad, the results are meaningful.

Surveys can be good if it’s what you need. But the problem is that they’re too easy to do, and people skip all the prior research you need in order to write a good survey.

Survey platforms—when Twitter was still a thing, I was fighting with the SurveyMonkey account. They were like, “Just run this kind of survey! It’s easy!” And I was like, “What are you doing?” Their incentive is to get you to run lots of surveys.

That’s why I didn’t include them in the first edition of Just Enough Research. But I did in the second edition—and in the 2024 edition—because I wanted to go into how to get a representative sample, how to write good questions, and how to understand your audience so you’re writing questions they can answer

I encourage people interested in research to take every survey they’re presented with for a week. Really look at them. Think: are they going to learn anything true from this? Who’s going to respond? Why would they respond?

Surveys are just a machine for generating noise. And the worst part is when survey results get reported in the news as facts about the world. Then they generate consent. They generate narratives. They become self-fulfilling prophecies.

So I think they’re really dangerous in the wrong hands. And too many people are promoting them as an easy thing anyone can do.

Yes. That’s beautiful. And it also occurs to me—especially with platforms like SurveyMonkey—is that they completely edge out the collaborative relationship between qual and quant. They position qualitative as unnecessary, as if it has nothing to do with what you’re here to do.

Yeah. The problem with all these tools is: everybody’s looking for a reason not to talk to people. Because people are scary.

Why do you think that is?

Because people are scary. They are. You have to start from that—it’s kind of a legitimate fear.

And again, it’s one of those things where we do what we’re taught. Last weekend I was at an event where there was an amazing talk by a fire captain about how she leads firefighters responding to an emergency.

And one thing she said—because tech people love to use “putting out fires” as a metaphor—is: what they’re doing is not that.

And what she emphasized is: you follow your training. If you’re in a high-stress situation, you do what you were trained to do. She talked about how she responds to a building on fire—which is terrifying.

I mean, San Francisco catches fire all the time, and I have so much gratitude for firefighters. The key is: we were not trained to interact with other humans—and those are high-risk situations.

It’s just treated as something you should know how to do, like maybe you picked it up at home.

But when you look at problems—at a small level, like people have with their families or at work, or geopolitically—it’s because people do not have communication skills. They were called “soft skills” because the military in the ’60s and ’70s divided up skills you can measure and skills you can’t measure. There were “hard skills” and “soft skills” for totally arbitrary reasons.

But communication—interpersonal communication skills—are so important. Nobody was taught.

And often, you’re in really consequential interactions with other people that are terrifying. And often, you’re right to be terrified, because you might be talking to someone who could fire you, or get you fired, or shun you as a friend, or break up with you.

There are all these risks, but you’re never trained to have good interpersonal communication—unless you go to therapy, right? Therapy is like training for being a human. But it’s really expensive, and totally optional.

And then people who haven’t gone to therapy become managers. And that’s why organizations are awful.

Beautiful. On that note, again, I’m just really grateful that you accepted my invitation. I really love your work, and I’m so glad you’re out there writing. I appreciate you spending your time with me.

Oh, sure. That was a great conversation. Thank you so much for inviting me.

Get full access to THAT BUSINESS OF MEANING at thatbusinessofmeaning.substack.com/subscribe