How Grubhub Utilizes A Culture of Experimentation to Maintain Its Position as a Market Leader

If you think back to just a few years ago, when someone asked you to name a company that delivered food, you’d probably only be able to name a few pizza joints or the local Chinese food place. But today, the world has shifted and online food delivery is a booming business. Last year alone, Grubhub sold $6 billion worth of food, and the company delivers more than 500,000 meals per day. So how did Grubhub enable this massive shift to digital meal purchasing? On this episode of Up Next in Commerce, we welcomed Alex Weinstein, the SVP of Growth at Grubhub, and he explained to us exactly how the company has been able to become a market mover. From the initial education process to then focusing on customer retention, Alex and his team have been deep in the weeds of it all, and they have built a culture of experimentation, data analytics and a focus on ROI to stay ahead of the curve. Alex explains it all here. 

 

3 Takeaways:

• Measurement and incrementality are important. You have to understand whether or not where you’re putting your dollars is making a difference, and sometimes the answer will surprise you
• True experimentation is necessary to create new methods of measurement, marketing strategies and growth opportunities. So the question you have to ask as a leader is how can you create incentives to allow people to take risks and learn?
• The time is now to learn about the newly-online customers that have trickled into your business due to COVID-19. In understanding their needs, you will be able to ensure retention and set yourself up for the new reality we live in

For an in-depth look at this episode, check out the full transcript below. Quotes have been edited for clarity and length.

---

Up Next in Commerce is brought to you by Salesforce Commerce Cloud. Respond quickly to changing customer needs with flexible Ecommerce connected to marketing, sales, and service. Deliver intelligent commerce experiences your customers can trust, across every channel. Together, we’re ready for what’s next in commerce. Learn more at salesforce.com/commerce

---

Transcript:

Stephanie:

Welcome to Up Next in Commerce. I'm your host, Stephanie Postles, co-founder of mission.org. Today, my stomach is rumbling, because we're talking all things Grubhub. Alex, welcome.

Alex:

Thank you for having me.

Stephanie:

Yeah, thanks so much for coming on the show. I just pulled up the app earlier to be like, "What should I have for lunch today?" Because it's 12:00, and it's time to order something.

Alex:

What did you end up ordering?

Stephanie:

I'm looking at pad Thai right now, we have a really good Thai place down the street. That's usually my go-to, but I started to get influenced by sushi, so if you have any advice, let me know.

Alex:

I don't know the restaurants in the area, but look for those that are well-rated, and look for deals. We have a ton of deals going on right now.

Stephanie:

Ooh, nice, that's perfect. You are the SVP of Growth at Grubhub, correct?

Alex:

That's right.

Stephanie:

I'd love to hear a little bit about your role there, and what brought you to Grubhub.

Alex:

Sure, sure, thank you. I've been at Grubhub for a little bit over three years. My responsibility is for the consumer business. That is, how do we get more new customers to try us out for the first time, and how do we get existing ones to order with us a little more often? And hopefully they'll return.

Alex:

This spans all aspects of marketing. We do a whole bunch of stuff in-house. I'd love to explore that a little bit later. But it also involves a lot of work cross-functionally, across the product. When I say product, I don't just mean our apps, but the totality of the experience that the customer has, from our apps to the delivery, to customer care, if that's ever necessary.

Stephanie:

Very cool. Previously, were you at, I think I saw Microsoft and eBay, or what did your past life before Grubhub look like?

Alex:

That's right, that's right. I actually am a very strange Head of Marketing. I'm a software engineer by training.

Stephanie:

Oh, interesting.

Alex:

I've written a bunch of code. I switched over to product management, and then darkness had me, and I somehow ended up in marketing. I indeed was at eBay before this, also for around about three years. Similar role, maybe a slightly more narrow role, focused on customer retention, marketing technologies.

Stephanie:

Very cool. I'm sure that was great help working at a marketplace, albeit not maybe a three-sided one, but still maybe a really helpful to transition to Grubhub with as your background?

Alex:

It very much was. I have to admit, I thought I knew marketplaces after eBay, then when I started Grubhub, I discovered so much complexity. Our business, exactly as you said, is a three side marketplace. Restaurants, food delivery drivers, and consumers. It is a hyper local business. People who live in Palo Alto whole heartedly don't care how many restaurants we have in San Jose, and how good our delivery network is in San Francisco, right?

Alex:

It has to be block by block, and we have to make sure that we have good restaurant selection there, good demand, and good supply of drivers. Otherwise, if the three sides aren't in alignment, bad things happen.

Stephanie:

Yeah, that seems like it would be really tricky to keep all that balanced. How have you found success keeping everything balanced? Like you said, it's so hyper local, I'm thinking there could be a driver over in Sunnyvale, and they're definitely not going to go to my local Thai place to pick up the order that I'm looking at.

Alex:

Yeah, this is where a lot of fun in this business comes from, and a lot of complexity in this business comes from. We have to be really good at predicting things, and predicting demand. And we have to be really good at engaging all sides of our marketplace so that drivers actually want to be online at the time when we want them to be online.

Alex:

Consumers end up placing additional orders if perhaps we have a little bit too much supply. Restaurateurs want to create deals. Basically, being able to influence three sides of the marketplace in a automated, personalized, hyper local way, is really the only way we can survive, right? This, to me, is super joyful, and super complicated, and where a lot of learning, personally, for me, has come from.

Stephanie:

Yeah, I'm sure every day it's adjusting a little bit more, and you keep have to kind of changing things up and experimenting a bit. How do I think about where Grubhub is at right now? To me, it seems like it's the market leader. How many meals are being delivered? How much is that in dollar-wise of food that's being sold? How do I think about that?

Alex:

We're a public company, all of those numbers are public. Quick summary for you. We deliver more than half a million meals a day. Last year, we delivered more than six billion dollars worth of food. Of course, with the arrival of the pandemic, the demand for food delivery has also increased. The expectation of all of our constituents, and of our community, all of us, have risen tremendously. Because, from something that restaurateurs really on for a portion of their revenue, they now rely on delivery as the majority of it.

Alex:

For consumers, where they would perhaps order delivery occasionally, now is the only way for them to order restaurant food. A lot of expectations on us have increased throughout these past couple of months, even though we already started from being quite a large company with high expectations.

Stephanie:

Yeah, have you had to adjust quickly with everything going on with COVID-19? What have you seen, other than increasing orders, and how have you had to pivot to meet the customers and meet the drivers in where they're at today?

Alex:

Yeah, absolutely. Well, most definitely, yes. First and foremost, we began by focusing on safety of all the participants of our marketplace, right? This began with our work on personal protective equipment for our drivers. We distributed hundreds of thousands of PPE sets for free for our drivers. We invested a bunch of work into enabling contactless delivery within our apps. Which, of course, is something that makes the entirety of the marketplace safer.

Alex:

We basically have to take our product roadmap, and, in many ways, revisit it fully, and focus on things our community demanded of us in that moment. Similarly, we had to do something like that with marketing, as well, because we had a certain strategy. You of course know that a lot of our effort is in making sure that consumers can get the best value on Grubhub. If you spend money on food delivery, your dollars will go the furthest on Grubhub. This really is our brand positioning.

Alex:

When COVID came, we had to take a pause, because this rewards positioning, or this value positioning, really had to take a step back, because consumer's interest... Sure, they were looking for deals, but they were looking to be safe, first and foremost. Secondly, they were looking to support their community. So we had to reposition a lot of our marketing work to make it so.

Stephanie:

Yeah, that makes sense. I'm thinking that could be a trend that stays around, even after everything's over, keeping that contactless delivery at least as an option, and thinking about how to actually prove you have the safety measures implemented, and you're tracking that every month. Are you all thinking about how to scale that and keep that for the long term, or is it more just a short term play until the pandemic's over?

Alex:

Couple thoughts for you. One is, I don't think that we're going to be looking at a pandemic being over and everything coming back to normal. I think we need to get used to the new normal, at least until the vaccine is here. Which means that people's lifestyles, their habits, will be fully adjusted by then.

Alex:

As such, it's not like we were developing a set of patches for three months, and then after that, we just turned those patches off. But also, there's meaningful, positives coming from this change, right? Like any crisis, it is both a danger and an opportunity. What we've discovered is this contactless delivery, for example, besides making everyone safe, it is actually making our network a tiny bit more efficient. The delivery driver does not need to engage with the consumer in-person. They can just drop it off, take a photo, and keep going, and keep working. Which shaves off a small amount, but in the grand scheme of more than half a million deliveries a day, this starts adding up. It helps our drivers earn more, and it helps our overall network be more efficient, which means food comes to consumers faster.

Stephanie:

Yep, yeah, that's definitely a good change. There's a lot of food delivery players in the market right now. How do you create an experience that's completely unique to Grubhub? Where people, they're like, "That's where I want to order through."

Alex:

All of this, in our minds, has to do with differentiation. And you're exactly right, maybe two or three years ago, where consumers didn't really know much about the food delivery category. A lot of what we had to do was to educate them about our existence, which is why a lot of our marketing, a lot of our product, was geared towards a first-time experience of someone who's never gotten anything delivered other than a pizza. Because really, that was the state of the world, right? You would ask an average consumer on the street, "Name a couple companies that deliver food," and they would name pizza brands.

Stephanie:

That would've been me a couple years ago, too.

Alex:

Totally.

Stephanie:

I'd be like, "Domino's."

Alex:

Yeah, yeah, absolutely. Maybe Chinese food, if you've ever tried it. An average consumer didn't know that there's hundreds of restaurants that deliver to them, and that they can find them on Grubhub. So that was the focus of our messaging.

Alex:

Three months ago, even before COVID, if you asked an average consumer to name food delivery brands, they would name us, and maybe a handful of our competitors. In that environment, I'm prompted, right? This is unaided awareness. Not, "Have you ever heard of Grubhub?" But, "Name a food delivery brand."

Alex:

Our work switched from creating awareness to driving consideration. Helping consumers understand, what is it that they get if they buy from us versus perhaps one of our competitors? Last year, a lot of our focus has been on stating this extremely clearly and delivering on that experience quite precisely. As I mentioned a little bit earlier, it is all about value for us.

Alex:

Now that we're entering a bit of a new normal with COVID-19, we're beginning to come back to some of this foundational brand positioning. Talking about rewards and value. We have a TV spot that's actually launching today and tomorrow on national TV. We're one of the biggest spenders on TV in both the category.

Stephanie:

Oh, interesting.

Alex:

Generally we're one of top 200 brands advertising on U.S. television that talks about rewards and value. You might be scratching your head and wondering, "Why in the hell is a digital first brand spending so much money on TV?"

Stephanie:

Yes, I was wondering. Tell me.

Alex:

It actually is kind of counterintuitive. We, maybe about three years back, we started scratching our heads and thinking, "Okay, if an average consumer doesn't really know what food delivery options are out there, how do we create that awareness? And how do we do that in a way that can confidently map the efficacy of our spend?" Because creation of awareness, let's face it, is the most expensive thing a company can do.

Stephanie:

Yep. Everyone wants it, but then actually implementing it, tracking it, and seeing how it did, seems a little tricky.

Alex:

It is so very tricky. Most mechanisms for doing this are actually kind of arcane, right? You do media consumption patterns, which, frankly is a large-scale survey that perhaps an agency would run and say, "Okay, New Yorkers, they absolutely do not watch any TV. They spend a bunch of time in the subway, true. And then they're all very much on digital."

Alex:

So, a brand that's trying to advertise in New York then would say, "Okay, television in New York, totally worthless. And our consumers are probably just like the average consumer in New York." That's kind of how the line of thinking typically goes. We, despite having a general applicability product, right? Everybody wants food delivery, right? Everybody from 18 to my mom, most definitely could benefit from food delivery.

Alex:

And yet, what we discover, is that the media consumption patterns of an average New Yorker are not the average media consumption patterns of our consumer. Moreover, what we discovered three years back was even though our intuition was that someone who orders food delivery online is most likely an early adopter of technology, and most likely a cord cutter, right? I mean, if you're about to order food online, you of course are ordering your socks from Amazon. You of course are watching shows on Hulu Plus without any commercials, as opposed to on cable TV, right?

Stephanie:

Yeah.

Alex:

Of course that intuitively made sense, which is why we've been spending a lot of money through digital video channels. That intuitively made sense. We stumbled upon a set of techniques that allowed us to, with confidence, compare the efficacy of our awareness spent between digital video and the digital awareness darlings of Hulu and YouTube and Facebook for some of the dimensions, here. What we've discovered is that the bull drought of digital first is actually not as efficient, not at all as efficient per dollar spent, comparing to the-

Stephanie:

Oh, interesting.

Alex:

... boring, stodgy, nobody watches it, cable television.

Stephanie:

Is it because of the audience that's there, where the digital, like you were talking about, advertising to them, they may already know about you and it's an easier conversion, whereas the people who are keeping the TV running in the background all day, maybe actually need the ad right then and there where it can put a little inception on them and they can hear about it a couple times while they have the news on?

Alex:

Yeah, I think that's one of the reasons. Other reasons are that, just on a per impression basis, your digital video is dramatically more expensive. Even though I'm a nerd of machine learning, and I love personalization, I don't believe that personalization can cover a five X price difference. It can make something 50% better, but not five steps better.

Stephanie:

So how do you think about creating that culture of experimentation like you're talking about, where most companies right now are probably not focusing on TV campaigns? How do you think about putting a budget behind that and actually empowering a team to do that, where when I think about teams who are running with marketing budgets, or just budgets in general, it's very scary to not show a great ROI, because you either aren't going to get that budget again. It's a use it or lose it type of culture, it seems like every company operates that way. Maybe Grubhub doesn't, but how do you think about creating good incentives and a culture of experimentation to come up with some of those projects?

Alex:

I think a culture where you ask for confidence in measurement for your spend is a good culture. Where you ask for feedback loops is a helpful culture. Now, you can take this too far, and you can start trying to map everything to revenue or [inaudible 00:16:56], and that doesn't particularly help with upper funnel marketing campaigns. But, the other extreme isn't particularly better. I see a lot of marketing organizations end up in that spot, where we say, "We demand perfect measurement," from what they call performance marketing.

Alex:

And the brand marketing side, the one where vast majority of dollars actually have to be spent to create awareness, is not working to the same level of rigor, and the same level of intellectual honesty with measurement. To your question about how to actually create those frameworks for the team, a couple things come to mind.

Alex:

The first one is, trying to pursue incentive alignment. If people on your team genuinely believe that learning and optimality of investment for the entire team is how they get promoted, is what the company actually values, they will pursue exactly that. Let me give you-

Stephanie:

Let me hear an example.

Alex:

Yeah, let me give you a counter example. A counter example is what happens if you hire an agency to manage your Facebook spend. Have you ever heard an agency that managers Facebook spend come back to you and say, "Your Facebook spend is terribly inefficient. You should spend less on Facebook."

Stephanie:

Definitely never.

Alex:

Right? That's what their incentives are, they get a portion of your Facebook spend. The same exact thing happens for your TV agency. The same exact thing happens for someone who's managing your Google spend, right? If you have a bunch of outsourced agencies, each of which is responsible for one of your channels, their survival, their ability to feed their children, depends on you being able to spend more money on the channel that they're managing for you.

Alex:

Of course, they don't have an incentive to try to tell you, "Hey, take money from Google and put it into Facebook." They will personally suffer. A setup like this creates a true misalignment of incentives. Let me contrast that with, let's say, an in-source structure, or perhaps a structure where you have a larger performance agency that is able to reallocate dollars between Google and Facebook without personalty suffering.

Alex:

In a structure where you in-source, which is how we operate, you're able to create a shared destiny, and you're able to say, "Hey, person running Facebook. Your incentives are all about learning." So if you have a current level of performance, which is a certain level of incremental CAC, and a certain level of incremental LTV. Your goal is to improve that by this percentage over the course of next quarter.

Alex:

Find some way to do so through whatever experiments that you're able to run. One of the potential outcomes is an improvement in efficiency by reduction in spend. They're able to raise their hand and say, "Hey, I actually want to spend your dollars. Take away some of my budget, and reallocate it over to TV, because they can spend it better. I hear they have a way to spend at a lower incremental CAC than I can."

Stephanie:

Have you seen that in your culture so far, of people actually being like, "Hey, you can have this budget, move it over here"? It seems like a lot of times, people are personally tied to their budgets, and whoever has the bigger budget is the more powerful one, and I haven't often, at least in my previous days at other companies, I haven't seen people say, "Hey, you can have this budget and move it here."

Alex:

You are exactly right. A lot of our, I guess, legacy from many of our previous jobs, associates the size of the budget with the influence in the organization, most definitely. This is where the job of a leader really is to create the right incentives and to catch people doing something right.

Alex:

If you hire somebody off of a company that had that culture, of course, their initial inclination will not be to raise their hand and say, "Hey, my area isn't working so hot." You need to indoctrinate them, if that makes any sense, into a world where it's okay to raise their hand and do it. The way you do it is by upholding folks who do this, and pointing at them and saying, "This person is doing it right," and celebrating their successes. And celebrating their experiments, where, perhaps, they didn't see the immediate success, but they learned something.

Alex:

So, as a leader, I think you have a lot of power to create these incentives. As such, structure what your team actually holds as valuable versus not. If you point to enough examples like this, you'll actually end up transforming the culture, even for someone who comes in from an organization that wasn't like that.

Stephanie:

Yeah, it seems like it would also allow someone to wear multiple hats, and kind of become a polymath when it's like, "I don't just focus on Facebook ads, or I don't just focus on this kind of marketing." They get to experiment with a bunch of different areas. Have you seen that happen in your organization?

Alex:

Oh, most definitely. My paid social folks, just like everyone's, they were super focused on Facebook. What we discovered is them raising their hands and being very creative, and being some of the first folks who ever tried TikTok, for example. This was a little while back now, but we were one of the first handful of brands to invest a lot of money into TikTok, and do large scale experimentation with them. What we've discovered is if you're one of the first ones, there's very meaningful... Effectively, arbitrages to be had, where you're able to not only get a great deal, but shape the product to your liking. As such, get a temporary advantage over the rest of the market.

Stephanie:

That's fun. How did you think about creating your first campaign on TikTok? When your team presented this idea to you, were you like, "Yeah, let's do it," or were you a little hesitant? What was the first campaign you had go out there, versus what does that look like today? Are you still utilizing it?

Alex:

Oh my God, this is quite a story, to be honest with you. The team came to me and said, "So, we're thinking about doing TikTok." My reaction at the time was, "TikWhat?" They explained this to me and I read up a little bit about it. My immediate reaction is, "Okay, you are attempting to sell a luxury product." Let's face it, ordering delivery, you're still buying food from restaurants. It is a luxury product in many of the cases, right? To, "You're trying to sell that to people who have no disposable income of their own. The average customer of TikTok at the time just could not have their own credit card."

Stephanie:

Yeah, they have allowances, maybe.

Alex:

Right? Exactly. "Why in the world could this possibly work, you guys? Our average consumer is fairly affluent, and you're now trying to go into a different demo. How is that even remotely possible?" But, luckily, at that point, I had already observed that my team knows better than me, and that they have much, much better ideas than I do. Essentially, we just did a test. We did a small test, and we experimented in earnest. Surprise, surprise, they came back and they showed me the numbers, and they were meaningfully better than Facebook at the time.

Stephanie:

Wow.

Alex:

We ended up investing more. That was genuine, true learning. Not just for the organization, but frankly, for me. There's multiple possible explanations for why it ended up being so efficient, and I can go into some of them, but the thing that matters to me most is that the crew felt inspired to pursue something new. They felt passionate enough about it to structure a test when there was no framework, really, out there. And they were unafraid enough to basically tell me that I'm wrong, and that my intuition is off.

Alex:

That made me feel like the culture is actually right. The culture is exactly what I want it to be. The opposite of that, where you're going with the highest paid person's opinion, if that makes sense.

Stephanie:

Doesn't work.

Alex:

It doesn't work, because all of our intuitions, no matter how successful we've been previously, we are sometimes wrong. Why hire smart people if you don't trust them to try things?

Stephanie:

I think there's a good mix between trust your gut, but also don't trust it, because you could be wrong. Yeah, go with other people's ideas, as well. How do you think about those efficiencies that you're mentioning when you're exploring new channels like TikTok?

Alex:

Sure. To me, it's indeed about being open-minded and experimenting with new types of media, and being unafraid to try things that aren't immediately, obviously, going to work. A similar type of experiment happened with Snapchat a little bit earlier, where I also was convinced that this can't possibly work for the same reason. Luckily, I, again, was wrong.

Alex:

I guess a pattern of learning is what inspired me to basically create this incentive structure for the team, where they're unafraid to raise their hand and say, "Hey, the way we've been doing this before is really off." If you want, I'll tell you a story of a channel that's not really a channel that I guess formed my opinion on that topic.

Stephanie:

Yeah, let's hear it.

Alex:

This is a story of a couple marketers that were attempting to turn a specific city around.

Alex:

As we talked a little bit earlier, we can be doing super well in one city, and not well at all in another city, or in a corner of a city. A lot of what we do has to do with how do we turn a specific city or neighborhood around? This couple folks, their job was to turn a specific city around, and I was expecting them to come to me and say, "Hey, I'm going to take the budget that you've given me, and I'm going to buy some Google ads, and I'm going to buy some billboards, and maybe I'm going to buy some Facebook ads."

Alex:

What they did instead, these were two marketers. What they did instead was actually really curious. They experienced the product for themselves. They placed a couple of food delivery orders, and they came to me and they said, "Hey, I don't want to buy any ads," they said. "Instead, whenever I was placing the order for food, there really weren't enough food photos. I was ordering from restaurants that I hadn't ordered from before, and I don't really know if their pad thai looks good. I don't really know if their sushi is something that I want to try."

Alex:

They were in your position. They said, "Screw it, I'm not going to buy any ads. I'm instead going to hire some photographers to come into those restaurants and take the photos. Then after that, I'm going to measure the incremental impact of the added photography, and see if the efficacy of that is actually comparable or high enough, comparing to the efficacy of ad spend." Effectively saying, "I'm going to open a brand new marketing channel, and that marketing channel is going to be photos."

Stephanie:

Photography.

Alex:

I'm like, "Okay, let's just do it."

Stephanie:

A whole brand new, the vision, of Grubhub, just photography.

Alex:

Exactly, exactly. These two folks get on the phones, start calling photographers, start calling restaurant owners and scheduling appointments to have the photographers come in there. That becomes basically their job for the next two months.

Alex:

Then they organize a really [inaudible] visitors for these specific menu pages see the photos, and others don't. They do some serious math to try to say, "Hey, here's the incrementality in here, and here's the efficacy of the spend comparing to what Google ads would be, or Facebook ads would be." They discover that those photos are actually a better way to spend marketing dollars, than any actual marketing.

Stephanie:

Yeah.

Alex:

I, at that point, am kind of floored. I come to them, I'm like, "Okay, you guys are on fire, this is amazing. Let's take your thing and give it to operations and scale up this thing." They say, "No, no, you don't understand, you don't understand. This whole project sucked. We spent our entire days on the phone with restaurant owners, trying to schedule appointments. We are going to make it better."

Alex:

I'm like, "Wait, what's going on?" They say, "No, no, instead of scheduling appointments with the restaurant owners to take photos, we are going to rent Airbnbs and photo studios around town, then order food from the restaurants, bring it to those Airbnbs. Our food stylist is going to make it look good, and we're going to take photos."

Stephanie:

Oh my gosh.

Alex:

I'm like, "Wait, wait, what? What?"

Stephanie:

That's another level.

Alex:

Yeah. My immediate reaction from this is, "Have you ever seen delivered food? It does not look good." They obviously told me to go pound sand, as they should have, and they showed me the first photos from these experiments. Oh my God, those first photos look much better than anything taken in a restaurant, because food stylists are really good at their jobs. If you were able to control the lighting, you're able to take much better pictures.

Alex:

When they actually tried it, they discovered that instead of doing two photo shoots a day, the photographer, who's the most scarce and expensive part of the whole operation, is able to do 20 photos shoot a day.

Stephanie:

Wow, that's efficient, that's amazing.

Alex:

As you can imagine, at that point, my mind was completely blown. We indeed operationalized this with folks whose day job was operations, as opposed to marketing. This was the example of really learning what learning means.

Stephanie:

Mm-hmm (affirmative). You kind of picked the markets to do that in, or you kind of see a market not doing so well, and those are the ones that you focus on getting the good imagery for, versus allowing that... UGC content to work well in other markets, or how do you think approaching that? Because it seems like something that would be really hard to scale, ordering a bunch of things all the time from every market in the U.S. How do you think about creating those campaigns?

Alex:

Yeah, yeah, yeah. With hundreds of thousands of restaurants on the platform, we indeed have constrained resources to do these photo shoots when we can. We can't do all of them next month. We had to be somewhat thoughtful on prioritizing things. A few things came to mind for being able to select the right restaurants to do this in sort of the right markets.

Alex:

First is, conversion. If consumers land on the menu, and end up buying stuff anyway. Well, that's cool, I guess they don't need the photos. If on the other hand, conversion isn't amazing, but the number of visitors to the menu page is super high, hey, this might be an opportunity to actually add some photos and improve that conversion.

Alex:

By digging into the data, and looking at where the majority of the incremental impact can be, we develop this framework for allocating this constrained resource, which ended up effectively being an investment of marketing dollars into a channel that's sort of marketing, but sort of not. Is it product? Is it operations? I have no idea.

Stephanie:

It's something, all the above.

Alex:

Right?

Stephanie:

How do you think about, you mentioned incrementality quite a bit. How do you think about that throughout your organization, when developing these experiments and seeing what works and what doesn't work?

Alex:

Sure. First, if you don't mind, allow me to define it as-

Stephanie:

Yes, please.

Alex:

Because I think that's super important. Incrementality, to me, is what would have happened anyway? If you didn't do your glorious marketing campaign, or this amazing product improvement that you just rolled out. This is a difficult question, because it's really attempting to attribute the entirety of this success, or entirety of what's happening during a campaign, to the campaign.

Alex:

Let me give you some intuition behind this, right? Let's say you go to, I don't know, gap.com or something like that. You see a banner in there that says, "10% off." Well, obvious, a lot of people are going to click that banner, and a lot of people are going to use that coupon to get 10% off of their transaction. The key question, though, is, what portion of those people would have transacted anyway?

Stephanie:

Yeah, they went there directly. They probably would have.

Alex:

Exactly, it's clearly not zero, because before you launched that awesome 10% off coupon, some people were buying jeans yesterday. Being able to, with confidence, judge what that incremental behavior is, and what is the incremental CAC, and incremental LTV, is super important. Simple back of the napkin as to how you judge this is, let's say yesterday, a hundred people bought those jeans. Today, 110 people bought those jeans. It's not a real AB test, obviously. But all 110 people used your 10% off coupon. You can wrongly suggest that all 110 converted because of your coupon, or you can look at the truth in the eye and realized 110 used the coupon, but 10 really only needed it.

Stephanie:

Do you think a lot of brands are missing this when they offer these discounts, and maybe unintended consequences that could come from it? I could see a lot of consumers, if they get used to you always having discounts, then they just wait. They're like, "I'm going to wait for that next 10% off coupon," then they don't have a buyer at all.

Alex:

Yeah, it is super dangerous. I do think that in some industries, there's exactly that happening, right? We know of the right times during the year to buy a TV, so we don't buy a TV until then. We know when the right time of the year to buy home improvement equipment, and we don't buy it until then. Exactly what you're describing is a real danger.

Alex:

It's not just a danger of delaying the purchase, it's a danger of create a permanently less profitable business. Imagine is, every Friday, Grubhub was to, let's say, give all our consumers three or five dollars off. Not only are Thursday orders going to be delayed, because our consumers are going to be like, "Hey, I don't really care when I get takeout. I'll cook one night and I'll get takeout the other night." They'll delay it until Friday, but those Friday orders are going to be less profitable.

Alex:

So we permanently teach our consumer base, if we take that route, to not only delay their orders, but to make them less profitable. That is a real issue and something you got to be super careful with, which is why you must measure incrementality.

Stephanie:

Yeah, especially right now. You see so many people discounting everything, it's kind of scary to think. How are you going to come back when your entire, everything on your store online, is 80% off? How do you come back from that?

Alex:

Most definitely. Now, if you have physical inventory, the opportunity cost is not zero. Right? Let's say if you're selling digital goods, for example, right? Let's say you're selling access to, let's say a song, or a book, right? Your fixed costs in that situation, your cost of an action, is terribly low, right? As opposed to if you have goods in the warehouse, and you aren't able to sell them, there's very meaningful fixed costs for you that you need to deal with.

Alex:

It might be, actually, quite reasonable to be running these high promotions, but if you are, you better be running it as a real AB test. You better be able to confidently say that this is the true incrementality of this 80% off coupon, and that's the true value that I'm getting out of it from both not needing to keep these products in the warehouse, but also from just sheer revenue from the consumer.

Stephanie:

Yeah, that makes sense. Do you have a good platform or way that you've set up metrics and things like that to measure that incrementality in a way that's not really manual, and then you can just kind of see how the campaigns and what they're doing is performing against each other?

Alex:

Yeah. In lower funnel channels, it is actually fairly easy to set up a platform for this, and we have. There are tools that you can use for it, right? Google Optimize, for example, or Optimizely, right? We have a combination of in-house and these third party tools to do product experimentation, for example.

Alex:

For things like CRM, couponing in the apps, or issuing emails with coupons, or push notifications, really good experimentation platforms don't exist off the shelf. We had to do some math ourselves. Some of that math turned out to be fairly fine tuned to Grubhub's needs. Here's what I mean by this. We're an LTV business. It's not just about the immediate transaction, it's about what happens after that transaction.

Stephanie:

Yep.

Alex:

For example, if a consumer ends up converting at a higher rate, and then afterwards has a poor experience and doesn't come back, that actually is terrible, terrible, terrible. Your typical, immediate conversion optimization tool, would just look at the first part of this. Oh my God, they converted at a better rate, great, awesome, keep it.

Stephanie:

Yay. Yep.

Alex:

We had to build tools specifically designed to capture these long-term effects. We typically look at the results of these long-term activities over the context of a month, right? So we need to see what happens to consumers for a meaningful amount of time to have high confidence that it indeed is net beneficial or not.

Alex:

Of course, we're able to look at things fairly early, and if something's a terrible idea, we're able to kill it early. But, in order to be able to confidently say what is the impact on the LTVs, we had to build tools. These in-house tools for many CRM things that we do today.

Stephanie:

Got it.

Alex:

Even then, it's just for lower funnel. It's just for CRM and product. How do you judge the incrementality of TV versus billboards? That is a whole other, super complicated story.

Stephanie:

How do you think about the intersection between your CRM and your content management system and your actual commerce platform? How do you create a good environment where they all interact together, and people can see a holistic view of everything that's going on?

Alex:

Great question. I don't think I have a perfect answer for you, other than enabling as many work streams for experimentation as are possible. That is, allowing the CRM team to run experiments on their own, without involving a bunch of product people, without involving a bunch of finance and analytics people. Similarly, allowing the front end or pricing optimization team to run experiments on their own, and do very specific price optimization experiments just by themselves.

Alex:

The more work streams like this you have running in parallel, the more you're going to be able to learn, as an organization, per unit of time.

Stephanie:

That seems like a great answer to me. It also seems like you would get a lot of, you could have a customer with a negative experience, but it would be because of maybe the restaurant. It seems like you guys would have a lot of insights into maybe how to help restaurants improve, where it's like, hey, every time someone orders this thing of sushi, you always forget the wasabi, and man is that making people upset. Do you ever send that data back to restaurants to improve the products as in their food, or the customer experience, or anything like that?

Alex:

Most definitely, you hit the nail on the head. We are in a really unique position of knowing not just who the people were, or when they placed the orders at your restaurant, but knowing exactly what they ordered. We can see exactly that pattern, right? We can tell you that on Tuesday night, the reviews for people ordering sushi, are actually worse than on any other night. We can help you see that, so that you can train the person that's working on Tuesday night.

Stephanie:

[crosstalk 00:43:21].

Alex:

These kind of insights... Yeah, totally. These kind of insights are exactly what we believe is what is something that we can uniquely provide to our restaurant partners, besides demand. Of course they come to us because they're interested in demand, particularly now. But we can do more, and we've been building a lot of systems specifically about that, that are effectively... you can think of this as recommendation systems in the grand scheme of the word of giving recommendations to the restaurants about how they can lend the totality of their business more efficiently. For example-

Stephanie:

It seems like that could be a whole different business for you guys to also operate.

Alex:

It's quite synergistic in our minds, right? If we're able to make our restaurants more successful, it actually makes us more successful, in turn. Because, those consumers who are placing orders and are not getting any wasabi with their sushi, they are ultimately not happy with Grubhub. We want them to have an amazing experience.

Alex:

Whether the restaurant wins just on Grubhub, or throughout the totality of their experience, because, let's face it, that restaurant might be serving other delivery platforms, and soon enough, hopefully, dine-in, as well. That retraining is going to help the restaurant across the board. We actually very much welcome that. That means that we're able to create the value not just for our platform, but for the restaurant, and increase the chance that this restaurant will, ultimately, be successful.

Stephanie:

Mm-hmm (affirmative). I think that's a really good point, especially as a lot of brands right now are shifting quickly to the world of Ecommerce and trying to figure out how to sell online. There's going to be a lot of new touch points that they maybe aren't anticipating that could actually hurt the consumer experience. If you've got the UPS guy throwing your box over the fence, and it's getting crush, there's a lot of things that actually, you maybe wouldn't even think of, as a brand, of, "That's not my job," when really, everything form start to finish to delivery and afterwards, and the follow-up, all of that's your job. And how do you think about controlling that experience with so many touch points?

Alex:

You are so right. The totality of this is their job. From the first ads that they see on TV, to what shows up when they look on SEM or on paid social and discover your brand there, too. The first purchase experience to the interaction with the UPS guy, to the interaction with customer service. All of that, in totality, is what the brand relationship really is, what the product really is.

Alex:

As marketers, we can't just care about that ads. As product people, we can't just care about the bits installed on the phone. They, in their separation, they don't particularly matter. As you saw from my story with the photos, that really was quite profound to me, right? We kept looking for a solve to get more customers and more sales through marketing, and that solve wasn't there at all. The most efficient solve was far outside.

Stephanie:

Mm-hmm (affirmative), yeah, such a good reminder for all brands to think about that, like you said, totality of the process. Because you have a software engineering background, I feel like I'm allowed to ask you tech questions. I saw on your, you guys have a blog on Medium, or your engineering staff does. They were talking about how they were creating discount codes using crypto. It made me wonder, what other kind of technologies are y'all experimenting with, or seeing success, or how did you think about running the platform that Grubhub's built on now?

Alex:

Sure. A few things are super important. One is having a scalable platform that can withstand demand, and that can withstand massive spikes in demand. As luck would have it, most people in Chicago, want to get dinner approximately at the same time.

Stephanie:

Yes, who knew?

Alex:

Right. What a pain in the butt. We've been trying to convince them to maybe come a different... No.

Stephanie:

Come on, 3:00's your time, come on.

Alex:

Exactly, exactly. Your dinner delivery window. Which, of course, creates formidable demand. Not just on the services in the backend of our systems, but a formidable demand on our logistics network. A lot of our work goes into being able to spike in response to customer demand. Let me give you one intuitive example of this. Outside of COVID, before COVID, when rain would start during dinner hours, demand would massively spike.

Alex:

At that moment, we're supposed to magically materialize a lot of drivers on the road doing deliveries. Being able to do so, technically, and when I say magically materialize, I'm of course referring to creating incentives and creating appropriate communication channels with our drivers so that they actually want to get on the road. A lot of our engineering work has to do with how we were talking about in the beginning, balancing the three sites of the network, and being able to respond to either a massive spike in demand, or response to a set of orders that were placed in the specific part of the city on the logistics side.

Alex:

Or, respond to an onboarding of an enormous partner, like Shake Shack, or Sweet Green, or Taco Bell, with their own unique needs. Remember, we work with such a variety of restaurants, right? We do point of sale integrations with a variety of our enterprise customers, which of course means that we have to have nimble systems that are able to onboard those same customers. They have to be resilient, as well. So, a lot of our work has to do with both scale and being able to deal with these spikes.

Stephanie:

Got it. Any favorite pieces of tech that you guys are implementing or trying out right now to help with those large spikes in demand? Or where you guys think the future is headed that you're kind of preparing for?

Alex:

Favorite pieces of tech. Huh. Huh. I'm going to think marketing tech. Braze has been an outstanding tool for our marketing teams. What we've discovered is it effectively enabled a whole work stream of experimentation for our CRM teams. They're able to run pretty sophisticated experiments completely independently from engineering, which increase our velocity of experimentation.

Stephanie:

Hmm, that's awesome. I'll have to check that out. Cool. So to zoom out a little bit, 30,000 foot level, what kind of disruptions do you see coming in the world of Ecommerce? What's on your radar right now? It doesn't have to be for Grubhub, it can just be in general.

Alex:

I think that the disruption is already here, where over these past couple of months, we've seen the portion of online transactions, and portion of consumers who have tried buying things online just catapult through the roof. All of those new consumers, let's face it, my 90 year old grandmother is using Zoom now. All of those consumers are a new opportunity. They have very different expectations. They don't yet know much about your brand.

Alex:

Being able to understand this newly online wave, and heightened expectations of the consumers that already happen online, but perhaps not as active with your service, right? Those, I think, are super important. This to me takes us back to velocity of experimentation, being more important now than ever. That is, truly learning from your customers. Observing them, creating experiments, measuring, and getting a feedback loop from them, so that you're able to focus and find the one thing that you can improve to make the whole story better. Maybe photos. Maybe it's something else.

Stephanie:

Yep. Yeah, I love that. It definitely seems like with these new people coming online, you have to have a bunch of different tactics to meet them wherever they are. The ones that have been working for the past year, might only work for a subset of the people because you have 50% more people that you need to market to, or develop a platform for, and it's going to be very different with how you approach those new consumers than what you've been used to.

Alex:

Exactly.

Stephanie:

All right, so, we're about to jump into the lightning round. Any higher level thoughts, Alex, that you want to share before we do so?

Alex:

If you're able to structure your organizational incentives to focus on learning and feedback loops, I think now you're going to see an even bigger reward for it in the form of market share, in the form of growth, in the form of being able to adapt to the world around you and leapfrogging the competition.

Stephanie:

Yeah, completely agree. All right, so the lightning round, brought to you by our friends at Sales Force Commerce Cloud. It's a fun and easy, quick round of questions where you have a minute or less to answer. Are you excited and ready, Alex?

Alex:

Very scared.

Stephanie:

Dun dun. All right, first one. If you are starting a podcast, what would it be about, and who would be your first guest?

Alex:

Whoa, what a fascinating question. What a fascinating question. I am obsessed with all things culture, and how do you actually create the right incentives for a technology/marketing organization? I love Simon Sinek. He is outright amazing. I learned a ton from reading him. I would probably to get him and if I can't, I'd get one of my former mentors in there, as a consolation prize.

Stephanie:

Oh, that sounds good. I would listen. I would be your first listener, and I would give you a five start review.

Alex:

Oh my gosh, thank you.

Stephanie:

You got me at least. What's up next on your reading list?

Alex:

Hmm, next on my reading list? I am reading Russian sci-fi novels these days, as a means of escaping from a tiny, one bedroom apartment.

Stephanie:

Any good ones that we should check out?

Alex:

I'm actually reading them in Russian, so I don't know-

Stephanie:

I was going to say, unless they're in Russian, then I don't know if I'll be able to read Russian quick enough to read it.

Alex:

Oopsie, oopsie, I do have a few people at my work who've been reading Tolstoy before the whole COVID situation started. I don't know if I'd recommend it now, Tolstoy does darkness extremely well. We have enough darkness around us now.

Stephanie:

That is true. Yeah, maybe not. Alright, well, what thing do you normally buy at a store that now you're just going to buy online after everything with COVID?

Alex:

What a great question. Only online now. Hmm.

Stephanie:

Tricky, tricky.

Alex:

I used to, actually a lot of my electronics. I used to come to the store and look at them and experiment with them. I have a feeling that I'm never doing that again. I used to come to a Best Buy and just try to look at different mice and monitors and all that. I got a new laptop and a new mouse online. I really like them, and I really like the experience. I was unafraid of returning them. That's it, online I go.

Stephanie:

Yeah, completely agree, especially as a lot of these companies are making the return experience a lot more seamless. Yeah, I could completely see the same thing happening. Buy things, test it out, and send it back if you don't like it.

Alex:

I was just chatting with a colleague about this exact same thing with returns around fashion. I think there's a lot of innovation to be had with moving the fear in fashion through that.

Stephanie:

Yep, completely agree, except I could see them having to now to figure out a way to resell those items in a way that proves that they've been quarantined, disinfected, and yeah. I was just thinking about that the other day. Man, that's tricky, especially for second hand market places to try and prove to the customer that these items are clean and good to go, and you can buy them.

Alex:

I agree. Solvable, I think, but I agree.

Stephanie:

It is solvable. All right, so the last final question. What's up next for Ecommerce professionals?

Alex:

I think we're going through a time when from being on the early adopter, early majority demand for most of the brands. We've become the critical source of revenue for every single brand. If you think that your company was going through a digital transformation, and is now trying to make digital just a better channel, hold on to your seats, because it's not the only channel, and the majority channel. So, the demand for expertise in our area is increasing very rapidly, and the demand for learning in our area is also increasing rapidly. I think this is a wonderful time to be in Ecommerce. I think this is a wonderful time to be learning and doubling down on Ecommerce. I'm excited for all of us to be right at the center of this transformation.

Stephanie:

I love that, love the positivity, and yeah, it's definitely an exciting time to be alive and experiment and try new things. This has been a blast Alex, thanks so much for coming on the show. This is your second appearance on a Mission podcast, so yeah, we're so thankful that you came back and joined us again.

Alex:

Stephanie, thank you very much for inviting me.

Stephanie:

All right, talk to you later.

Alex:

Cheers.