335: How to Run an Early Access Program

[0:00:00]

Steli Efti: Hey everybody, this is Steli Efti.

 

[0:00:03]

Hiten Shah: And this is Hiten Shah, and today on the Startup Chat, we’re gonna talk about what Steli calls a beta program and what I call a early access-

 

[0:00:11]

Steli Efti: Access program.

 

[0:00:13]

Hiten Shah: … Program. And that’s okay. This is one of my pet peeves. I don’t like calling it beta, so I’m gonna jump right in and talk about how to do this correctly.

 

[0:00:21]

Steli Efti: Boom.

 

[0:00:22]

Hiten Shah: I like talking about and framing it as an early access program because we have to grown to think of beta as buggy. So when you want to bring people into your product early, or a new feature, it doesn’t matter, you don’t wanna call it a beta ’cause that implies it’s buggy. Also, I don’t think the general population out there knows that a beta really is. It is a very techy type term.

 

[0:00:49]

Steli Efti: Fair.

 

[0:00:49]

Hiten Shah: So for me, I like calling it an early access program. I like framing it like that because that feels more exclusive. It really pulls in people that we would consider early adopters. Also, by the way, never call your customer an early adapter to their faces. That doesn’t help you. They are just a human being, they’re a person, and they’re eager to use your product early because they really want this problem solved, or they’re really just excited. So that’s what I’m gonna start with, Steli. It’s called early access and it’s meant to make people feel special, not make people feel like robots.

 

[0:01:23]

Steli Efti: Boom. All right, Hiten’s on fire. I can already attest. You’re right. You’re very right. You know, when I talk about it in a sales context, whenever I talk to start ups about how to get the first few customers, I learned this phrasing from you a long time ago and I always talk about early access. But when I talk about, when we talk about it internally about giving a certain subset of our customers access to a new feature, we still call it beta. So I’m gonna work on changing that because I really like your reasoning and we both deeply care about words and communication, the impact it can have on things.

 

[0:02:09]

Hiten Shah: I’m gonna go a little deeper. So what we’re doing and my company is basically aspiring towards, and I say it’s aspiring because it’s just something that I think takes time and process to get right, but we actually have a three phase release now. We do a internal release. I saw someone on Twitter talk about how drift.com does this and they call it a tracer bullet. I don’t know what that it is.

 

[0:02:35]

Steli Efti: I have no idea either.

 

[0:02:36]

Hiten Shah: And I know that, but they have all kinds of cool stuff going on internally with crazy names, so that’s cool. But basically, I think what they’re implying is that this is an internal release, this is what I like to call it, an internal release that is for the team internally. It could be what we call feature flag, which just means turned on for the internal team or put on some URL that no one can else see, whatever. And your playing with it really fast internally. It’s not full-featured, it might be really janky, but you can start seeing it. Whatever the user experience in, whatever the engineering problems are, you start seeing them to the surface and you design your product development like that. Then, after you’re relatively happy with an internal release, at the first time when you’re like, “Oh, this is good enough for a customer”, you call it early access release. This is what we’re really talking about on this podcast, we’re talking about the early access release. But what I think is important is what comes before and what comes after because that can help people understand why an early access program is so important. Well, the internal release makes you ship really fast and not feel so critical of what you’re shipping ’cause a customer might not see it, or probably won’t see it until later. So that’s really important ’cause that gets the rubber to meet the road on the product, or the feature, or whatever it may be. Then you have an early access program, which you know we’ve talked about a little bit, we’ll talk about it a little bit more. And then you have what I call basically the full release. Full release is a marketable release. Drift calls these marketable moment, I just call it a full release, Product Hunt launch if you use Product Hunt a lot for your product, or a PR release where it’s a full release and it goes in market. Anyone can sign up for it and use it, that’s what that is. Early access, not anyone can use. You are bringing in people in order to get feedback on the product. The whole idea is to get to that full release, that’s awesome. One that has maximum impact. That’s the key. So the early access programs are designed for you to learn so that you can figure out what that full release looks like. If your product is new and you’re still working on it, that early access release, you might release it, 50 people use it, 10 people, 20 people, and you might throw it all away and start over with another internal release ’cause you learned a bunch of stuff, and then early access release, and then you keep going between the internal and early access release, even throwing away all the code, whatever, until you get to a point where you’re like, “This will work.” Whether it’s you’re looking for high retention or you’re looking for high growth or you’re looking for high MPS, whatever you’re metric is in the early access program that you’re trying to get past, then would go forward and have the full release. So that’s what you would do for a new product. If we need to talk about new features, we can talk about that too but early access programs for new features is super important. That’s why I was saying, at my companies, we aspire for three releases for any feature, any new product, anything. That way we can really sequence what we’re looking for in each release and exactly get to that full release, that “Dang it, great. Polished and ready to ship.”

 

[0:05:39]

Steli Efti: I love it. It’s funny enough, the sooner we release, we don’t really release new products, but we do release new features.

 

[0:05:47]

Hiten Shah: You’re about to though. You’re about to.

 

[0:05:49]

Steli Efti: Yes.

 

[0:05:50]

Hiten Shah: Fess up. There’s a new feature, but it feels like a product.

 

[0:05:52]

Steli Efti: Yes.

 

[0:05:53]

Hiten Shah: Just saying.

 

[0:05:53]

Steli Efti: It’s a big one. It’s a big one. But it’s always like, internal first, then we do early access,-

 

[0:05:59]

Hiten Shah: Yep.

 

[0:05:59]

Steli Efti: … And then you do a full release. Let’s talk a little bit about two things. One, when is the right time to go from internal to early access? When do you know when you’ve accomplished whatever the internal release is trying to accomplish and you’re ready to invite some early access customers or users to test, to play around, to give feedback? And then how do you run that early access program, from inviting people, to using it, to eliciting feedback, to acting on the feedback, to then deciding whether you’re ready to rock and roll? Let’s talk about those two things.

 

[0:06:37]

Hiten Shah: I think internal releases are designed so that that internal team can play with it and see it for real and get real, tangible feedback so you can write tickets or whatever you do, and product manage properly. What I mean by that is, you get an internal release so that you can figure out what’s wrong with it. You get an internal release, you can figure out what’s wrong with it before you’re willing to let customers see it. So for me, internal releases are quick, they’re dirty, and they might require a lot of work to get to an early access release, or they might require a little bit. But, I feel like if you don’t have an internal release and you just aim for early access release, there’s gonna be some process that’s still an internal release, which is like the equivalent of internal release helps you do QA. It helps you really understand whether it’s gonna hit the mark or not, or whether it’s polished or not. Another reason for internal release is ’cause you just wanna build it fast, ’cause the estimates you’re getting back from engineers and product people and the whole team are pretty high. So then you’re like, “Okay, what can we quickly build?” And I think it’s really about quickly building and not thinking too much about the customer or anything like that. You’re obviously thinking about a customer ’cause you’re gonna build a feature and they’re gonna see it, but you’re really thinking about, “How can I get this right? How do I make sure it’s polished? How do I find all the things I need to fix versus find them during the early access program?” ‘Cause early access program is about your customers and learning what they think. The internal release is learning what you internally think and what you still need to do, which is a lot different than the early access feedback, which is really about customer sentiment, right? Internal release is about basically, is it good enough? Is it good enough for a customer to see it and what’s it gonna take to make it good enough?

 

[0:08:11]

Steli Efti: I love that. Yeah, and the other thing is there’s always one thing, conceptualizing something, or planning it is a very different thing when you actually see it live. Even if it’s a janky, buggy version of something, there’s a magic to a functional feature. If it’s poorly functional, to give you a real sense of experience and to give the team a sense of what’s lacking or what’s already good or what needs to be done. The other thing is, you get to figure out a bunch of really obvious problems with the new product or new feature by just having a few team members play around with it. So it’s kind of a waste of your customer’s time. It’s a way to be respectful of your customer’s or your user’s time. Do not give them something that seems that you have been completely thoughtless and you don’t value their time. It’s like somebody sending an email asking me for feedback and I can tell they never spent 10 minutes really thinking about this careful. They’re just shoveling shit my way, right, mindlessly. So there’s a bunch of obvious mistakes or bugs that I think a modern customer would expect you have figured that out before you send it over to them. You can figure these things out pretty easily when you an internal release. Now, when it comes to rolling out the early access and running it, real quick, let’s run through two or three tips in terms of how to do this well. How many people or companies or customers should get invited typically, how do you invite them, how do elicit and ask for their feedback, how do you make sure that you get the most of out it?

 

[0:09:53]

Hiten Shah: I like opting people into that. And I like doing it in batches usually. It also depends on how much feedback you need or how polished that early access release is. If there’s still a lot of questions you have about what people are gonna think at that early access release, whether because there’s more research you wanna do or it’s a bigger feature or it might have some bugs ’cause there’s things like people on Windows, people on Mac, people on different browsers might have issue with it for whatever reason, then I like to do it in batches so that I can fix in batches. I also like to get people to actually tell us, “Hey, I want this and I’m gonna give you feedback.” So it’s a request for feedback. Really, an early access release is a request for feedback on something early. You want to find those people that want that feature so badly that they will deal with giving you feedback. They want to give you the feedback. So you’re really committing people to it. You’re not just saying, “Hey, here’s a new tab or a new part of the product”, people can randomly click it and all of a sudden they have it. That’s not the purpose of this. The purpose of this is to be very deliberate and tell people, “Hey, we’re gonna ask you for feedback. We’re gonna ask you for feedback at least once a week. Is that cool?” Or at most once a week, whatever it is you want to do. And then you encourage it and you actually email them and tell them, “Hey, you’ve been using the product”, or “You haven’t been using this feature”, or whatever. “Tell us why. Hey, this is that feedback opportunity I told you I wanted, right?” So to me, it’s all about feedback and people opting in to the early access with the idea that they’re gonna need to give you feedback. Then feedback becomes easy.

 

[0:11:24]

Steli Efti: I love it. I think the way that you then communicate with those customers during the early access, the way that you respond to their feedback, the way you follow up with them, it really can either strengthen the relationship or not. In our case, we’ve seen a lot of times that the customers that we invite to early access, they are so pumped and psyched to have access, they are so pumped and psyched to be able to give feedback and have influence. A big reason for that is that they feel heard and they feel that we care and they feel like their time is valued. So when they send feedback, it’s not like quiet for four days and they don’t get a response from us or when something doesn’t work and they ask for us to fix it, it’s not quiet time. A lot of times companies get all excited about inviting people to their early access, but they don’t have somebody manage that process or the founders realizing that they’re gonna have to manage and share in that process and really dedicate time and energy to these customers that they’re inviting to use the product or try the product and get feedback. In our case, a lot of times we also will tell people, “Hey, we’re gonna want a lot of feedback. We’re gonna work really closely with you and if this is truly creating the value for you that we both think it is, you’re really happy with it, we’d love to take all those conversations and all that feedback and potentially publish it and use that in our marketing in the future, obviously with your permission.” So we typically turn those early access times also to create content and material and testimonials and case studies so we’re ready for the big launch, or the public launch. We have a ton of firepower to go behind it. All right, I think that that’s everything that people need to know to get started with it, and running it. It’s really not that complicated, but doing it well, I think, has all to do with the intention and the energy that you give it. Instead of just sending an email to a few hundred people and saying, “Now you have access to this”, you need to think about that time of running the early access program as something where you’re gonna have to follow up, and send emails, and have screen shares, and really be high responsive, and really prioritize these people, and make them the most important users or customers that you have to make this really worthwhile.

 

[0:13:51]

Hiten Shah: Yep. You’re making champions.

 

[0:13:54]

Steli Efti: There you go. All right, I think that’s it for us for this episode. We’ll hear you very soon.

 

[0:13:58]

Hiten Shah: See yeah.

[0:13:58]