41: Leading Experience Design for the Military (ft. Colt Whittall)

Transcript

What follows is a lightly edited transcript produced by our podcast software. It may retain some textual glitchiness.

Jesse: I’m Jesse James Garrett,

Peter: And I’m Peter Merholz.

Jesse: And we’re finding our way,

Peter: Navigating the opportunities

Jesse: and challenges

Peter: of design and design leadership.

Jesse: On today’s show, “mission critical” takes on a whole new meaning when you’re the Chief Experience Officer for the United States Air Force. The first person to take on that role, Colt Whittall, joins us to talk about getting things done when you have more influence than authority, finding meaning and purpose in government service, and taking risks in an environment where failure is not an option.

Peter: So Colt, what I’m curious about is, I think this is a new role, right? This role didn’t exist before. I don’t know if it was created for you or if someone had the idea and then found you. But why were you the right person to be the first Chief Experience Officer at the Air Force?

Colt: So, great question. First of all, yes, this was a brand new role. Well, let me just tell you how it was created, and then we’ll talk about, why was I picked to do it? So it was, it was really created because the Air Force knew we had a user experience problem, primarily with IT but also with software. We had a lot of complaints. And this has been going on for years, and I think there was a general sense that maybe things were getting worse and not better.

And so the CIO at the time knew he had an issue. He also had an understanding of the importance of software, which, the CIO role is a little different in DoD, their control of enterprise IT is fairly direct, but their control of applications is less direct as in a corporate environment. There’s generally control of those that’s spread out amongst a bunch of different organizations. He knew he had an issue. He had some authority over it and wanted to create this role. Then why me? I came out of about a 20, 25 year career in the digital agency space, digital consultancies.

I was at Deloitte in the nineties, built one website after another and mobile app and so on from. 2000 to 2018 with a digital agency that we spun out of Deloitte and did a lot of fantastic work in a variety of industries, media, health care, wellness, travel, hospitality, financial services, and some federal government.

And Bill Marion, the CIO, had been a client of mine at one point. And so I had a, probably a fairly typical background for people that listen to your podcast, right? A mix of digital strategy and user experience and customer experience. And by 2018, I was at the equity partner level, practice lead kind of level. And we sold the company and we were looking to get out. And so that’s what I did. And took a little time off.

I had always had an interest in federal government and user experience and technology in the federal government. And when I was in graduate school I pursued it a couple of different ways and kind of built on that. Did internships in DC. One of them had to do with high definition television. I worked on the high definition television project for AT&T, interacted a lot with their government affairs group with Bell Labs up in Summit Hill, New Jersey.

And then I did an internship with the FCC. It was set up by my thesis advisor, who’s named Dale Hatfield. And worked on the first ever narrowband PCS auctions. This whole idea of the intersection of technology and government’s always been a big passion of mine.

And so after a career in the digital agency space , I happen to know some people in government. I approached Bill Marion, Bill Marion had a problem. He knew me, didn’t know a lot about me, but he, he knew enough about me. That on LinkedIn , I basically just said, “Hey, Bill, I’m thinking about going to work for the government for a while.” In my mind, I’m thinking US Digital Services or something like that, which you may have heard of.

I wanted to talk to Bill because I knew him and I need a little advice. Like, how would I even go about this? , I don’t want to just go looking for, jobs on USA Jobs and just start submitting resumes. I’m like, there has to be a better way. And Bill responds on LinkedIn five minutes later, says, “meet me at the Pentagon on Tuesday.” That’s literally what he said. And I flew up and I met with Bill and we started talking about the challenge of user experience and performance across IT and the Air Force. And I think we began to formulate what this role could look like and that’s really it.

Most of my background, as you just heard, is more in the software and application, and more consumer and outward facing. However, a lot of the tools we need to fix our problem internally for airmen– which is what I focus on, I barely touch the external facing web stuff at all– but a lot of the same tools apply internally for the applications and systems and IT that airman use.

The Scope of the Chief Experience Officer

Jesse: So I think it might be a little challenging for folks to conceptualize what those needs might be inside the Air Force. Why you would need a chief experience officer. I would love to hear a little bit about, to the extent that you can without committing treason, the communities that you serve, a little bit the broad use cases that you’ve served just to get a sense of the scope of the challenge that you’re really talking about here.

Colt: Sure. The scope is huge. I mean, first of all, think about all the missions that we do, right? You know, first of all, we’re the, air force and the space force.

Jesse: Right.

Colt: Delta Airlines has, I want to say eight or 900 planes. We have over 6, 000. All right. So it’s huge. And we have a vastly wider range of aircraft, right?

Both fixed wing, rotor, drone, all of it. Right. Plus we have the space force. We fly satellites. We work with contractors that launch us into space. We do all of that. Right. Then we have land base missiles. We have air defense type stuff and radar installations of all types. All of that it, all of those systems have an IT component to it. All of them, right?

Just one, for example, we’re replacing the Minuteman missiles that sit in ground based silos. It’s a program called Sentinel. That will have big deadlines coming up over the next few years because we have certain timelines that we have to hit. But there are 10, 20, I don’t even know what the latest number is, there’s a number of systems that support all of that. Everything from the logistics to the control, to all of it, right? All of that has to be designed and built. So we have, you know, I was talking with the CTO of a major, well, I’ll just say it was Coca-Cola, several years ago, and he was telling me about the number of systems they have.

First of all, they have more than I would have thought or a global organization, right? I thought there’s a lot of duplication, right? They’ve acquired a lot of companies over the years. Yada, yada, yada. We have many, many times more than that, right.

Jesse: Mm

Colt: It’s a big complex. Okay. Now, so you got all these we have, let’s call it 700,000 total airman contractor. That’s active duty guard, reserve, civilian, and contractors that are like on our networks in our systems.

Call it 700,000 plus or minus. And they are interacting with, call it, there’s maybe 40 systems that have more than 50,000 accounts, quite a few of those over 500,000 accounts. And then there’s this long tail of systems that go all the way out to very tactical things, some of which are extremely mission important, that might only have a few dozen or a few hundred accounts, right. And they’re doing everything imaginable, right? They’re flying satellites, they’re handling logistics, all of it, right? They’re putting together schedules of flights and sorties and getting fuel on planes and recording the hours of pilots and crews and making sure it’s scheduling of all of the above.

And then you’ve got really mundane things like getting people paid on time and dealing with health care and booking their flights. It’s everything that a typical large organization does, plus all of these other long tail of missions. And then on top of that, just to make it even more complicated, we are literally around the world and supporting a lot of hardware out there.

Across Levels of Security

Colt: And then we’re doing it at multiple different information security levels, which if you’ve never dealt with information security in this kind of an organization, it gets a little crazy, right? Because you’ve got completely public information, right? Then there’s unclassified and essentially public, like the public could have it. And then there’s controlled unclassified information, which is just things that aren’t publicly releasable, but they’re not classified.

And then there’s secret, top secret, and various flavors and things above that, So, it’s a complicated environment. And these systems, think about it, some live at one level, some live at another level, some live at the other level. So it, it is a complex environment.

It’s not unique to the air force and space force, right? This is across all of DoD.

Developing an Agenda

Peter: Given this complexity, I would imagine a big challenge in your role is maintaining focus, figuring out what, what to work on. And I’m wondering, when you joined and then maybe over your time there, something that we heard from prior heads of design was the importance of some vision. Not necessarily destination, but generally a direction they were trying to move things towards.

And I’m wondering, did you have something like that when you joined? Or what did it take for you, because it sounds like this was probably a new environment, to realize this is what I’m here to do. This is my personal North Star. And then how did that help you make sense of this complexity, where it would be very easy to get overwhelmed by all the things you could do, and you needed to focus on the things that actually advanced your agenda toward that vision.

Colt: So we definitely have a vision now. But it took a little while to evolve. It’s not that the vision evolved. The way that I talk about it got clearer and clearer. And it can probably be clearer still. When I took this role it was June of 2019 and like any kind of incoming executive, I kind of set the expectation there was going to be about a 90 day sort of study transition kind of plan, and then I’ll come back with an approach. And the good news is, after that meeting at the Pentagon with Bill, I had about six months while they were creating the job, and I had a pretty good idea that I was going to get the job. So I had a lot of time to prepare.

And it was a little bit my sabbatical, or you know, I call it my gap year. So I had a lot of time to prep. So I came in with a lot of preparation. Then I had my 90 day transition. I booked a lot of plane tickets. I visited a lot of places. I talked to a million people. I looked at a lot of data. I’m a career professional consultant.

So I came back with a nice, big, thick document. And I thought it was very well- structured and clear. And , like a good senior executive in the federal government, my boss said, “Love it. On board. This is exactly what you need to do, but it’d be better if you could put it in a placemat.”

And I’m like, okay, can do. Put it in a placemat. And did a lot of talks and you can actually go out and you can go on YouTube and you can actually find videos of me presenting my placemat in public events and the placemat was helpful, but it wasn’t clear enough.

Colt: And so not long after that, people were asking me, Colt, okay, love the placemat, but what’s our strategy? And we needed something that applied to both enterprise IT and software that people can understand very quickly. And so essentially the strategy is, we’re going to treat airmen like they’re customers.

Which is a big shift in mentality in a government organization. So we’re gonna treat our airmen like they’re customers. As if we’re a major IT services company, like a Microsoft or Google or somebody else, right? And so we treat airmen like customers, and then we’re going to measure the experience from their perspective. And then we’re going to track that over time, figure out how to make it better and manage service levels. Fundamentally, that’s a strategy.

So it’s an outside-in type of user-centric approach focused heavily on measuring experience, because that’s something that we can do at scale. I mean, think about the scale of the organization I described earlier. I can’t think of another way to do it. Well, there’s many challenges with user experience, but one of the challenges with user experience is, at the end of the day, it improves system by system, application by application, user journey by user journey.

And so we needed a way to kind of say, okay, how do we go set the bar, measure the bar, figure out where the bar is, and then start trying to get the entire culture to move the bar up. We needed a clearer, more succinct strategy and that’s when I got down to, okay, here, the strategy is fairly concise: treat airmen like customers, as if we are basically a big IT services company, and measure user experience from their perspective as they’re doing the mission. There’s implications to all those things, and then track it over time and manage service levels, you know, to improve them. And that is fundamentally the strategy.

People seem to be able to understand that very easily at all levels. I talked to general officers, totally get that because they apply similar techniques everywhere. You know, the organization understands all aspects of that. They understand about measuring service levels and managing service levels. Once you start putting user experience into those terms, an organization as big as with these kinds of missions can grab on to it and say, yeah, that’s a good way to do it.

Jesse: This is fascinating because it feels like such a shift for you, away from consulting work, which I know that you’ve done for many years. And I’m curious because it feels like there’s an element of consulting, which is about get in, make the strategic impact, get out and move on to the next thing.

And this is about, getting in and going deeper and deeper and deeper and getting sort of more into it. And I wonder, what was attractive to you about the shift to this context? And what did you discover when you actually got to the other side?

Colt: You know, you’re right. Career professional consultant going from project to project. And we had a very good run. I mean, I was with essentially the same organization the entire time, right? I was with Deloitte, we spun a company out, we sold it so I was essentially with one organization the whole time. But never really got to see a lot through in the way of our client work. I even had accounts for years, one account 7, 8 years, so saw a lot of things through, but not like this. So my goal was, let’s see if we can move the needle in a massive way for an organization this big. And I was fairly convinced that we could, but that was what was really attractive to me is, let’s attempt to measure user experience in a way that’s meaningful and relevant to airmen doing the mission, and their mission. And then let’s track it over time. And then let’s figure out what levers we need to pull in order to move the metrics and deliver a better service level. And have everybody agreed that we did it.

I mean, fundamentally, that was what I wanted to do. That’s what attracted me to doing this. And I went in fairly convinced that we could do it. I didn’t know exactly how we were going to do it. But I had a lot of ideas on how we could do it. But you’re going to go look for what are the biggest levers you can find to shift how airmen perceive the service level though they’re getting from IT and focus on those areas.

Peter: I want to unpack a couple of terms just to make sure that we’re all using the same words to mean the same thing. I’m not used to thinking of user experience as having a service level mindset, apart from, like, my design team will have a service level commitment, maybe to some part of the business, but that’s not what you’re talking about.

So what do you mean when you’re saying service levels in this context and what is UX’s responsibility, if it’s separate from the rest of it.

Colt: Yeah. No, great question. Okay. So when I’m talking service levels, I’m using the term a little bit broader than you’re probably thinking about it. So it’s, performance response time and all of those kind of aspects of computing. You know, my computer not crashing, stuff like that, it’s just service levels.

Peter: Kind of what we would think of as quality and quality assurance.

Colt: Quality assurance. Exactly. But I’m also just thinking of making sure that the applications that you are using are easy to get to, meet your requirements, and are easy to use without having to get a lot of extra training that is specific to the application.

There’s a lot of training that you need to do your job, but you shouldn’t need so much training in the tool. So I’m using service levels just in a broader sense.

Peter: Almost like, yeah, how we would think of quality.

Jesse: Yeah.

Colt: One other thing just to clarify, because this confuses a lot of people, and don’t even think about it so much when I talk about it anymore. But keep in mind that I work for the CIO. But my job spans both enterprise IT and software and applications.

So, think Google. Google could produce the most fantastic search results on the planet, but if it took 10 seconds to get the search result back, you would still think it was terrible. And we have that kind of problem, right?

Trying to putting it on our terms, our search results wouldn’t be that great, and by the way, it would take 10 seconds. So we got to solve both problems. And if I’m trying to move the bar up in a significant way, that is noticeable and meaningful to airmen out doing their part of the mission, then i’m looking for the ways to figure out how to move the bar up on wherever I can get it. So in some cases that’s on the software side, in some cases that’s on hardware systems, networks, operations, wherever I can move the bar up and deliver a better service across all of it.

We ask a Big Question

Peter: I want to unpack that. ‘Cause something Jesse and I love to talk about is multi-channel, omnichannel, very broad user experience, right?

Experience

Jesse: strategy.

Peter: Yeah, we all come from an environment where people thought UX was screens, very simple kind of software mindset, but we all know that an experience mindset can be brought to bear on a much broader set of challenges.

What I’m wondering is, you’re talking about hardware as well as software and other systems. I’m realizing I’m curious about the makeup of your team, right? You’re the first Chief Experience Officer. Who are you assembling in your organization in order to address this variety and complexity of challenges that you’re now facing?

Jesse: Well, I actually have a related question that I’d like to piggyback on this which is what did you inherit when you stepped in? Your role was brand new, but the work being done was not, and I’m curious about what you were handed, how you shaped that into something new, and what, to Peter’s point, what did you need to create along the way?

Colt: We had to create a lot. Okay. So why don’t I start with that? And then let’s come back to Peter’s question. So I didn’t really inherit much. There was not an existing team. And to Peter’s question, I don’t really have a team. So I’m what’s known as an HQE appointee or highly qualified experts.

It’s basically a type of senior executive service equivalent hire. It’s equivalent to a one star general or a first level senior executive service, where the government goes outside directly to commercial industry and brings somebody in with a particular set of expertise to solve a particular problem.

And then by law, maybe it’s by policy, but regardless, you have to be done in five years. And one other kind of odd rule is that as an HQE you can’t really manage government people. So it’s kind of like going out and pulling a commercial person into the government for a short period of time.

And so when you saw the reboot of healthcare.gov going back about a decade, I think they brought in one, two or more HQE’s. It’s not that unusual, but so I can’t go and like build a team and have a lot of people working for me.

What I can do, remember, career professional consultant, is go figure out who’s allied with this cause and line everybody up and get them all working on the same thing.

And so that’s what I was able to do. There are multiple organizations that have some stakeholder piece part responsibility for user experience.

Most of them don’t call it that, they’re DevSecOps organizations doing software development, or they’re software factories, or they’re software acquisition teams, or they’re IT organizations that are supposed to be optimizing the performance and the security of our networks and systems and desktops and everything else.

So there’s lots of organizations that do a piece part of user experience, or they own some part of it. All of them had to be lined up, put together in teams virtually to go after this. All the organizations existed, but the virtual teams going after these problems, thinking about it as UX and performance and then going after it, none of that really existed.

We did have software factories. Our largest is called Kessel Run. They’re an interesting group. You may want to talk to them sometime. Over a thousand people now, and they have a pretty robust UX capability. So they existed, but there wasn’t a lot else.

And then what had to be built? So the key things that had to be built were,- -keep in mind what my strategy was, let’s go treat airmen as customers, measure UX from their perspective, track service levels, manage service levels, make it better. Okay, so what do you have to do?

First thing you got to do is start measuring user experience. And you got to do it across enterprise IT. And then you got to do it across software. It’s a lot easier to do it across enterprise IT. There’s a lot more tools and products, and I don’t need so much consent from individual applications.

We’re in an environment with thousands, probably, of individual applications and platforms, if I had to go to each one of them and get permission to monitor their performance, and we can’t use Google Analytics inside our firewalls, but let’s say we could, we wanna attach Google Analytics and maybe do usability testing or something on a thousand applications. I gotta work with a thousand different programs. Can’t do that.

So how do you do this? So on the enterprise IT side, we had a thing called the Air Force Survey Office. They use Qualtrics at scale and there’s a FedRAMP moderate version of Qualtrics so we can attach it and we can do surveys.

And this is the way, like if you hear about a study in DoD of, pick a topic, whether it’s diversity, inclusion, or retention, or sexual harassment or any other topic people research, this is the platform that they do those studies on. So what I did is, I met with them and then we set up a, what’s now it’s evolved a bit, but starting in January 2020, we began doing a pulse survey of it.

And then we launched digital experience monitoring, which gives us performance of all the software running on individual computers. So now we know where it takes a long time to start up in the morning.

We have people complaining of 20 minute startup times, boot up times. And, and we know where it is, geographically, bases and everywhere. And then we added a network of boxes that plug into the routers that run a set of tests, looking for things that tend to disrupt performance and mess up performance of software , tests that check our PKI infrastructure, that check various changes that sometimes happen with DNS and in various other things. And so now we’ve got sort of, this is gonna get really geeky really fast, but all seven layers of the OSI stack we’re now monitoring.

So we understand the whole technology stack. Really, we understand at the top level what the user is experiencing from the technology stack. That’s on the enterprise IT side. On the software side, more complicated because in order to instrument an application with just basic web analytics… Now understand, you can’t go hook up tools like Google Analytics to our systems, right? Even the unclassified systems, you can’t really do that.

On the public facing site, yes, there’s actually a really good public facing version of Google Analytics that’s run by General Services Administration. It’s a great capability, but we can’t use that for our, call it… just inside the firewall stuff. So we, we stood up something called user experience management, and it has a user feedback capability, just a simple link so that users can provide feedback with a simple 3 question survey. And then it has an open source web analytics tool and we’ve gone through two years of security approvals to get it what’s known as ATO or authority to operate on our networks, so that all of our applications on the unclass side of the Air Force for now, and we’re going to try and take this to the classified side, now, all these applications can get basic web analytics and user feedback.

That sounds very simple. Like, if you were to go start any business on the public internet, those are probably two of the first things you would attach to your website, right? It’s just a feedback link and web analytics, but our stuff, we can’t, we have not been able to do that.

And that’s a problem in the government, by the way. When they did the analysis of what went wrong with healthcare.gov, one of the findings was that they didn’t have web analytics. And I know why they didn’t have web analytics. It’s actually kind of hard to get that in a government site. So we solved that problem. Now that’s being rolled out. There’s several other things we’re doing at the individual application level. But one of the things that I think is unique that we’re doing is we needed a way to start managing the portfolio overall.

Influence Without Authority

Jesse: What you’re describing sounds like a tremendous effort of orchestration cross-functionally, and I find it especially interesting that you were able to achieve this in a role that is expressly designed to limit the amount of direct control you have over what actually happens, right? You’re expected to wield a lot of influence, but you have very little, it sounds like, direct control or direct authority to actually tell people what to do. Is that true?

Colt: That is, that is true.

Jesse: How do you get a job this complex done when you have those constraints?

Colt: So a couple of ways. I put so much of a focus on metrics measurement because once you have that, then you can provide, what we call in DoD, situational awareness up the chain of command and get buy in. So in other words, if you can show that performance and user experience is bad with certain bases, certain applications, certain missions, and the data is clear and presented clearly, then you can, get buy in, because that’s information that has to be dealt with. That’s, I think, the most powerful thing that you have. I think that’s why I’m such an advocate of a data driven approach. That’s number one.

And number two is, you gotta have, you gotta have air cover, because it takes a while to put in place the tools to get that kind of data in quote unquote situational awareness.

So you gotta have air cover to be able to get you through to the point where the data is out there and it’s clear and it’s compelling and actionable.

And then the third thing I would say is, you have to get a little bit… lucky is not the right word, but you have to get a little bit lucky in the sense that once you start having good situational awareness across a broad portfolio of technology, where you get a lot of data and you can see what’s really going on, then what you need is you need leverage points where sort of the, the 80/20 rule comes into effect or the 90/10 rule, and you can go and say, okay, if we solve these 1, 2, 3, 4, 5 problems, even if we solve them 75%, we’re going to get a 20, 30, 40, 50 percent improvement in our metrics. And it’s identifying those points of maximum leverage across the enterprise where you get the biggest result. And if you’ve got enough data visibility, enough visibility across the whole environment, you can start to identify those points. Fundamentally, that’s it.

So that’s how we’ve been able to move the needle.

Peter: Much of what you’ve been explaining, feels a lot like lifting a floor, right? There was a broadly unsatisfactory level of quality across the systems and your initial orientation was to really like dig in, get that data, really understand kind of where the breaks in the chain are in order to raise the floor.

And I’m wondering, because you were, you’ve been there four-ish years, was there a point at which you started to also get to look at raising the ceiling, to think about innovation, to think about net new, or has it really been this very kind of practical, tactical, we just got to get these things working, and then that in and of itself is a win.

Colt: So I probably spend at least 50 percent of my time on the “get it working.” And that’s a huge win, by the way, if you look at what airmen complain about most. That’s it. At least in IT. However, to answer your question, yes. So when we conceived my role initially, one of the ideas was there are several, I call them the mega user journeys that apply to almost everybody in the Air Force and Space Force that are not very well automated. They might be partially automated, but they’re automated by multiple different systems. There’s a massive amount of paper involved. The churn involved is insane. And so there were those that I wanted to take on.

Peter: You mentioned getting kind of cover from your leadership, but how are decisions made? How do you get on everybody’s backlog or whatever, across this extremely complex organization in a coordinated fashion, such that you can make the progress that you know you need to make. Like, is that just you in a lot of meetings having a lot of conversations? Is there some communication, operational kind of a set of people and practices that can weave this together? Like it just feels daunting.

Colt: Well, I mean, it probably is a little bit, but I was very fortunate that I had I’ve had some good bosses and it’s important to have a good boss. I’ve had some good bosses that I was very in sync and they were very in sync with what we wanted to do and what was possible.

And so I had air cover when I needed it. But a lot of this is just a tremendous amount of communication. So one of the things that I use to our advantage, when you’re in my role, you do a lot of public speaking and conferences. And so two or three times a month, I’m giving some kind of a public talk and then probably a couple times a week for the last four years, I’m giving some sort of a large group presentation inside the Air Force, going through, okay, on improving performance of enterprise IT. Here’s all the things we’re doing. Here’s the vision. Here’s strategy. Here’s the measurements. Doing demos. Whenever we stood up the tools, people have to believe this stuff is real, right? And they have to experience the benefit.

And so I do a lot of demos, demos of our digital experience, monitoring demos of our survey platform, etcetera, etcetera. And then on the software side, same thing. A lot of demos. The job is fundamentally a communication job, making people aware, showing people it’s real, getting people to buy in, coaching people, teaching people, but doing it at scale. And you know, if there’s one thing the DoD does really, really well, we do a lot of things well, actually, but one of the things I do think we do well is communications at scale. We do well.

Jesse: So I’ve never worked with the military. What I imagine of that environment is one in which the default answer is going to be no. Unless you can make a pretty strong case for yes, because of the need for things to absolutely work without question.

Colt: You’re right.

Jesse: And I imagine that that creates a certain amount of institutional risk aversion. It creates a certain amount of skepticism toward new ideas, especially… Experiments that we’ve never tried before. And you came into this role having to advocate for all of those things. And I’m curious about, aside from having really awesome metrics to back you up, how did you make that case? How did you convince people to take on the risk of changing their processes, changing their approaches, looking at things in a new way?

Make Change by Connecting to their Past

Colt: Great question. And metrics were a big part of that. But there’s a few tricks. You gotta be able to learn and speak the organization’s language and speak to what matters to them. That’s important.

And by the way, the notion that you’re talking about, in our environment, they call it the frozen middle.

Jesse: Mm.

Colt: When it comes to user experience and performance, we actually have, I would say, senior leaders at the Department of the Air Force , we’re talking in the what we call the glass doors level. So I think the top six or so leaders in the Air Force, the second half, the chief, the undersecretary, the vice chief, and the chief master sergeant at that level, they actually use the term user experience which is encouraging. They know what it means and they know what it means to them. They’re not experts, obviously, but at that level they really do get it. And they support it and they want to know how to make it better.

And then at the sort of the airman level, they totally get it right. Technology is like air to them. They breathe it. It’s part of their lives. we hire a tremendous number of young people and they just breathe technology. So what everybody talks about is the frozen middle. It’s all these folks that want to say no because they don’t really want to take a risk.

So I use Lieutenant Colonel Fitts story a lot.

I do a million conference presentations and internal, I close practically every presentation with Fitts because it shows… The point is that human-computer interaction and user experience and human factors engineering, these things are all within the DNA of the Department of the Air Force, right?

And you guys know the history here. Frankly, you probably know it better than I do. But in case some of the listeners don’t, the story at a most basic level is that we were losing hundreds of B-17s in World War II in Europe. You can read Bomber Mafia by Malcolm Gladwell, good book about it. And, but a lot of these planes, in fact, really thousands we were losing, and but hundreds of these planes flew back into Britain and the pilots would just make a lot of mistakes and the Air Force and our infinite wisdom basically said, Hey, we’re getting crappy pilots.

And so they decided, Hey let’s go do a study on this? Right. ‘Cause we’re the government–studies, we do that. And so they hired a, psychologist who had been Lieutenant Colonel in the Army Air Corps.

Gone back, got a PhD, comes back as a psychologist and he starts digging into this problem with the intent of helping hire better pilots or choose better pilots, select better pilots. And starts looking at the records of the plane crashes and realizes that there’s patterns in the data. Same types of things were happening again and again and again, right?

Pilots that would fly a mission over Europe, come back, land in Britain, and they would open the bomb bay doors instead of put down the landing gear. Things like that. And so he and a pilot and others started looking at the planes and began to realize that what we had was not crappy pilots.

It was a badly designed plane. Certain controls were too close together. They looked exactly the same. They worked exactly the same. You know, if you put yourself in the position of these pilots and crews, I mean, good God, I mean, they just flown for hours over Europe.

There’s probably smoke in the cockpit, maybe one of their buddies is bleeding in the back or worse, and I mean, the stress that it must be is unimaginable, and then add into that that you flip to a console and the controls are all the same, it’s a miracle anybody got back at all, and so they began redesigning the cockpit of the plane, focusing on things like shape coding of the controls, and it’s interesting, those techniques are used to this day, anytime you tell this story in our environment, there’s almost always an Air Force pilot or a former Air Force pilot on the line and they get in the chat on our team’s call. And they’re like, oh, yeah, that explains why , this control is shaped totally different than this and this one is like a gear and this one is shaped like a flap and I’m like, okay, it turns out those things are still done to this day. So, but the point of telling that story is that anybody in our environment can understand it. They all respect it because there’s a lot of respect for that kind of history in our culture.

And it conveys a few things. It conveys being data driven. It conveys solving the problem. It conveys the connectedness to the mission. It conveys frankly, it just conveys that this is within the DNA of our culture and organization already. And if we can tap into it in order to solve problems like that, we can tap into it in order to make the user experience better for our weapon systems today. In World War II, they were a B-17. Well, today, they’re still weapon systems. They’re different, right? There’s a huge IT component to our weapon systems today. It’s not just all about super expensive hardware that circles the globe or whatever. There’s a lot of software. So that’s how we do it. That’s the technique to cut through.

Jesse: I love that story. You know, it’s interesting because I asked you, what did you do to talk people into doing something new? And what I got from your answer was that your approach was basically to convince them that it’s not new at all, right? That it’s actually already a part of the culture that they’re a part of.

And that this user-centered thinking is really just an evolution of what’s already in the organization.

Colt: Totally agree. Think about it. I mean, what are the odds that me, an outsider, are going to come in and change the culture of the Department of Defense? Not going to happen.

So, you got to be realistic about that and then… Use the culture for what it’s really good at. And this is something that our culture has been good at in the past and can be going forward.

Bringing a Service Mindset

Peter: I actually want to build on that, because you wrote a post on LinkedIn and one of the words that I noticed in your post was, “if any of my friends from the commercial world are interested in serving,” and serving is not how we typically talk about the kind of work we do. And I’m wondering, what it has meant to be in an environment where to serve and serving is the value that everybody, to the highest level, is bringing to the work and how that’s affected how you’ve approached it, or evolved and changed how you’ve approached it.

Colt: First of all, this doesn’t apply just to DOD, right? I mean, what we’re talking about here really is government service.

it’s service to your country or your state or your county or city, it’s government service. I think military work in particular can be a bit of a family business, right? My grandfather was in the Navy. My dad was in the Navy.

I was not, but I do this. We need people who are willing to go serve and it doesn’t have to be in the military. It’s a million ways to do it. I have the highest respect for Jennifer Pahlka, who just put out a book called Recoding America and the work that people do in that on the civilian side of the government, and it’s all vital and it’s all technology that has to work, frankly, for the institutions of our country just to operate.

So I think it’s a important thing, an honorable thing, and I encourage it for everybody. We have some countries like, I believe, Switzerland and Israel basically require a certain amount of military service. I don’t think it has to be military but I think it’s a good experience for everybody to do some of this, and I think it makes you more connected to , frankly, everyone in our country and around the world, I think whenever you spend some amount of time providing service. So something I feel strongly about, and I would highly encourage anyone in the commercial sector, as I was, if you’re interested in doing something like this, if you’re a UX designer, software developer, whatever, and you’re interested, reach out to me anytime.

Peter: I’m wondering how your posture in leading this kind of work shifted as you approached it with this serving mindset versus a more consulting approach.

Colt: Yeah, really good question. From my perspective, you have to take sort of a balance. There was a two star general that I respect tremendously, General Schmidt, who was in acquisition and in a week or two after I started in the job he kind of asked me, okay, well, how are you going to approach this?

And of course, I didn’t have much of anything at that point. And the first thing out of my mouth was, well, it’s going to take a lot of humility. And I think that’s one, that’s a big part. So when you’re going to be a change agent and you’re going to try and really move the bar in an organization this big, and you gotta have a certain amount of humility because there is so much you have to learn and you have to just recognize that at the same time, you also come in with a completely different perspective and you’ve got to be confident in that. And do your homework and rely on a lot of experts. So as I was preparing to take this job. I leaned on friends that were former CIOs, current CIOs and kind of laid out the problem to them and they all kind of pointed and here’s sort of what your strategy is going to have to be. This is the only way that will work.

So I would say humility plus confidence, not arrogance, but a certain amount of humility plus confidence, making sure you’ve done your homework and have in your back pocket a plan that has been proven to scale elsewhere and will apply.

Jesse: I’m curious about the future for you, and where you see all of this going, and what role do you see for yourself going forward?

Colt: Sure. Let me hit four things. So Air Force, DoD, federal government, and then me.

So Air Force. We have a new CIO, Venus Goodwine, and she replaced our last CIO, Lauren Knausenberger, just last month. She is committed to building on and expanding what I have been doing. And I’m not going to make any announcements beyond that, but what we’re doing will go forward in the Air Force.

DoD, I think you’re going to see some exciting things when it comes to UX and performance software and I T. The CIO at the Department of Defense is named John Sherman, and he’s absolutely committed to this. They have already stood up something called the Office Performance Management. And they’re going to be doing some of the similar things. And frankly, you’re also seeing the same type of playbook we’ve executed here at the Air Force. You’re now seeing the Navy pick that up and run with it. And I think you’re going to see it elsewhere in DoD and probably increasingly coordinated DoD wide by the overall CIO for the Department of Defense.

So this is going to become more than just an Air Force thing. It actually already has.

And then federal government. There’s a tremendous amount of activity in UX and CX across the federal government. And I’m not really qualified to talk about it at length. But I am part of those networks, at least in terms of all the email distributions and some of the conferences and things.

It’s super exciting. You’ve probably heard of the 21st century IDEA Act and there’s various federal policies and policies from the White House and OMB to raise the bar for citizen experience across the federal government. They’re also creating a lot of resources for people like me and organizations like this one , tools, resources.

So the survey tool that we’re using just for feedback on applications, it’s called Touchpoints. That’s a GSA tool that we can use even in our environment behind our firewall, which is very interesting. So they’re, they’re doing some amazing things.

And then last, me, personally. So I have a rough plan. And part of it’s going to be advising regarding user experience within DoD. I won’t go any more to it than that. So I’m going to continue to be engaged within DoD. And then I was with a small company before and I’m probably going back to a small company. And in fact, I’ve got a idea and a little bit of a plan for a startup.

So, I may be in the startup world here in the near future.

Jesse: Very exciting Colt. Thank you so much. This has been great.

Peter: Yes, Thank you.

This has been really eye-opening.

Jesse: Colt, where can people find you on the Internet if they want to follow up with you on this conversation?

Colt: Honestly, best way is LinkedIn. I’m not a huge Twitter user or X user. LinkedIn’s pretty reliable.

Jesse: Fantastic.

Peter: Excellent.

Jesse: Thank you so much.

Colt: Thank you.

Jesse: For more FindingOurWay, visit FindingOurWay.design for past episodes and transcripts. For more about your hosts, visit our websites, PeterMerholz.com and JesseJamesGarrett.com If you like what we do here, give us a shout out on social media, like and subscribe on your favorite podcast services, or drop us a comment at FindingOurWay.design Thanks for everything you do for others. And thanks so much for listening.