Measuring UX: Challenges and Strategies

Transcript

Jesse: I’m Jesse James Garrett,

Peter: and I’m Peter Merholz.

Jesse: And we’re finding our way,

Peter: Navigating the opportunities

Jesse: and challenges

Peter: of design and design leadership,

Jesse: Welcome to the next phase. Joining us today to talk about what’s next for design is Amy Lokey, Chief Experience Officer for the enterprise software platform, ServiceNow. We’ll be talking about building a team that unifies product experience with customer experience, defining experience metrics that actually matter, investing in her own growth as a leader, and the real implications of AI for digital product design.

Peter: Amy, thank you so much for joining us.

Amy: You’re so welcome. I’m happy to be here.

What does a Chief Experience Officer do?

Peter: So, in the introduction that Jesse will have recorded before people hear us talk, he will mention that you are a CXO, Chief Experience Officer at ServiceNow. What is a Chief Experience Officer?

Amy: Well, Chief Experience Officer, I’ll say first is importantly not a CEO, because you can only have one of those. So that’s why I’m A CXO. But I run product experience and also customer experience at ServiceNow. My wheelhouse, my background, is primarily in product user experience design. My team includes all of the functions that you would expect in user experience design team.

So we have a large research organization. We have a large design organization. We have design operations and various operational functions that keep us kind of working together smoothly and things running. And then additionally, I have a large content team as well, too, that produces content for our technical documentation, best practices content, and is really the engine behind a lot of the content that helps our customers be successful with our products. And as part of that, we’re also responsible for a lot of the digital experiences that our customers use to be successful.

So everything from… We have a learning experience where our customers and developers can get credentialed on ServiceNow. We have our customer support site. We have a product called Impact where our customers get kind of white glove customer support and work with squads of people that help them get up and running. So there’s a number of digital experiences. Those are just a few that my team also is responsible for. And so that’s the customer experience side of it.

And then our product suite includes customer support software. So, there’s an intersection of our own product, is what we use for those experiences, so a lot of what we build to support our customers is the same thing that we’re building as a product that we sell to customers too. So there’s kind of an interplay in how all of that product experience design works and how we’re using it.

Peter: And just to kind of establish some boundaries here, when you’re talking about customer experience, what is your relationship to marketing and kind of that front end of the funnel? And do you do any design or research work on that side? And then on the other side, more kind of typical customer service and even maybe even customer success. It sounds like those are outside your purview…

Amy: Those are outside of my… yeah, yeah. From an organizational standpoint, so, we have a great CMO Colin, he leads our marketing organization. They do have a couple creative teams within that organization that work on various parts and pieces. And so our corporate website, for example, his team drives that.

The digital experience, that we work really closely to bring those together in one unified navigation. So we want, from a brand and from a user experience design standpoint, we want it to be really seamless. So whether you’re looking on what might the corporate site, meaning, like, you’re looking at our products, you’re looking at our marketing communications, you’re evaluating the company, or you move into, Hey, I need to deploy this new thing that I got and I need some detailed information on that.

You move more into the customer experience side of it. That’s my team. But we want those, you know, boundaries to be invisible to our customers. So we do have kind of an architecture that we work on from an information architecture standpoint, even like a universal login standpoint. So that’s, hopefully, not visible to customers, even though we have two different teams working on those parts.

Peter: Sometimes customer experience means customer service. Do you have a relationship there?

Amy: Absolutely, I don’t run customer service. So there’s another leader that runs customer service in terms of our support organization. But my team does work on the digital experience. So we design that entry point, right? And that’s, again, using our own product and then building it out to fit the needs of our particular customer service experience.

So the digital experience we are responsible for, we support that. But the team of people that are behind that support system, like they’re managed by a different person. Yeah.

Jesse: Within your team, you have an unusually broad mandate. In terms of bringing together both product experience and customer experience capabilities, and content as well. And I find myself curious about the organizational impetus to unify these functions. What’s the value of having a unified team with such a broad mandate that in most organizations is kept separate?

Amy: Yeah. it’s not all organizations that are separate. I will say, like reflecting back at my time at Google, when I led user experience for G Suite, I was also responsible for the technical documentation. So, I think there are cases where you might have one org leader over both. The reason why I think it makes sense here is, the documentation of, like, how the product works is in many ways just this foundational piece that all of our other marketing and other content pieces are built from. We’re kind of the source of truth, and the people who write that content really sit side by side with the designers and the engineers building the products.

So they are very much integrated into the product development process. And I think that’s an important part of why this all works. So, you know, my peer groups in my organization are, you know, product management and engineering, just to like very general terms, right? And then I have kind of the everything else bucket of user experience plus product, content and research and so on.

But all the functions within my team are very, very close and tight with the product development process. So they’re working with product management, engineering. We are part of that release cycle. We’re part of, like, QE processes. So I think that’s why it all makes sense. It’s just fundamentally, I’m part of an R and D org and all of the experts within my team are part of that process that deploys software and then builds the things on top of it that help our customers be successful with that software.

But we have to have that subject matter expertise of really understanding what it was designed to do and what it was built to do, and how it really, really works to be that source of truth. So we’re, you know, the customers that come to our documentation, they see it as very objective. It’s intentionally not positioned in a marketing type of framing.

It is just factual because then it’s very, very trustworthy and seen as like this is the absolute source of truth of how something works.

Peter: On your LinkedIn profile, when you define yourself as a CXO, you mentioned design and research, product content, design operations, but then there’s this phrase, information strategy. What is that?

Amy: It’s how we deliver the right information to our customers. And so we are doing a lot of work to think about the best way to deliver vast amounts of information and what that strategy is. So it’s very much information architecture. We have a team internally, we call it Lexicon, which is just how we actually name our products, how they’re actually affiliated into our customer support experience, right?

So if you file a ticket about our software, you’re gonna say what product you’re using, what functionality you’re using, that all has a data architecture in the backend that helps us tag it to particular products. So then we can kind of document the throughput of information back to the product team.

So that information strategy and information architecture, it’s not only how we externally organize the content, it’s also how we internally tag it and map the data basically. So we have that throughput of information.

The Holy Grail of Information Architecture

Peter: This is like a holy grail. Well, so Jesse and I came up as information architects, and I think one of the challenges that I’ve seen UX teams face is, when they want to get to data models and content models, who owns those relationships with engineering, a lack of broad horizontal view of a company’s information standards and platforms and, and practices, and so it gets in the way when you’re trying to create, say, a unified navigation across a multi-product suite when every team has kind of done things their own way. And I guess usually this is solved by enterprise architecture or something, but it feels like your team has a bigger voice in this conversation, I guess recognizing there’s a user experience component to it, not just a technology platform kind of component.

Amy: Completely. And, the team that does this work did actually have roots in engineering. So it started out as part of our engineering team. Hence, the affiliation to the data modeling and how we track that throughput of, you know, someone logs an issue in a customer support case, what part of the product is it actually affiliated to, and then can we even tie that to an epic and a record to show that we fixed the thing in the product, right.

So, because a lot of that information architecture team had roots in engineering, it moved to my organization about a year or so ago. It did move because there is an implication on the user experience and there’s this relationship to our content and product content and so on. There was a natural time and place to move them to my team, but we have this great history and this great foundation of the data architecture underlying it.

And then the other piece is just… ServiceNow is built as one unified platform with one data model. And so that’s just in our DNA here of like how we do things. So we continue to build on that. We have an internal tool like I just mentioned, where we can track our research insights and connect them into design records that connect into the epics and stories.

And like I said, we could even connect that to user experience, customer service issues that we see. So the fact that we can have all of that data in one system of record is pretty powerful because we’re getting to a place now where we can track our user insights, we can tie them to design artifacts that show that we’re fixing the thing.

We can then attach that to a PRD and then the, you know, epics and stories and we can show this connective tissue of, Hey, when we release the thing that solved that user insight, we actually saw a decrease in customer service issues related to that particular thing, which is pretty powerful to be able to draw that story together using data and show that that investment in user experience actually lowered the cost of customer support on the other end.

Jesse: So with all of this interconnectivity between your group and its work and the work of all of these adjacent groups that you’re intertwined with, it actually gets me wondering about the discussions that go on at the top level of the organization and you know, you are a chief experience officer.

A lot of organizations don’t even have a C-level person with your purview, and I’m curious how that changes things for you to be in a C-level role, speaking to experience issues.

Being in the C-Suite

Amy: Yeah, that’s a great question. I mean, I have to change altitudes quite a bit. I mean, I still, like I said, my roots are of being a product designer, so there’s nothing I love more than being in a design review, looking at the you know, product taking shape, giving feedback, working with the team to ideate on things.

Like, for me, that gives me energy. But I do spend a lot of time thinking about our broader experience strategy in the ecosystem of our product, right? Which is, we have an enterprise software product. We design out of the box software products, right, that we sell to customers, but they’re on a very configurable platform.

So we also have to make sure that we’re continually training our customers and our implementation partners on how to use our best practices to design and build the right experiences on top of our product.

So it’s sometimes, I joke, it’s like the movie Inception where you don’t know which level of the dream you’re in because there’s kind of the root of it, which are our components, and then we’ve got our out of the box products. And then on top of that, our customers might configure and extend them. They might theme them. And we have a partner ecosystem as well too. So the role that I play now is much more externally facing, much more customer facing and much more involved in our partner ecosystem and the enablement focus.

It’s one thing to have a great internal product user experience team. A lot of my journey here has been growing and developing and maturing that. Now I need to mature the user experience practices outside of the walls of this company to make sure that our customers are successful with the product and have all the best tools, resources, and talent to build on our platform in a way that delivers the potential that it has to deliver a great user experience. So a lot of the work and the strategy that I do is focused on that outbound work to enable the ecosystem.

And then secondly, of course, I spend probably the largest majority of my time at the executive level on our product and business strategy, right? And the focus that we have on experience there is all around the transformation with AI, generative AI, agent AI, and so on. And that’s gonna be a massive shift in how people interact with technology.

So my team has to be ahead of that curve and also helping the executives understand where that’s going. So those are the two pieces that I would say are outside and beyond maybe the typical UX leaders remit. But that’s not to say I don’t love doing that core part of my job, which is helping shape the product design.

Peter: You just said the trigger word, or trigger letters, of AI, and it sounds like so much of what we’re seeing when it comes to AI in user experiences is some kind of, not quite shovelware, but it’s just being kind of spaghettiware. They’re just throwing AI at you in hopes that they’ll find some application that sticks. And given what you’ve been saying and the connectivity that you referred to, right, and there’s like a, detailed understanding of how user experience connects with these other kind of aspects of the business and, how value is realized in the business, I’m assuming that you are approaching AI with a different kind of consideration, and I’m wondering, what role is your team playing in contributing to that product strategy around AI, distinct from, or in partnership with product management, engineering, I’m assuming the typical function, sales, since you’re an enterprise software firm, I’m sure they’re getting involved.

Like, what is the chief experience officer’s team bringing to that conversation to ensure that it’s not spaghettiware AI, but, really things that your end users will find value in?

Designing with, and for, AI

Amy: I’d say there’s been multiple phases of this and it keeps evolving. But I think one of the most interesting phases was at the very beginning, right? So generative AI hit the market. Everyone’s trying to figure out what to do with it. And our research team played a really key role in looking at where we might find the most value and deliver the most value, and using a System Usability Score kind of usability testing methodology.

And so, you know, like everybody else, we’re wondering, well, where is the most valuable place to insert this technology? Right? And for what, what user, what persona, and in what parts of our software? And then obviously you’re looking at the business value that that might deliver and how you might monetize it, but also how you communicate that value to the customer. And largely the benefit of this technology is all around productivity and time savings, right? So how do you save time? And fortunately for us, our software is very much productivity software that’s typically used at scale by teams of people doing pretty predictable and often repetitive tasks, right?

And so our roots are all in like workflow and process automation and so on, where you can say, okay, given any enterprise or business, if they have a particular workflow, we can start to digitize that. We can start to automate it where possible. You can start to enable humans to be able to do more versus doing whatever that activity was before, right?

So instead of copy-pasting data from one system to another, we can automate that, right? So they can do more intelligent work. So AI was like a perfect fit to what we already did, but we had to figure out like, where are we gonna invest? Because you can’t just put it everywhere. There’s a cost to that. And so we used research and data and analytics to say, okay, what kinds of activities are most repetitive and happening in at volume and at scale, you know, so they happen over and over again.

So they’re fairly predictable. They happen at a lot of our customers and they’re happening at scale where many, many people, our hands have to touch that thing. And so it was just basically a simple equation where we could say, if we put AI in this particular place, then how much time does it save? So we used a usability assessment to say, how much time did it previously take someone to accomplish this?

So say that’s summarizing a case or writing resolution notes when something’s been closed. If that previously took them an hour and you can summarize it in five seconds with generative AI and they can edit it and complete that task. Now in a number of minutes, you can multiply that time savings across how many people, how many times a day at what scale in our customers, and you can start to extrapolate true value. And that even helped us figure out, How do we price this? So, so we were really formative at the very beginning, just trying to figure out where would it have the most value, how do we articulate it to customers and how do we price it? And that was very much based in those roots of just a before and after picture of time on task.

That was two years ago now, but it did help us invest in the right places. Now, you know, fast forward, we’re more strategic, too, in how we measure the quality of the output and the value before we take something to market. And that’s been a learning process as well, too, because as you’re working with LLMs and they have degrees of unpredictability and variability, we’ve had to work really closely with our QE team to come up with both automated and also observational ways of testing the value of what’s delivered.

And that’s been an interesting challenge. And, have to do with the pace that’s needed as well too. Generally working with internal simulated datasets before it gets into the hands of your customers. So that’s still a work in progress and something that we’re evolving, but we’re getting to a place where we have a pretty clear metric-based evaluation, too, of assessing the quality of the product before it goes out in market.

But that’s been a multi-pronged effort as well in terms of getting like data to work with and all that.

Jesse: What do you think designers and design leaders often miss about AI and how it fits into everything these days?

Amy: I think it just depends. I’d say like in the last year or two I’ve been able to participate in various like panels and discussions and summits about AI. And I think I’ve been lucky to be working so firsthand with it all along the way. And so I feel grateful that I am part of these meetings, whether they’re at the executive level or the team level to really understand from like our research scientists how things are working and evolving and our engineering team, how things are working and what we’re learning.

I think the most important part for any designer now is to truly understand how the prompting work, how orchestration works, what the variability is, and to get ahead of the quality in a way that’s harder to do than ever before. Because I think what’s really tricky with AI is to get the quality right, and to really understand what the end user experience is gonna be before you ship it.

And that I think is the new science and expertise that we’re all kinda still figuring out as we go along, because it evolves so quickly. So you’re designing for hypotheticals and you can’t predict the hypotheticals necessarily, so you have to find quick ways to assess and, modify and change.

Wielding UX Metrics

Peter: When you mention getting the quality right, and the quality of experience, and, earlier you’ve referred to some UX metrics. You’ve referred to some usability assessments that you all conducted. I’m actually curious of the story of UX metrics at ServiceNow.

Maybe it’s at least since you’ve been there and, when you joined, was there some UX metrics in place that you were able to kind of start with and then build? Or is this something that you brought in, in terms of how UX metrics are used? Not just, it sounds like within your team, but how your team uses these metrics to communicate outside of UX or CX and like, what that journey of metrics… It’s literally probably the most common question I get from design leaders is how to measure, how to value, how to define quality. And it feels like you’ve got your hands around this a bit more than perhaps most do, so I’d be curious what that story is.

Amy:

Yeah, really proud of that. Thank you. And huge credit to the research team and their leadership on this. But I would say when I joined, our, our research culture at that time was much more on the foundational research side, which is great and super valuable. But at that point in ServiceNow’s growth, we were expanding into a lot of different products.

So we’d started as kind of IT service delivery, and we were expanding into customer service. We were expanding into HR, we were expanding to all these different verticals throughout the business.

So the research team, and rightfully so at that time, was very focused on foundational research. What product should we develop? What would be the right product market fit. Almost a little bit more like market research than product user experience research. So when I joined, I had been at Google when a similar methodology had been formed. I think it started within the YouTube team, and then kind of grew across Google.

And now the name that they called is kind of escaping me, but it was based on something like called Toothbrush Journeys, which are like, you brush your teeth every day. So look at the most, oh, CUJs, Critical User Journeys is the acronym Google, I believe. So you look at the most critical user journeys that people take, and usually there’s data that you can look at in terms of what are people, where do they start, where do they end, what are they trying to do?

And I remember when YouTube first started forming this approach, they were just looking at uploading a video, what does it take, right? And they assumed, of course people are using YouTube, they must be having no problems uploading videos. There’s actually all of this drop off happening where people could not figure out how to upload a video.

And so that was just like a critical user journey for the product to be successful. So the methodology is simple. You just establish what those 10 to 20 activities are and you measure the usability of them. And that is based on that SUS score, right? Which is like time on task, I think, you know, level of correctness, in terms of do you get through it appropriately, do you make mistakes, do things break along the way, or do you do the task correctly? There’s some qualitative in that as well too. How do you feel about it? And so you’re looking at both like the success rate as well as the time on task and then how, you know, do people feel like it was a good experience?

And so you can kind of triangulate all that together. So we based what we call internally here now, UX quality, on that same approach, right? So determine what the most important things are that you’re hoping people can accomplish with the product. What are those journeys? And then you measure how effective they are and how long it takes and then qualitative, how do they feel about it.

So we’ve kind of expanded that system into a benchmarking study that we try to run at least semi-annually on a product. We do semi-annual big family releases. So ideally at least twice a year we’re kind of looking at, you know, a new version of the software and reassessing based on that same evaluation.

And we’ve had products where we’ve been doing this now, I’d say, over about six or seven releases, and we’ve seen tremendous improvements. The score ends up netting out to a percentage score on a hundred percent scale. And I believe in the industry, 80% and above is typically a considered a consumer grade usability score, where people can generally get in there, they can use a product. They don’t need tutorials. They don’t need documentation. It’s approachable, and it’s usable and simple, and they can be successful with it. So in enterprise software, the scores are typically in the fifties and sixties, right? Like typically much harder to use, more complicated, not as easy. So in the products that we’ve been running this methodology on, and addressing the feedback, ’cause it delivers really clear actionable feedback, we’ve had products that have moved from the 50 percentile up to some of our products are scoring in the 95th percentile, which is incredible for enterprise software that’s quite complicated.

So it’s just been a great tool in the toolbox and we’re actually now using these in our executive product reviews. So we do quarterly product reviews and now we’re bringing what we’re calling overall this UX health scorecard to the mix that includes this UX quality benchmarking where we can investigate, you know, is there a usability problem here at the product or is that fine, and actually there’s an adoption problem, where there’s something else working here that we could apply over there. So it’s been great to see that continue to get momentum here and continue to add value in how we look at our products objectively and make sure that we’re fundamentally delivering a product that people can use and enjoy using and they get value from.

Jesse: What do you think are some of the larger cultural factors that support that work in being successful? Because, like, I hear from leaders that they all say, well, our executive leadership absolutely wants to be research driven, data driven until they get data that they don’t like. And then, the data goes out the window, the process goes out the window, and executive fiat takes over. And I’m curious about the decision making environment that enables this more, you know, rational decision making.

When UX is aligned with corporate values

Amy: I think what helps is this kind of data is what our customers value as well, too, right? So again, like the fundamental value of our software is to deliver on business and end user productivity, right? So the fact that we have a methodology that shows very concretely improvements in user productivity and using the software, there’s no argument there, right, because that’s what we’re committed to do for our customers.

And we can also show our customers like, Hey, look at the improvements we’re making. You know, your users are gonna get through their work x amount of time faster because now all these hurdles are taken outta the way, or we can generate these things much faster for them.

So I think it’s just so well aligned with the value of our product, how we sell it, how we talk about it, that there’s never been an argument against it. There’s just been a hunger for more. I think in the consumer world it might be different, right? Like I’ve worked in consumer products quite a bit as well too, and in those cases, you’re not necessarily creating product that helps people be more productive. A lot of times you’re creating product where you want people to spend more time.

Jesse: Right,

Peter: right, right. “Engagement.”

Amy: And so very, very different very different motivations, very different design measurements, right? And so usability, it’s probably still important, especially when you’re looking at, can they sign up for it? Can I get them registered? Can they subscribe? Can they pay, put in their credit card information or whatever it is.

So, however that business model thrives, typically those usability scores will be very important. But in other cases, I mean, YouTube went for a really long time where people had a really horrible time uploading videos. It didn’t matter ’cause some people were figuring it out some way or another, you know, they were getting their videos up there.

And the product was successful, but it probably became a lot more successful when you kind of streamline that process. So I think it just depends, like, how the product business works, and can you align metrics that are both in good service of your end user, as well as that are helping demonstrate how you can deliver more business value.

Peter: It sounds like your team has generated some of these various UX metrics, but they’re now being placed on product team dashboards. And I’m wondering, where accountability lies now, right? is your team primarily held accountable for this, or is there joint accountability?

And then, how is accountability handled, particularly for your CX team? I’m curious if you are somehow expected to deliver a distinct kind of value within ServiceNow, or if it’s just broadly recognized that CX is valuable, like these other functions, and your contribution is not isolated necessarily.

Amy: Yeah. I mean, I think in terms of accountability, it’s definitely a shared accountability. So when we present these metrics, even though my team is helping produce the studies and create the dashboard and create the measurements and all that, we are making a hundred percent sure that we’re not blindsiding anyone with that, right?

Like, we don’t wanna go into an executive review and be like, ah, gotcha. See this thing’s in the red. What are you gonna do, right? so we’re very much in lockstep with our product and engineering leadership team. So everyone understands how the study was conducted. They understand the data, they trust it, but they also feel aligned on what they’re doing about it, right?

And so that accountability is definitely shared. It’s not just on us to fix it. And there’s also times where you might say, Hey, it looks like this is actually in a pretty good state, so as a cross-functional leadership team, maybe we need to invest in something else, right? Like, so I think it’s also how you’re making your investment decisions in terms of what needs the most right now.

And in some cases you might have a product that’s very new, and you really wanna get it adopted, you wanna get it out and, you know, getting traction. Well then probably a big investment in UX is really good. You might have another product that’s actually quite mature and successful, maybe there’s another area of the business that needs that investment.

So I think it helps us make like more thoughtful decisions across the board about what to invest in.

Peter: Is there accountability specific to your team or is accountability shared…?

Amy: It’s shared. Yeah, it’s shared across the product team. Yeah, absolutely.

How to know to grow

Peter: How has your team known, for example, how to grow, you know, you have hundreds, if not close to a thousand people in your organization.

That’s expensive. What is the investment calculus in growing your team as opposed to hiring another product manager, hiring another engineer? ‘Cause clearly your team’s delivering value, but what is that conversation in terms of, like, the money, the funding that you get to grow your team, versus maybe other places where that money could be spent?

Amy: Yeah. I mean, every company has different funding models. It’s a cross-functional conversation, absolutely. But ultimately the way that ServiceNow works is there’s a GM that at the end of the day makes those calls, right? So, they will get investment, they can choose to invest in sales, they could invest in engineering, they could invest in QE, they could invest in product content, there’s many competing areas that you can invest in.

We use a ratio based model to just have some form of a guideline of what a healthy team looks like. And then cross-functional leadership gets together and we have interlock meetings where we’re looking at what the investment plan is, making sure that we can deliver on that investment plan. Making sure even the quarterization looks good, right?

Like, are you hiring the right people in the right order? You don’t necessarily wanna hire engineering first and research last, you know, so things like that. But it’s a cross-functional conversation, which is really healthy. And, I think it’s rare to find that kind of operational health in terms of how things are funded.

But like I mentioned, there may be times as a leadership team where like, the number one thing we need is actually like translation services for this particular product, or it’s, we need more QE for this particular product. So I think we do a good job of putting on objective hats around, like, what will truly make this business successful, and making the right investment decisions. Not just fighting for, like, we need more design.

You know, I don’t approach it in that, way, but ServiceNow has grown tremendously over the past few years. So the team has grown tremendously during that time. I do think it’s an interesting advent with AI coming into the mix a lot more.

Like, I think we’re all looking at how do we adopt AI to increase efficiency and potentially decrease the need to continue to expand at the same rates, you know? So, I think that’ll be an interesting shift as well too, because we just wanna make sure that we’re enabling our teams that we have to be really, really successful. And continuing to add at the same rate may, like, probably is diminishing returns at a certain point once you get to a certain scale.

Jesse: So if you’re not in there advocating for more design, more design, more design, what do you see it as your role to advocate for on behalf of your team?

Amy: I’m not saying I don’t do that. There’s definitely times where I do that again, like we look at the makeup of the entire team and decide what we need, right? So there’s, plenty of times where we’re saying, Hey, we need more designers or more writers, or more researchers, and I, here’s, here’s what won’t happen if we can’t get those folks.

So trust me, like I have to advocate, but I think that just the culture here is one where there’s also times where I’m like, you know what? Instead of research, really what we need is more QE because, we do these interlocks, we’re looking at the bigger picture. So I just think that is a great business practice so that you are looking at objectively how is the team shaping up and how is it growing together, and that you’ve got everything you need to make the entire business successful.

And so having that visibility, I think is really great for the cross-functional leadership team as we do those interlocks, because then you can see the perspective of, okay, I understand we didn’t get a researcher this time, but I can now tell my team why, and that I understand that it was in service of a bigger picture need.

Jesse: What do you think is it that makes those interlocks successful, as opposed to simply being the recurring, repeating the same argument every meeting between the same people over and over again, over the same things.

Amy: Well, I think naturally we see people respond to those conversations, so, you know, maybe culturally that’s different at this company than others. But I think the transparency in having the conversation is the important part. And then I see people make accommodations, you know. So we have a GM and we’re like, Hey, we really do need a researcher for this, or we’re totally understaffed on this other thing. Like, generally they’ll do what they can do, you know? So I think just hearing that and actually trying to make adjustments based on those needs and feeling that there’s always this give and take and level of understanding to try to do the right thing is most important. Then it doesn’t devolve into just a battle.

Jesse: Mm-hmm.

Peter: When you’re saying GM, I’m assuming there are multiple GMs. You have different business units. But design or CX is centralized, it sounds like it’s organized to align with those business units.

Amy: Correct.

Centralized vs decentralized

Peter: Um, You are getting to a size where, in many companies, there would be a discussion of decentralizing, and I actually see it in much smaller companies where when they go to a GM model, the designers report up through that GM.

And I’m wondering if that conversation has taken place at ServiceNow. And what are the, considerations around when does it make sense to maybe no longer be centralized as a function…

Amy: mm-hmm.

Peter: And yeah, just how is that discussion going? if it’s even going at all? Or maybe you’ll say, no one said it.

Amy: No one said it to me yet. I am very cognizant of that. I’m very cognizant of that.

Peter: I mean, Google,

Amy: Microsoft, Google, I think Salesforce, like most other companies, at a particular scale, you fork the teams. The reason why I don’t know if it’ll ever make sense at ServiceNow is again, it goes back to like our core, core value of our technology is it’s one platform, one data model, one architecture, one front end.

So we’ve been able to hold that together and no other company has, right? If you look at Salesforce, they’re all separate clouds, right? If you look at Google, all those products are different data models, different front ends, different back ends. The data won’t talk to each other. And that’s problematic when you wanna create cohesive experiences.

So, one of the reasons that I was drawn to ServiceNow is I was like, this is pretty unique. You know, like the front end component system truly powers every product. So do that, you have to have strong horizontal connectivity, right? And so I do believe that makes a lot of sense when it comes to our engineering and all of our EX. We are centralized under functional leaders. Our product management teams not quite as much, but again, they’re kind of oriented around big areas of the business. So my team does serve as that glue that holds a lot of it together and make sure that, you know, experiences are cohesive, we’re taking systematic approaches to things, and there’s a lot of leverage and efficiencies that come from that as well too.

Could it be decentralized? I mean sure there’s, and never say never, right, but at this point I think there’s a lot of value in having it together.

Jesse: Speaking of the value in having things together there are lots of design organizations that find themselves kind of caught between a value proposition that is very much oriented toward the delivery of great design work, right, and we’re gonna be an awesome design factory, versus a value proposition that is not as delivery oriented and is much more sort of forward looking and innovation or product strategy.

Amy: Mm-hmm. Mm-hmm.

Jesse: I wonder how you balance and reconcile those things across your teams as you are wielding this cross-functional influence.

Amy: Yeah. So are you, talking about kind of like the dichotomy between like craft or design with a capital D in terms of like creating something that is so beautiful and amazing and gorgeous and all that versus product strategy and business value and like are we actually delivering the thing that’s driving the… yeah, you know, that’s a, good question.

I would say on the spectrum of things, ServiceNow is probably a little bit more skewed toward driving business value, which is I think, the right thing for us. That being said, we’ve invested a lot on the aesthetics and are focusing a lot more on getting those details to a point of polish and beauty and, you know, brand finesse that we’re known just as much for that as we are for the foundational value that the product gives.

But our roots are really in value, right? Like, so again, that ability to automate and streamline businesses and, you know, we have a product called process optimization, for example, where when you’ve digitize processes on our platform, we can show you exactly where there’s bottlenecks, where maybe like an approval process is slowing something down or a lack of some particular thing is, and then we can help you automate that piece or streamline it.

So our software, like fundamentally helps businesses run more efficiently and effectively, which means better customer service, better employee engagement, better, you know, productivity with their products. So that is, I think, just core to what makes us successful. So I do think like the UX leaders on my team are very strategic, very much thinking about product strategy and influencing product strategy.

But we are also getting like much and much better at the beautiful aesthetics of it too. We launched a horizon design system this past year and it’s just absolutely gorgeous cutting edge design. We published principles around designing ethical AI. We’ve published all of our accessibility approaches as well, too.

So that work I think is very much in that category of a capital D where like, if you look at the motion and the interactions and the visual aesthetics and all of those things that I think designers just love, there’s a lot to be proud of there as well too.

Peter: When you mentioned publishing ethical AI and your accessibility standards, is that, I’m assuming by publishing you mean make public, like we, any…

Amy: Yeah. Yeah. They’re public.

Peter: What was the motivation behind publishing that?

Amy: Great question. I think it’s just kind of intrinsic to ServiceNow’s values. We knew that, obviously there’s a lot of nervousness and trepidation around AI, and we wanted our customers to feel really confident that we were doing this in the right way for people. And so even our marketing language anchors on that, which is you know, make AI work for people, right.

Put AI to work for people. And so people is always part of the statement and everything that we’ve done in our user experience design is to make sure that there’s humans in the loop at any kind of important juncture. You know, we’ve launched Agentic AI. Now we make sure that Agentic AI can work on all of the kind of research functions, but if it comes to, you know, like read versus write, it can do all the read stuff, it can pull things together for you.

It can tell you all the things that it has done. But when it comes to like write something, meaning make a change or click submit or take action on that information, there’s a human in the loop. So it’s important to us, ’cause it’s the right thing to do and it’s important to us ’cause it’s the right thing for business, right?

We need our customers to know that we’ve done this in a responsible way. And the reason we publish them externally is it’s literally a guidebook for customers and product development teams to use to make sure that every step of the product development cycle, they’re asking the right questions. They have the right representation in the mix and their counteracting the bias and their thinking ahead around where things might hallucinate and how to continually make sure that there are the right guardrails and checks in place so that their business isn’t compromised, or high stakes things are not at risk, right?

Because using AI in an enterprise context, and especially as we move into Agentic AI, you’ve got businesses with very, very high stakes things that could be at risk if you’re not handling it in a responsible way.

So we have to be very, very transparent about that, to keep that trust with our customer and not lose it and make sure that they have full transparency into how our models are built, how we built our own software, and then we can guide them, too.

It’s available on the Horizon website. So if you search for Horizon design system, those HCAI guidelines are all published there as well, too. And, you know, it’s, one part thought leadership as well too, right? Like ServiceNow, a big part of our brand evolution is that we’re now perceived at the forefront of AI and that’s been something we’ve worked very hard at the past couple years.

So all of these things, both in what we’ve delivered and shipped within our product and the success our customers are seeing with it, as well as these publications are important part of that whole story.

Amy’s journey

Peter: I wanna go back to when you started at ServiceNow, which I think was about five years ago.

Amy: Yep. Correct.

Peter: You started as a simple VP and global head of Design.

And I’m wondering, when you joined ServiceNow about five years ago, how much of the path that you were on, did you foresee, like, did you have an agenda?

Amy: Such a good question.

Peter: Did you have a sense of, you know, what the next three to five years would be? Or, and how much of it has been a response to how things have unfolded and how have you figured out when were appropriate points of growth to, you know, ’cause your mandate has expanded over time, like, given that growth that you’ve had in your role, how much of that was planned and how much of that was realized?

How was it known that that was the right thing to do?

Amy: So did I see this coming? I got an inclination that it might happen, but I was also skeptical. I am a, pretty healthy, you know, cautiously optimistic, somewhat skeptical type person by nature. So when I interviewed here, there were a number of things that drew me to the company.

It was not necessarily this promise of, oh, the company is gonna scale tremendously and you’re gonna scale the team tremendously and your career might grow along with that. I joined here more because I really like the culture, I like the leadership team, and I saw a big opportunity with the product that I thought was really interesting.

And so that’s what drew me here. And the culture was very much hungry and humble. And I like that because I worked at LinkedIn previously through kind of that formative growth time at LinkedIn and that was one of my favorite kind of experiences in my career. So I wanted that, again, I wanted to be part of a company that was still shaping where I could have influence, that was malleable and that was on a growth path because, you know, growing is fun. And I do like scaling UX practices and maturing teams and finding great talent and figuring out how to expand on product experiences. So, that part definitely drew me in.

When I interviewed, I remember being asked like, well, can you scale a team? And I thought, well, sure, I can scale a team. And when I was at LinkedIn, I think I joined and there was like six people on the whole user experience team and it grew to like 120. I’m like, that’s pretty big, you know? I can scale.

And I think here the team when I joined was probably over a hundred people, but under 200 maybe, let’s so call it 150. So I was kind of coming into a team that was equivalent to say the size of the LinkedIn user experience team or even the team at Google that worked on G Suite. So I thought, yeah, like it was already a pretty big team.

Can I scale it bigger? Sure. Did I think it would scale to over a thousand people in five years? I wouldn’t have believed that at that point, if you would tell me, and I remember the person interviewing me kind of said, yeah, well if you look at the growth rate of the company and what we plan to do, like, and you extrapolate the growth of your team, you’ll be at like over a thousand people in five years.

I remember being told that and I was like, nah. I was like, sure. I nodded. Oh, okay. Yeah, no problem. And I remember thinking in my head like, that’ll never happen. But here it did. So I mean, yes, was, I told that this could happen? Yes. Did I believe it a hundred percent? Not entirely, but but it’s tremendous that it did play out that way, and we’re continuing to grow and expand.

And it’s been a huge privilege. And, actually some of the best advice I got on this came from a engineering leader I work with. We were having dinner recently and they’ve been here longer than me, and I said, well, what has it been like? You were here, like, really early days of the company. And now you know, you’re managing, I think he’s got like a, say a 4,000 person engineering team. It’s like, well, every year the team’s grown like 20 to 30%. So every year I think about how do I get 20 to 30% better at what I do? How do I become that much better as a leader? And I was like, oh, that is so just resonated with me.

Like that’s what you have to do. You have to challenge yourself to grow, to meet the needs of your team and the business. And if you can’t grow at that same rate as you’re developing as a leader and someone who’s bringing strategic ideas to the table and making a difference, then probably you should bow out at a certain point and let someone else come in who can work at that scale, you know?

So it is been a privilege, but it’s definitely kept me on my toes and challenged me tremendously to lead at this scale and to continue growing along with the team at that rate.

Jesse: In what ways are you challenging yourself as a leader these days?

Amy: Getting much more involved in the business side of the house is a continual area of growth and challenge for me. So whether that is getting involved in really tricky customer negotiations or escalations and helping turn things around in a positive way that’s been a real good area for me to get comfortable with so much more of kind of an outbound role than internally facing, you know, and I, as someone who started out as a designer who really just wanted to put headphones on and hang out behind my monitor and design things, like becoming more outbound facing and extroverted in those kinds of types of roles are definitely a growth opportunity.

And then I think continuing to advocate for what my team needs. I mean, we talked about like, you know, needing to say, I need more designers. It’s kind of bigger than that now in terms of the story that I need to tell.

And being also expected to deliver a future vision continually when technology is changing so fast and helping people understand how the ways that humans interact with technology will evolve and what it would be like two years from now, four years from now, that’s getting increasingly more challenging because of how quickly the technology is moving.

But that’s a big part of the job and how I have to keep challenging myself too.

Growing as a design leader

Peter: I’m curious what the mechanisms have been for you to figure out the nature of your growth. As in, trial by fire, you’re just thrown into a customer facing conversation and you have to make it work? Or are you getting mentorship? Are you getting formal coaching? And not just with that, but generally, like what are the resources that you turn to, to help you as, you’re trying to figure out what’s the next 20%, to use that conversation you had with your engineering partner.

Like, how are you figuring out what it means to be better? What are the resources you’re drawing from to unpack that? Because this all new to you.

Amy: Yes. Yeah, no, every day is new in some ways. Lots of resources. So I definitely, I definitely don’t sit back and just expect it’s all to come, you know intrinsically. So obviously feedback. I do welcome and, you know, get feedback from various different avenues to figure out like, how am I showing up? How can I improve? What does my team need? What are their expectations? What are leadership’s expectations? So I do welcome that and focus on that, certainly as one avenue.

I have absolutely engaged with professional coaching all along the way, so that’s really important for me too, in various formats, different people. And I’m actually working with a coach right now that coaches a bunch of professional athletes, and that’s been really interesting too, because I do find, especially in these outbound facing roles, representing the company on stage or going into a high stakes meeting, like it’s very much a performance athlete kind of role, right?

Like you’re under pressure. And you’re getting evaluated continually. It’s competitive. And the psychology behind all of that is really important, right? Like what are the head games that you’re playing with yourself that might be setting you up for success or not. So that’s been really, really valuable recently.

And then I’ve got, you know an advisory board and friend group of a lot of other you know, UX leaders, design leaders and so on, that I continually nurture those relationships and friendships and their incredible sounding boards.

And you know frequently in these roles, you’re the only one in this role, right? When you get to a certain point where you kind of the head of a function, it gets lonely real quick because no one else does what you do. And if they do, they’re probably at a different company. So you can’t necessarily like, disclose everything, but you can certainly talk about the leadership challenges you’re facing or best practices you might be using or what you’ve tried, things like that.

So that advisory group of, other professionals in this space is incredibly important for me to continually nurture and learn from.

Jesse: Amy, what conversations are you most looking forward to as we move into this next phase for design?

Amy: I think what will be interesting is I do think the ability to design things will continually get more democratized, right? Like I’m sure you’ve seen things around generative UI, right? So anyone could describe an interface and it could be produced. You don’t necessarily need to know the tools, right? The Figma or the Photoshops of the world.

With that though becomes obviously a lot of room for error for those who maybe don’t have the same design principles or fundamentals or understanding or gut instincts, for maybe things to happen that are again, like not usable or not great product experiences. So. I do think it’ll be interesting to see, like, do our roles shift from being doers to being those who design the experiences that shape good experiences, right?

Which is kind of like the role my team is in. We create building environments like developer tooling, and we have a product called UI Builder, right? Where it’s the place where you can build user interfaces on our product. So how do we build the right expertise into those products or into the models that are producing the generative UIs to still guide and deliver great experiences.

If I put it into a metaphor of like an orchestra. You’re moving more and more people out of the roles of playing the instruments into being the orchestra conductors, but they have to know how to bring all those things together and make music. So how do you train someone about what good music is and how do you teach that?

So I think that’s like probably where it’s headed, where we’re gonna have more and more people that ultimately become the creative directors, the art directors, and they might have an idea and a vision. They may not have the classical art and design training. So how can you train that on the fly or provide that right guardrails?

Amy: And maybe that’s where design expertise sits in the future.

Peter: Hmm.

Amy: What do you guys think?

Jesse: I love that vision. I think it speaks to an evolution of design’s value proposition that potentially is a leveling up.

Peter: Well, it, starts to beg questions. I mean, that kind of orchestration or coordination type mindset is how product management is often categorized, as kind of the chief corraller of cats, depending on the organization, the people don’t report into the product manager, but the product manager has some decision-making authority, but they’re there to really try to get the most out of their team.

And so it starts begging the question, well, where does design and user experience end and product management begin? I don’t know if you have thoughts on that.

Amy: I would say I think there’s places where those lines are blurring more and more, maybe more so than you might say between UX and engineering. Although I think those lines will blur too. This tooling is getting so powerful that given it’s all conversationally based and we all have conversation, right?

We all have the power of language. We can all describe things. That becomes, again, a very like democratizing skillset. And the artifact that gets created, whether it’s code, whether it’s UI, whether it’s a product requirement document, matters less now. You don’t necessarily need the skills to create the artifact, you just need the skills to describe what you want.

It’s gonna turn us all more into those visionary roles where you have to understand what the need is. I think there’s still tons of need for research and psychology and understanding how to empower humans, but bridging that, how to empower humans with the technology.

The pieces that are in the middle of that, I think are gonna kind of merge together a little bit more, and there’ll be less distinction over the subject matter expertise of each part of that development process.

Jesse: Amy Lokey, thank you so much for being with us.

Amy: You’re welcome. It was nice to be here. So great to chat with you both.

Peter: This has been fantastic. Do you like to be found, and if so, where can people find you?

Amy: You know, I have to say I’m not a prolific poster of things aside for LinkedIn is kind of the main place I go. I worked there for a number of years. I love the product. I think it still provides a lot of value. So I would say anyone wants to reach out to me. A message on LinkedIn is always welcome.

I am not so much on the X type of platforms these days, so I think LinkedIn’s a a nice place to converse. So I’d say find me there.

Jesse: Terrific. Thank you so much.

For more Finding Our Way, visit findingourway. design for past episodes and transcripts. You can now follow Finding Our Way on LinkedIn as well. For more about your hosts, visit our websites, petermerholtz. com and jessejamesgarrett. com. Peter recently launched The Merholz Agenda, his semi weekly newsletter. Find it at buttondown.com slash petermerholz. And if you’re curious about working with me as your coach, book your free introductory session at JesseJamesGarrett. com slash free coaching. If you’ve found value in something you’ve heard here today, we hope you’ll pass this episode along to someone else who can use it. Thanks for everything you do for others, and thanks so much for listening.