Leah Krauss: Responsible AI and Content Design at Microsoft – Episode 31

Leah Krauss New AI products like Microsoft's Copilot can be powerful productivity enhancers, but if designers aren't careful they can inadvertently introduce into the product the bias and other hazards that can come with large language models. As a content designer working on Microsoft's Copilot for Sales product, Leah Krauss helps her colleagues understand and follow the responsible-AI principles that the company has developed. Leah's advocacy helps her design and product teams create a product that balances the need for transparency about the use of AI with the prerogative to keep customers in flow as they use the product. We talked about: her work as a content designer on Copilot for Sales at Microsoft and her advocacy there for responsible AI how she collaborates with her data science team, which had established a relationship with the content team even before Copilot on other products the evolution of their AI product-development process how their design system supports the implementation of responsible AI the six principles that guide responsible AI at Microsoft: fairness reliability and safety privacy and security inclusiveness transparency accountability how she advocates for responsible AI on the Copilot for Sales product team the balance between keeping customers in their flow and being transparent about AI features the concept of the "human in the loop" and how they apply it in the Copilot for Sales product the importance in AI product design of always being aware of edge cases and possible misuses of the product her encouragement to anyone working on AI products to stay curious, ask a lot of questions, and to bear in mind the importance and relevance of our language expertise Leah's bio Leah Krauss is a senior UX content designer at Microsoft. She works on Copilot for Sales, Microsoft's AI software for salespeople, where she also collaborates closely with the data science team. She champions responsible AI to anyone and everyone who'll listen, including inside Microsoft and at various UX conferences. Outside of work, you can usually find her reading, or spending time outside with her family - hiking, exploring cities, and hanging out on the beach. Connect with Leah online LinkedIn Video Here’s the video version of our conversation: https://youtu.be/VItdSUgzkZE Podcast intro transcript This is the Content and AI podcast, episode number 31. The introduction of AI tools like Microsoft's Copilot creates new opportunities for content designers. But as with any innovation, the new technology can be a two-edged sword. For every customer workflow that is streamlined there may also be an opportunity for bias or hazard to get into the product. As a content designer and champion for responsible AI, Leah Krauss helps her colleagues at Microsoft understand and apply responsible AI principles in their product design work. Interview transcript Larry: Hi everyone. Welcome to episode number 31 of the Content and AI podcast. I'm really delighted today to welcome to the show Leah Krauss. Leah is a senior UX content designer at Microsoft where she works on Copilot, which many of you may have heard of. So welcome Leah. Tell the folks a little bit more about what you're up to these days. Leah: Hi Larry. It's so nice to be here. So yeah, as you mentioned, I'm working on Copilot for Sales, which is a flavor of Microsoft Copilot, and that's been really exciting to be in on kind of the ground floor of AI at Microsoft. And responsible AI, which is what we're going to talk about today, is one of my most favorite topics to talk about. I've done some conference talks about it and my coworkers are really tired of hearing me go on about it. I actually serve, no, that's not true. It's only half true. I serve as actually a responsible AI champion, one of the responsible AI champions on my team. So it's sort of my thing and I think it's so exciting the moment we're at here and how big a part content designers can play in it. Larry: Yeah, I think when you're working with language models, you think the content people would have a leg up on some, but that's really, first thing I want to follow up on is that Copilot, I think it's sort of like multiple products then. It's a suite, well I guess Microsoft knows about suites with the Office suite, so there's integrations of copilot with each of the Office elements, Word and Excel and all that. And then there's also specific tools like Copilot for sales. How big is that little budding empire at Microsoft? Leah: Well, so there yeah, like there's one Copilot and then the different flavors depending on the user's needs. So as you said, if people are using Office, they'll see Copilot in Word and they'll see it in PowerPoint and Outlook of course. And if they have more specific needs like a salesperson, then they can use our more specific flavor, which has the kind of email summary that a seller might need. Did my customer mention the budget? Or things like that. And it can also pull from other sources so that the seller can have everything they need right in that one place. Copilot is a big thing at Microsoft, as you know, and I think it's only going to get bigger in the next couple of years. Larry: Yeah. As you're talking about it too, I'm reminded of how this thing is coming together. It's like there's this one, I assume it's based on one of the GPTs from OpenAI given a Microsoft relationship there. But it sounds like then that most of the applications, each of these flavors of Copilot, is a lot of your work around fine-tuning the model then for that specific task? Leah: Absolutely. I work really closely with our data science team and the first thing you sort of have to, there's the GPT prompt, but then we also write our own prompts on top of that. So we tell the GPT that this is a selling audience, for instance. And I'm really involved with working with the data science team to define what a good output looks like because while they're the experts in the model and how to create an output, I'm the expert in what makes it good and what makes it human and what makes it useful to a seller and scannable and valuable and things like that. So the great thing about this project has been that we get started early on working together. As we content designers know, sometimes content people are brought in too late. And that is definitely a danger here too, like for people who are listening, you can always do something with the data science team, but if you're there from the beginning and you're talking about the prompt together, then you can move forward and really have an impact. Larry: That's really interesting because there's all these new collaborations that are emerging along with these AI tools. So you're working, is it mostly data scientists? I've talked to other folks who've worked with the machine learning engineers and other new collaborators. What does the team you're working with look like these days? Leah: So at Microsoft it's called data science. At other places it may be called machine learning, but it's the same group of people. Larry: Yeah. Leah: There's the people who care about the algorithms, basically we can call them. And then we have the people who care about the words, which by the way is not only content designers, it's also product managers and interaction designers, but content is really leading the way. Larry: Yeah, that's fantastic to hear. And what you just said too is it sounds like there's more opportunities or is there more opportunity or have you just made it happen to get in earlier or do data scientists see the need to involve word people earlier on? Leah: Well, we've been working on AI features and machine learning features even before there was Copilot and even before I joined the team actually. So my manager who has been on the team longer than I have was working with the data scientists when we had a feature called Conversation Intelligence. And what that did was when you would record a meeting, a seller would record a meeting with their customer, then Conversation Intelligence would analyze afterward and give sort of a recap that has the main action items and a bulleted list of the highlights of the meeting. So my manager whose name is Erga Herzog, and she was the one who in the sales organization really built that relationship with the data scientists. And also I work with a lot of other content designers too. So basically I was lucky to come in about two years ago with this really strong base that was already very far along. Leah: We had a good relationship with the data science team and there was already that conversation was starting. So what we're doing now is we're sort of trying to formalize that process because sometimes it relies on the individual content design or the individual data scientist to decide when we start talking about a certain feature. And we'd rather have it be more formalized into a process that like, okay, at this point we start talking and then at this point we start looking at sample outputs and maybe we first decide together along with the PMs and the rest of the product squad and the leadership of course, what we want out of this particular AI feature. So that's something that we're working on right now, which is really exciting. Larry: That is exciting because this is also new and at some point you have to, like it must be, I can only imagine the pace of work there, but so being far enough into it to kind of step back and go like, oh, hey, this is how sort of the routine way that we, or not routine, but the common way that we do things. Is that sort of, do you think that's common across the other flavors of Copilot or, like because you mentioned you were talking to your peers as well as your immediate colleagues. Is there sort of patterns emerging around how those, like you just said,