Chelsea Larsson: Building an AI Learning Machine at Expedia – Episode 27

Chelsea Larsson The arrival of generative AI gives content designers a whole new toolkit. As with any new set of gear, there's some learning that comes with the new capabilities that the tools afford. At Expedia, Chelsea Larsson is leading her team of content designers into the AI design future with fresh takes on the planning, design, and evaluation skills that designers have always relied on. We talked about: her work as a senior director of experience design at Expedia how she is facilitating with her teams the shift from product development design to AI design how she has identified new capabilities that AI brings and is incorporating them into product road maps how content strategists and architects help them decide whether to use generative AI or structured-content methods their shift from front-end content design to working with back-end engineers and architects how new LLM-driven applications of conventional content-evaluation criteria permit them to scale up their content design work their goal of creating good-quality content at scale how content designers are shaping the future of conversational ecosystems how AI lets content designers do more strategic thinking, in particular about how to apply their insights at scale her take on the recent rounds of tech layoffs one of the new roles that are emerging for which content professionals are well-suited, like the new position of model designer the origins of their AI program in a simple application of gen AI to partner content creation how to bootstrap the implementation of AI content practices in your org how to identify opportunities to help your customers by matching their content use cases with your AI capabilities her message to content designers: "don't be afraid" and keep learning Chelsea's bio Chelsea Larsson is a Sr. Director of Experience Design at Expedia Group where she leads the B2B Content Design team, partners on strategic design initiatives, and builds AI travel tools. Chelsea loves to chat about Content Design in genAI and UX design for travel. She shares her thoughts on both topics via the Smallish Book newsletter and conference stages around the world. Her favorite book to gift loved ones is the delightful Chirri and Chirra series. Her favorite sandwich is a turkey club. Connect with Chelsea online LinkedIn Smallish Book Video Here’s the video version of our conversation: https://youtu.be/qKr7o5aKQrM Podcast intro transcript This is the Content and AI podcast, episode number 27. The arrival of generative AI tools gives content professionals a whole new palette of design capabilities. Learning how to take advantage of these new opportunities so that they can shift from product-development design into content-driven AI experience design challenges many content folks. Chelsea Larsson sees these challenges as a chance for both her and her team at Expedia Group to stretch and grow and to scale their impact as design professionals. Interview transcript Larry: Hi everyone. Welcome to episode number 27 of the Content and AI Podcast. I am really delighted today to welcome to the show, Chelsea Larsson. Chelsea is a senior director of experience design at Expedia Group. And welcome Chelsea, tell the folks a little bit more about what you're up to these days. Chelsea: Thanks for having me, Larry. As you said, I'm a senior director of experience design. I lead the B2B content design team at Expedia Group. So we call that the partner content design team, because we work with Expedia partners. I also lead the Generative AI Experience Design Program, which we'll get into later and lean in on a couple of strategic initiatives at Expedia. Larry: Cool. And I think one way we were talking before we went on the air is we were talking about the idea of these AI learning machine, and that seemed to resonate with you as a way to describe what you're up to. Can you tell me about the machine you're building there? Chelsea: Yeah, so when I first started getting into AI, which I think was around a year ago, and talking about generative AI here, of course, I saw a kind of paradigm shift in how content designers specifically could work in AI fields, and it kind of led me to create what you called the learning machine, because when you're working with AI features, the planning is different, the designing is different, and the evaluating is different. It's not fundamentally different, but there are new layers to consider. Chelsea: And those layers led to a lot of questions on, well, how do we plan for the right AI opportunities in our product roadmap? How do we design these AI interactions, questions, when do we disclose that AI is being used? How do we signify that AI technology is being used without words? So what kind of iconography do we use? And then how do we evaluate the output differently than we would evaluate the output if humans had generated the content? So, when you think about those three different pillars of work, planning, designing, evaluating, we were led to, and I spearheaded this, create a program of critiques, guidelines, leadership forums, ways of working, which kind of has created this learning machine as you called it, which I love, and I can get into that a little bit. Larry: Yeah, and I love that, they sound like familiar practices, but talk a little bit about how each of them manifests differently in the AI world. Chelsea: Yeah, so when you think about, they're absolutely familiar processes and it's what we've been doing as product development designers for a long time, but there are new considerations to take into account. So when you think about the planning, let's start there. Your company's not going to just put AI into the experience. AI is not a solution. It is a avenue to get to the outcome that you want as a business, but you do have new capabilities now. You have text generation, you have text classification, summarization, you have multimodal content generation. You can create photos, you can create videos, you can pull out sentiment analysis. So with these new capabilities, you can matrix those to the outcomes that you already want to have or the user problems that you have in place. And by matrixing those with the new AI capabilities, it results in a change to the roadmap. Chelsea: You can plan for new outcomes because of the capabilities that you have with AI. Without understanding those capabilities, that is a hard conversation to have. So that was the change that needed to happen is educating our designers, our content designers and our product folks on what these new are. And that education at Expedia has kind of come about in these forums that I spoke about where I have taken machine learning scientists, product people, and designers and kind of for the first time, put them in a shared space and critique where we are sharing with each other ideas, capabilities and user problems. And those are kind of coalescing in new road mapped opportunities. So that's kind of a different way that we've started approaching planning these AI opportunities. Larry: Right, and I'm wondering if each of those parties you mentioned the ML folks, the design folks and the product folks, do they each bring different perceptions of those capabilities and is the mix different than it was before in those kind of relationships? Chelsea: Yeah, so there's also a fourth person, a fourth role who I've partnered with quite a lot in the past year, which I guess I'll call them a content architect at Expedia. They're called content strategists, which I know is going to be super confusing for this community. They're people who are really highly skilled in structured and unstructured content. So they're the NLP experts of the world. They understand BERT, which is a bidirectional language assessment pattern. They understand structured content in a way that makes it really easy and valuable, to have them on your team, to let you know as a content designer, if your solution should be generative AI or if it should be structured content. And that partner brings that knowledge to the table, they let you know kind of what the content landscape is and what the best content tool is to use. Chelsea: I think in the future, our roles will probably become one because they also usually typically have a writing background, a taxonomy background, a library science background, but they also have kind of a data, an engineering understanding. If content designers could lean more into that content modeling, content architecture side of things, these roles would basically overlap. But right now, those are two different roles where I work and are very helpful for partnership. Chelsea: The machine learning scientists, they bring all of the LLM knowledge, so they're helping us understand the base model, the behavior of the base model, what we can expect. They're helping us fine tune the model based on our prompts, based on our system instructions, the definition of good that the content designers are creating. They also help us understand the cost of scaling out some of the proposals that we have. We have to pay for the tokenization of the outputs, so how expensive is it going to be to generate this type of content? Chelsea: So they're the system and scaling experts, and we work with them really closely on the behavior and the output. We work with the content architects where I've talked about before on the inputs. What does this content need to be, how does this content need to be structured as an input and with the machine learning experts, how can we fine tune this output to get to the place that we want? All of these people understand the input and output, but they all have different levels of expertise where I work, I think it's different at different companies. Chelsea: And then the product folks are still doing what they've always done,