Managing Prompt Libraries for Effective AI Operations

“My hope is that in the future, we are so focused on acquiring knowledge in our schools and our educational system, my hope is that we will also be focused on acquiring wisdom, if we know how to measure it and how to develop it.”

– Charlene Li

What you will learn

• Exploring the edge of disruption with Charlene Li
• Amplifying cognition through generative AI
• The transformative role of AI in research and writing
• Developing the AI-powered development editor
• Customizing generative AI for personalized guidance
• Generative AI in email communication and productivity enhancement
• Crafting strategic business proposals with AI insights
• The importance of sharing and learning AI prompt libraries within organizations
• The strategic impact of generative AI on competitive advantage
• Harnessing generative AI for customer experience and operational efficiency
• Wisdom hacking: Enhancing decision-making and leadership

Episode Resources

• Generative AI
• Large Language Models (LLM)
• Custom GPT models
• AI prompt libraries
• Blaze.Today

Books

• Winning with Generative AI: The 90-Day Blueprint for Success by Charlene Li 
• The Disruption Mindset by Charlene Li 
• The Engaged Leader by Charlene Li 
• The Seven Success Factors of Social Business Strategy by Charlene Li 
• Open Leadership by Charlene Li 
• Marketing in the Groundswell by Charlene Li 
• Groundswell by Charlene Li 

Transcript

Ross Dawson: Charlene, it’s awesome to have you on the show.

Charlene Li: Thank you for having me.

Ross: So, how do you amplify your cognition? Which is pretty Apple to start with?

Charlene: A good question. Well, I do research and write my books. And I, basically it, especially with AI coming along, I looked at all the things that I do. And I realized that AI can address 70% of it and just, it just makes it so much easier. So a couple things. First of all, just researching, it’s so much faster now, with general AI, I use it constantly. Whenever I have a question, I’ll use one version of an LLM, probably perplexity and being also as well, to just get me the information and get into it a lot faster. I may find all those reports, bundle them all together and ask them to summarize it. And then a specific point, am I going to dig down deeper, it’s almost like having a research assistant. So it’s all the things I used to have a research assistant do for me, now I just have generated AI to do it for me. And it’s much faster and better and an expert on every subject that I could ever imagine. 

And then I also use it when I’m writing. And so I’ll write a first draft, like a fat outline, and I’ll give it to an AI and say you’re my development editor. Tell me, what am I missing? Ask me questions. So it’s literally a thought partner to make my thinking better. Because it can look at things from a different perspective. And I can give it different roles, different perspectives, you’re an expert, you’re brand new manager, look at this topic from many different directions and ask me questions on how I can make it better. So those are things and then I can use it to actually help me piece together some of the writing later on in the process.

Ross: Just to dig into each of those a little bit in terms of first of all out on that development editor. So, is this a to and fro process, so its development manager comes up with this list of suggestions, which you then ask for or you manually used to refine the text? So what is so drilled into that sequencing if you’ve done a draft? Okay, that’s what I think that’s one interesting point, you do the first draft first yourself before you give it to the AI? So what is the detailed process? Do you just simply say, take the role of development editor? And then take all leave it suggestions? Is that iterative? How do you get there?

Charlene: I actually have my own private GPT. It has all of my writings in it, it has my book outline and has all the previous chapters that I’ve worked on, because I am working on this book about how to use gendered AI, as a business. And I say, you know, put all that into context has all of that has general instructions on how to be a great editor. So I don’t have to repeat that over and over again. So it’s customized, I call it the book wise editor. And then I just started asking her questions, here’s my latest draft, what are the strengths and the weaknesses of it, identify three or four ways, whatever number of ways to improve it. Don’t touch the texts, and give us me and then I have a discussion. 

I’m like, well, this first point that you have, what’s your logic? Give me examples from the text, you know, what else would you expect to be here? So I have a conversation and drill into each of these. And I asked it to ask me questions along the way. Well, what do you need to know? Do you have an example that says this? How do you back up the data points that could make this happen better? So it is as if I had an editor sitting there, literally, with me, 24/7 whenever I need them, to give me feedback onto the book. And the cycles of feedback are so much faster than I could have before. But it’s a back and forth, and I find it to be so valuable.

Ross: So you’ve crafted this GPT yourself, including not only your body of work, but also the instructions on what a good development editor should do. 

Charlene: Right. It’s pretty high level and just like your development, you’re a business book development editor. And you’re knowledgeable about technologies too, as well. So I wanted to have this certain perspective. So they’re an expert, but they also are functioning as a book editor who can look on behalf of the reader. That’s what development book editors do. And I don’t have to spell that out. Just everybody knows what it does, because I’ve trained it. It doesn’t have quite a memory. So these are the things that I have to make sure it’s doing. And I’m asking sometimes to look at different audiences too. As I mentioned before, look at this from the perspective of this audience versus an audience. How do I square that because I can’t write to every single audience separately, what’s the common denominator across them that’s needed for this particular section?

Ross: So you and I and many others use generative AI extensively through our working days. So what are some of the other highest value applications you found in your work?

Charlene: Well, I think it’s there. But it’s also just helping me from just tracking tricky emails, summarizing them. I have it built into my email program, because I use superhuman AI built into it. So it’s pretty fast and easy, I can use it with just a couple of keystrokes. I can say a drafted response, that response this way I can review it, or I can draft something and say, improve my writing. And then if so, that’s just a very simple, fast way. I use it constantly, all the time. And I’m very careful to check everything before I press the send button, there were a few mistakes, I’m like, Oh, I didn’t carefully check as much as I would like to go on, that’s not the way we should have got. 

And then the other thing I do is in my marketing and preparations for things, oftentimes I’m crafting individualized pitchers proposals, and I can do background research on a company, create a profile for them, put in my have, again, uploaded my offerings and say what are ways that what they’re doing, and what I’m doing could overlap are some ways that I can pitch them. And so, they’ll find me very identifiable, very specific ways to phrase it in their language. So, it feels like they are talking to themselves. And you know that anytime you’re speaking, the language of your client is always going to be better. So versus me having to figure that out spending a lot of time on the website, I can just pick up all that language, interpret and mash it all together very, very quickly. So again, in addition to the research assistant, I have a sales and marketing assistant too, as well. 

Ross: I remember, back in my very early sales training days, Tom Hopkins says use the language of your client. And that definitely makes things flow better. So is this for crafting emails or approach letters or proposals or what kinds of the above again, I have some longtime clients that it’s helpful to just understand how they work and think. And so just having that body of knowledge and not to the point where I’m doing so much work for them that I have a lot of work, just built up over. But there’s this history of work that I’ve done. And so having that context can be very helpful. It’s not built into my CRM system yet, but I do have AI summarize my meetings, and it shows up and it gets added to my records of that meeting for that person. So that’s always super helpful to have just a quick summary, readily available in my CRM. And I’m just using Pipedrive. It’s nothing fancy. Right?

Ross: So let’s say you were working with an executive team of a decent sized organization, they’re all technology savvy, they’ve all played around with these tools, and they’ve found some ways to use it. So how would you? What would you suggest to them, either individually, or as an executive team, and how they could use generative AI to enhance their most complex, difficult challenges?

Charlene: Well, first of all, I hope that they have a good handle. They’re not just using it, but they’re actually transforming the way they work. Because I find that until you have done and seen the transformational power firsthand, personally, it’s really difficult to extrapolate that into your business. And then what I advise and this is what my book is about, it is all about starting with your business strategy, and really, truly understanding the components of your strategy that drive your competitive advantage. What are the assumptions that you’ve made about the marketplace? Do they hold true in a world with generative AI? How could generate a threatened competitive advantage? And also, how can it reinforce it? And most interestingly, does it create new competitive advantages for you? 

So one organization I talked to, and they are very large global organizations with hundreds of 1000s of employees, doing business process outsourcing, such as call centers and back end so they’re highly, highly susceptible to being, again, disrupted by journal AI. So they’re like, we’re going to get on this right away, and within the first few months, within two months of launching, so in January 23, they already had it deployed into their call centers, training people on how to use it and finding productivity gains that was the first arm first leg of their strategy, that then freed them up to do the second leg, which is to create new products that could serve their clients. From the very beginning, they told people, we’re not going to use this to get rid of people and get rid of her account that would be shortsighted of us. We don’t have too many people. We don’t have enough people to do the work that we need. On behalf of Our clients, so we can free up time, we can free up cycles, create new products, and then from there look for new ways to create competitive advantage. So they map that out right from the very beginning. And by the summer of 23, we’re launching new products already. 

So this is, again, you go back to the core of your strategy. And you say, instead of looking for the individual small used cases that are going to be safe and easy to deploy, you look for the biggest impact and then say, how do we start small, but you’re thinking big, starting small, and then scaling fast?

Ross: Yes. And I think one of the key things there is, but it does depend on the use case, of course, they say, how do we amplify our people, as opposed to saying, how do we take away from other people already do. And that’s kind of the theme of the amplifying condition is, we have incredible bonds of the executives and the people in the organization and generative AI can be used to enhance what they do, make them incredibly more productive, and create more value. And it’s just a mindset shift, which is not always out there.

Charlene: But in the challenge, because they want my CFO is going to come and say I need to take money right away. I’m like, look, this is we know this is a transformative, paradigm fish shifting technology. You have this once in a lifetime chance literally, to be able to start new businesses to create new products for your clients and your customers, with no additional people with not a huge investment in a fraction of the time. And you want to blow that not take advantage of that and just find one or 2% benefit and cut people like how shortsighted can you be on this. So that’s why it’s so important to get the tool into the hands, I think of your top two levels of your executive team. This is not something that you just keep shunted away into, you believe it’s going to be paradigm shifting, then the top layers of your organization, your top executives first and second layer need to be hands on with this and thinking strategically about how to do this. So get a playground set up, train your top two levels. And then the third part that relates to cognition is to create channels in places where you can share the questions, the learnings of best practices, and you start treating your prompts. As intellectual property, your prompts are IP. And yet what we have are people doing all these individual things, and they’re not sharing the best practices and you’re not learning as an organization how you want to use this incredibly powerful technology. That learning has to be organizationally centralized, but decentralized in its execution, so that learning has to come back. So you can start creating the processes and the governance and the best practices of how you’re going to use this to transform the business.

Ross: So what have you seen inside organizations, this idea of I suppose prompts, discovery, we can almost call it or creation, where somebody finds it useful for their work. And it’s then passed on to others, it was actually recently at a meeting where I have a number of marketing agencies, and the person presenting it said, Okay, put up your hands, whoever shares their prompts in their organizations, and nobody put their hands up except for me. But there is a cultural aspect of this, of course, you need a culture of sharing to be able to share your prompts, you know, the stars become valuable for the individual. So how have you created the scene, the conditions created? Well, for the people to recognize this is valuable for others as well themselves to be able to make that available, to make these disseminated, to make the best ones, you know, prizes to be shared.

Charlene: Right. And I think, again, this is why that initial channel is so helpful, and how you set it up and set the expectations of how people will contribute, how they were asked questions. And so there can be multiple parts in there. It’s like asking a question and getting an answer. There’s another one for best practices. There’s another one that says there are policies like this is how we’re going to go do it. And again, setting up your custom GPT is having a set group of documents that you will use continuously. They say these are the official documents, or we have trained our LLM. And it’s already uploaded; our strategy or marketing in all the personas that we have — these are our company documents. And this is the way to work with them. Here’s the thing, you get a chatbot right in any of these MLMs and there are no operating instructions, because there’s nothing and so as an organization you need to create the operating instructions of how to use this powerful tool to use it this way, and not that way. And we may find new ways to use it that we had never thought of before. And so you are rapidly as an organization discovering all the crazy, imaginative, amazing ways to use this technology. And that needs to be centralized, gathered, disseminated out into the organization because you draft an email here, same kind of thing happens there. But how do you make it as relevant as possible? Same thing with data analysis? How do you use these tools in new unique ways that are specific to the way your organization does it? And that’s how you get that competitive advantage.

Ross: So just custom GPT can be a useful tool to encapsulate a prompt which people can then sort of access and a library and so on. Are there any other ways in which you’ve seen those crystallized prompts?

Charlene: Yeah, people forget, you can put a huge amount of information into the prompts like hundreds and hundreds of pages. So again, having those saved in a repository or inserting custom GPT. I also love using Blaze. Blaze.today is just snippets that are created on your keyboard, so you can customize them for yourself, or you can share them across your team. So what are the shortcuts…So you can store entire prompts inside of those snippets, and share those like a whole library of them. And that’s one another a library of prompts that you can see and access. So again, some there, again, some people just do, here’s a Google Doc with links inside them to the prompts that you can copy and paste. I have seen everything under the sun, all of it works, you have to figure out the way that it works inside your organization, technology and process-wise, but also mentally wise, culturally wise, sharing. Just the fact of collaborating across different groups and organizations is not something we normally do very well.  

So this is why the role of AI ops becomes vitally important. Like your first person you want to hire with a specific AI role should be this person who just runs operations, making sure everyone has it. But in particular, managing that Prompt library, and building up the IP of how to use it as training people, getting people up to speed, making sure that the elements are actually working, that data governance is being protected, and used correctly. All those things, you need somebody to have to have oversight over it, or else it’ll just run off and not be this incredibly powerful asset for the company.

Ross: So going back to the original questions, you posed the executives, as in, you know, what is the impact in your business? What opportunities does it create? How does it change the competitive landscape? So how is generative AI best used for these kinds of strategic questions from your perspective?

Charlene: It’s like anything under the sun you could imagine. I mean, that’s the power of it. And the frustration is like, what could it do? Like it could do so much. So give me an example of how you see your competitive advantage being. So I talked to one organization, and they said, Yeah, we do this thing like everyone else, but the thing that differentiates us is customer experience. So we make this stuff, but really it is the way that we make it and the way we interact with people, we just have a higher level of customer experience than our competitor, which is much more responsive. It’s something we invest in heavily. And so they decided early on, how we can use AI to increase our customer experience. And they’re looking at all the things from their call centers, to the way emails to the way they fulfill their sales, to say how to reinsert AI into each of these areas. And so it’s not one single thing. But it’s this overarching strategy to say, all the ways that we have invested in customer experience in the past, we’re now going to look at every single one of those functions and activities and say how can generative AI support this? So that is our strategic look. And looking at crossovers, prioritizing, then seeing what’s feasible. And then moving forward with that. 

I’ll give you one example. One company said, you know, we basically have call centers, and we have people coming in. And there are a whole set of people that we just don’t even answer their calls, because we know from their phone number, we do enough analysis that it’s probably a low production call, so we’re not going to even answer their calls, because he doesn’t want to talk to them. And they said, ‘what do we have to lose?’ Let’s use general AI. Build a voice. Again, agents who can go and answer those calls, identify themselves and just prequalify by them, and they built it for a couple of $1,000. And within the force the first four weeks had generated a quarter of a million dollars in revenues from just stuff lying around that nobody even asked me to call us on. So that’s a pretty good return on investment. 

So people ask me, what’s the ROI on like, it’s sitting there right in front of you. What’s holding you back from trying it? And it’s usually fear of uncertainty, I don’t know if it’s safe. And so I like to say that if you want to go fast with these tools, with any of this technology, you have to have good brakes, like a Formula One driver, you can build the biggest, fastest engine, but unless you know, you can stop when you need to, you’re not going to go fast. So what are the brakes that you have inside the organization to feel like you can stay safe? And as long as you stay within these parameters, you’re going to be fine? We don’t have that in most organizations today.

Ross: Yes, that’s that sandbox, you got some things on the borders. And within that you play as much as you want to have those borders. 

Charlene: Yeah. I think of the playground. Look, this is literally a playground. And when you see kids on a school playground, if there’s a fence right up against the fence, right, pushing the edges of that, there’s no fence, they don’t know where they can go. So they say really close to home base, they don’t go anywhere, because they don’t know how far they can go. Indeed.

Ross: So…you have an eye on the future. So imagine in five years from now, you know, what is the potential? I mean, there’s obviously many, many, many directions which organizations can go. But we can reimagine what organizations are, and bring together the amazing capabilities of humans with this amplification with AI. So what are some, some aspects of how that might unfold and the most interesting organizations of tomorrow?

Charlene: Well, something that I’m you know, a side project that I’m very focused on is wisdom. And I’ve always been interested in wisdom and I defined wisdom as the ability to make good decisions based on your past knowledge and experience. And cognition is a big part of that, which is the ability to acquire knowledge and integrate it and then apply it. And so we talked about how AI can help with that. But wisdom comes from being able to be reflective, that you have the ability to learn from your past experience with introspection and reflection. It’s also the effective meaning of the ability to connect with others emotionally and intellectually and with empathy. And then there’s also situational wisdom, where you can apply your past experiences and balance multiple perspectives in a very complex, uncertain situation. So, if you look at how cognition reflects reflection, action, and then situational wisdom, AI can improve all of those areas, and improve our ability to make better decisions and actually hack wisdom and become wiser, faster and earlier in our careers. Because if we can just make 10% better decisions, we’re all going to be better off. And if we can amplify that within organizations and create wise organizations, then I really see the ability for us to break down, look beyond the biases, to see the situation as it truly is versus the way we perceive it to be. And AI has the potential to help us be better humans, I believe. I look at something as simple as empathy. We all have biases; we all have barriers and challenges to see things from a different perspective than where we sit. And so it can sit there and just wait, let me challenge you a little bit because you have this tendency to have this bias. And it knows that because it’ll be my personal AI by then. And it’ll catch me before I make these mistakes before I fall out of wisdom and make bad decisions.

Ross: That’s fabulous. So you’ve been looking at wisdom hacking. So let’s say what are the principles? What are the types of principles if you want to hack wisdom? What should I do?

Charlene: Well, again, there are very specific things. And so just at the very beginning of this research. I was doing this research and they got distracted by generative AI. It’s just things like being from a cognitive perspective. It’s having this mindset of being curious, but also, again, that intellectual curiosity to gain knowledge to apply it and then also have this ability to have a strategic longer term vision so that you can take the immediate things and apply them to a vision of what the future could look like. 

On the reflective side, it’s things like journaling, setting aside the time to reflect, to think about something that went well, or more importantly, something that didn’t go well. And ask yourself a lot of questions like, what actually happened in the short term? What were the impacts? How did my values come into play here? What assumptions do I make? What was the influence of other people coming in that made me believe you make a certain decision in this way? What lessons that I learned and what actions will I take in the future? So that kind of reflection is something that you can learn over time. And actually, that that practice can help you become wiser, effective, again, that connection with people having compassion? How do you practice equanimity, meaning that you can look at things and manage stress and face adversity, without that emotional connection that can make you move in one direction or the other? It’s, again, practicing equanimity, where you are looking at things and immersing yourself in a situation without judgment, without preconceptions, and that’s very difficult to do unless you train for that. And then situational scenarios are a great way to train for that. Like what other situations you can actually expect that you have to encounter? That you’re going to have to practice applying wisdom, your knowledge and experience to make good decisions? And how would you deal with a situation where you can actually be graded by the depth of your decision making? How far can you look? How come? How much complexity can you handle? So these are all things that we can train for. But you have to actively train for it. As I worked with a bunch of coaches, I asked them, when was the last time a leader came to you and said, Please help me be wiser. Like never. But if we want to be wiser and make better decisions, we can actively train that and develop that. And my hope is that in the future, we are so focused on acquiring knowledge in our schools and our educational system. I hope that we will also be focused on acquiring wisdom, if we know how to measure it and how to develop it.

Ross: Today, we certainly need wisdom to deal with the challenges we face and we will undoubtedly need deeper wisdom in the years to come. So I think these are very important things for us all to adopt. It’s not just about amplifying productivity, getting things done faster and having strategic opportunities. It’s having the breadth and the perspective to make the most of it for ourselves and others. So Charlene, how can people find out more about your work? 

Charlene: Well, you can come to my website, CharleneLi.com. That’s probably the best place to find anything and everything you want to know about me.

Ross: Thank you so much for your time and your insights, Charleen. It’s been a delight.