Exploring the Impact of Interactive AI in Idea Evaluation and Innovation Processes

“AI has an amazing amount of limitations, horrible in a lot of things. But when you use it smartly, it becomes an important part of your team. And as an important team member, your team gets better.”

– Bryan Cassady

What you will learn

• The impact of AI on innovation; enhancing efficiency and creativity
• Bridging knowledge in AI and innovation for systematic success
• Transforming idea generation; the synergy of AI and human creativity
• AI’s role in identifying and defining the right problems for innovation
• Leveraging AI for more effective team alignment and idea evaluation
• Exploring AI’s capability in improving communication and idea pitching
• The importance of diversity in teams, augmented by AI for better outcomes

Episode Resources

AI (Artificial Intelligence)

Idea Generation (Ideation)

Idea Evaluation

Market Scoping

Jobs to Be Done (JTBD) Framework

Brainwriting Technique

Book

CYCLES: The simplest, proven method to innovate faster while reducing risks by Bryan Cassady

Transcript

Ross Dawson: Bryan, it’s awesome to have you on the show.

Bryan Cassady: Thank you for having me. I’m glad to be here.

Ross: So you dig deep into AI innovation? So what’s the premise of AI innovation? What’s it mean? And how do people start on that journey?

Bryan: Innovation is really, really important. It’s what makes the world turnaround. And the question that comes up for me time and time, again, is how can you be more effective in innovation? How can you be better, faster, do it easier. And I came across AI about a year and a half ago. And I’m just amazed at how much better things can become. And my personal take is trying to find the facts that back it up where it works, and where it doesn’t work.

Ross: So let’s say as a starting point, go to an executive team, they say no, right? Innovation is important to us. We’ve got some processes now, what would be the first steps in being able to introduce AI into their innovation process?

Bryan: I think, you know, for me, the challenge is to have two bits of knowledge, one knowledge around innovation, what is innovation? What are you trying to do? What are the facts? And then secondly, its domain expertise in terms of AI, and you have to bring the two of them together. And the first and foremost, most important bit of knowledge around innovation is AI works when the system works. It’s a system driven process, we tend to look and say, no, no, it’s those people. They’re not being creative. They’re not doing what they want. But if innovation is not working in your company, it’s what you as management is doing. It’s not what your people are doing wrong. 

And the second thing is to look at innovation as a process. I mean, it’s, you have to do a lot of things, right. It’s not enough to do one thing, right? And everybody seems to focus on idea building. But idea building is actually the easiest part of the innovation process. And I think what I would look at is ways that you can use AI to get aligned better, that you can communicate better, that you can pitch better, that you can evaluate ideas better. And there’s lots of cool stuff that can be done there. And I hope we get a chance to share some of the cool stuff we’ve been doing around this and in the next few minutes. 

Ross: Absolutely. Well, I want to dig deeper on a lot of levels. But I mean, to ground this, can you give a specific example of how AI has been introduced in being valuable in the innovation process of an organization?

Bryan: Well, you know, everybody looks at ideation as the core of innovation. So I’ll start there. That’s because that’s what people are interested in. And there’s a lot of research showing that if you take a typical person, you give them AI to use to become more creative and more effective. And the impact is biggest on the lowest or lowest performing people in your organization. And sort of it’s an evening out factor. But what people forget in the process is you have to use it effectively. And we just completed some research, we evaluated 5400 ideas generated by humans and AI. And we found a few things that came out. One is AI, on average it doesn’t build better ideas, AI builds a lot more ideas. But when you take AI and humans intelligently together, your hit rate goes up, and your hit rate goes up amazingly. 

And hit rate I mean is what is the percentage of really good ideas. So if I look at an average human, we hit one out of 100, out of the park. If we look at a human’s plus AI, without any training, they hit about one and a half to two. But if you get humans plus AI, and some good training, you’re hitting somewhere between five and seven. So this is five to seven times more ideas that are really good ideas. And that’s a big difference. 

Ross: You said something to the effect of bringing together intelligently, humans and AI, and also about the training. So what are the ways in which you can bring humans and AI together? What are the sort of steps or processes or where that’s done? And if what is true, what’s the training, which enables people to do this better?

Bryan: So I think if I was going to start out a training, I would look at the place where innovation fails most often. Innovation feels most often because you’re solving the wrong problem. And in fact, AI can help you an amazing amount of trying to identify what is the right problem and defining the problem correctly. It’s and the advantage you have with AI is it can give you sort of a naive viewpoint on your problem that you didn’t think of before. And you know what I see there is an incredible operator to start at the beginning. And just to put a framework here, I talk about innovation in terms of the ABCs. It’s ABCs, because it’s easy to remember, maybe your listeners remember, it’s to align, build, communicate, check, systematically improve. And at the end of this, you’ve got, you’d have to pitch your ideas for. And what I look at is ways that AI can be used in the whole process more effectively. 

And the first part is alignment, figuring out where you want to go. The second part is building ideas. And by the way, this is the easiest part of the innovation process, but AI can really knock it out of the park. And for me, you know, the superhuman characteristic of AI right now is the ability to evaluate ideas. I just talked to you a second ago about a study with 5400 ideas. Imagine what would happen if you went back to your company and said, you have to evaluate 5400 ideas. AI is not going to complain, your people are gonna complain a lot. And the other thing that comes up within the innovation process is if you want to get ideas moving forward, you have to present them effectively. And I actually do a really good job of taking human ideas, and making them better structured, and easier to understand for other people. And then lastly, you know, the part at the end is, how do you get these things in the world? How do you look at where the weaknesses are? And AI is very good at identifying system problems, to where things don’t fit together? So that was a long answer to a simple question, Ross.

Ross: Well now, what is the structured one so that it makes it cognitively tractable? So the first step was a line. So the way I put it is framing. So when you start off with a human plus AI process, the first step is framing your objectives, challenging the context. So, when I usually say that is mainly a human role, because the human understands the context and the objectives and the frame? And so what, how do you go about the Align process is that mainly a human in terms of defining the terms of the engagement as it were, or AI and humans work together to be aligned to ensure that when you are building ideas, and assessing them, and so on, that they are in the right context?

Bryan: I wish I could answer this completely. It’s sort of a moving target right now. I’ve spent the last six months building an AI for innovation toolkit. And within that AI for innovation toolkit, there’s 40 tools around alignment. And those tools are typically around market scoping. You know, what is the market we’re in? What are the needs? Which are there? What are the jobs to be done? And what you’re trying to identify as, first of all, are you pointed in the right direction, creating a product that somebody wants? And then at the end of the process, are you communicating where you want to go in a way that your team can understand? And one of the challenges that comes up, I can only identify qualitative ways that AI is helping in this process. But I’m finding it really hard to identify quantitative waste, how can you prove that the team is more aligned. And if you have some suggestions for me, I’m all in for it. But I think what’s important here is what you’re talking about is the framing of the problem. If you solve the wrong problem, you get the wrong solution. And AI can be your friend and identify what solution you should be looking at. And how do you get yourself pointed in the right direction? So I mean, to answer your question here, it’s hard. The alignment heart is a heart heartbeat. And I do think that remains the domain of the human. But the domain of the smart human is to use AI to augment what they’re doing, and to think more effectively. And, you know, for me, typically, when I do an alignment project with a company, it was taking me a day and a half to two days before. And now with AI, we’re doing it in a half a day. So it’s not just qualitatively better it’s quantitatively faster along the process.

Ross: So as you said before, around the ideas and the idea of filtering, you can generate an unlimited number of ideas. And you can use AI to assist you in the filtering and you know, some kind of ranking or assessment. They’re also in that aligning price as you’ve talked about things like market scoping, or adjacencies. And there’s a whole lot of useful prompts or tools which you know, really are about looking at what is the position of the organizations? What are adjacent opportunities? What are different ways of extending current capabilities, for example, but again, we start to often be quite verbose, or again, proliferation of ideas. So let’s say we’ve got a strategy team and the building, you might want to bring more people into that process as a broader brainstorming ideation process. But in the aligning, it’s probably a bit more efficient to have a strategy team or executive team. So how might you then use some of these tools to generate some of the scoping or alignment or framing issues, and present them to a strategy team so that they can efficiently and usefully get a better understanding of where they are? And where opportunities might lie?

Bryan: I think, within any team, what you need is diversity of viewpoints and different perspectives. The one thing you can’t ask a CEO, at least not very effectively, is why is your company going to fail? Or why? Why is this project going to fail? And one of the things that I see very powerful in using AI is to be that questioning that Doubting Thomas within the room and saying, ‘Look, are we pointed in the right direction, find me holes in what we’re doing.’ And nobody wants to kill the AI. But if this was a human person in the meeting room, they certainly would have all the weight of the world on their shoulders that they’re talking about all the reasons that things would fail. But I would say, for me, the role of AI, it’s sort of like, you know, when you were a kid, you had this telescope, and you’re sort of trying to get it more clear view of where you’re going in, it’s what you’re trying to do is bring in more light. And as you bring in more light, you know, the direction you’re going becomes clearer and clearer. It doesn’t give you the answer, because you’re still left with a human to interpret what you’re seeing. But it certainly adds clarity and depth, and features that you might not see within your typical management team. And the answer here is using AI just to be smarter than you were before, to question better than you did before. And, you know, I guess for me, one of the things that came up is my story with AI. I just didn’t believe it could do these things. But the proof is in the pudding. When you actually start using it, you said, wow, wow, we can do this that I didn’t expect. So, you know, I think the process of using it is to add clarification and ask the tough questions that a lot of humans don’t want to ask.

Ross: That’s really interesting. So I’ve, I usually frame the devil’s advocate, the red teaming challenges and so on at the end. So you might have a strategy or you might have an investment decision or some kind of decision, which you can then you know, AI is extremely useful, as you suggest for being a, you know, hopefully not emotionally – It doesn’t create emotional responses, but can actually, you know, quite succinctly and well articulate the reasons why a particular decision might be wrong. But I mean, you’re very interestingly raising this idea of saying, well, take where we are now, in our current position, and to challenge that as a starting point, rather than simply being able to look at that as a set of challenges at the end of the process. 

Bryan: One of the things that I’ve seen is very effective use of AI is, you know, if you have a management team, and for example, you have them do brain writing. So, you know, in a very effective part of the innovation process to have people get ideas off their chest, and you just have them all type games, what are we focusing on today? What are we focusing on today? And then you feed that all into the AI. And you say, Aha, try to imagine what is John thinking, what is Mike thinking, and you look for the common themes. And it’s, it’s amazing the speed which you can go through, and it’s this process of interaction where you have people doing deep thinking and then the AI summarizes and synthesizes it in a way so that people can even think deep. So it’s not letting me think for you, but it’s helping you in the process.

Ross: Yeah, I’ve often used recording so basically get a group together, record it, and then immediately transcribe it and do an AI summary. And so people can basically see what it is they were talking about live or potentially pass that on to other groups to bounce back. And so that actually if you’re doing that live, it can be a very powerful tool for, for being able to, you know, distill that but one of the other points coming out of what you’re just suggesting is this idea of clustering or categorization. So classically, you stick to posting it up on the whiteboard. And you work out where you know how they make sense together. But that’s actually something which AI can be very useful for, as well as be able to say, well, here are different ways in which we might cluster these different ideas and how they might emerge with some kind of sense of the idea of scope we’re generating.

Bryan: I think, the clustering of ideas, and also the identification of divergent ideas. Because if everybody’s thinking the same thing, a lot of people don’t need to be there. And what you need to do is capture, what are the divergent ideas in the room. And, you know, go down that path for a bit and see where you’re headed. And everybody gets smarter along the way.

Ross: So you’re, you’re suggesting that AI assesses how divergent ideas are as part of the process. 

Bryan: I think one of the things that comes up is there is a degree to which you’re looking for convergence among people. But you’re also looking for the outliers. And you’re trying to say, who has a different point of view? And then you ask, ‘Ross, why do you have this point of view? Why are you seeing this differently than the rest of us?’ And, you know, it might not come out in a typical conversation, because we lose some of those nuances when we’re listening to one another. And we tend to hear what we want to hear. And yeah, AI has an ability to actually hear what we do here, what we’re really saying a lot of times more effectively than our human ears do.

Ross: You one of the tools I really like doing with boards is to use live polling, to identify the range of opinions. And you can see, well, are they clustered? Or who’s on the outside? And then you can sort of say, well, who, who has that very different opinion to everyone else and why and that’s a really great way to surface conversation.

Bryan: But I think, you know, what happens is I see innovation as a process where you go from step by step, and the starting point is alignment. And alignment is defining the problem. And if there’s one thing that’s proven within the innovation process, and proven with the creative process, the most important part of getting good ideas is defining the problem. If you don’t define the problem, right, you get the wrong solutions. And finding the problem, right, actually can define half the solutions that you find. And I think, using the clarification, using polling, using different things that bring ideas out, you get much better than you would just sitting there by yourself. 

Now, the challenge that comes up there as a human, especially if you’re the consultant in the room, I mean, do you really want to give away the power to AI at that point in time, because you want to be the smartest person in the room? Or at least I think most consultants do. And, I think there’s a certain degree of humility, and realizing, in fact, that this buddy in your room, this extra team partner, which is AI can do certain things that you might not be able to do more effectively.

Ross: Absolutely. Right. And so I actually say I don’t do consulting, I facilitate. And so I facilitate people and now facilitate humans and AI together. So it’s facilitating the process. Yeah, as you say, it is a process. And I think, the role for people like you and I and our peers is to how do you enable that process of smart people together with extraordinary tools to flow in a way because you can’t map it out mechanically you know, it’s an enabling process.

Bryan: So Ross, I know you’re the one asking the questions today. But I’d like to turn it around for just a second, where have you seen the biggest impact in AI getting teams aligned?

Ross: Getting teams to work together better?

Bryan: Working together better, but also just to decide what they’re working on?

Ross: Oh, well, in terms of working better together? I haven’t actually seen that. That’s a really interesting use case. I mean, actually, there was an interesting study recently where I was generating, basically being contrarian and arguing other cases. And for some people, it was useful. And it’s a little bit similar to the cognitive abrasion, creative abrasion with people where it can be useful to have different opinions. And I think there may be a role for AI in that, but I haven’t personally experienced that. But the way, I suppose the starting point would be seeing that I do start with that human frame from the beginning whether humans do the frame and start off by saying this is what we wanted to embark on. This is our context. And even just a preliminary definition of that. And that’s when you start to get other opinions or perspectives. And actually one of the most useful ones I found is that creating diverse perspectives from AI, so to generate a whole array of different personas that are relevant or possibly even, not even relevant, to be able to create things and that just being part of that journey, but my I practice has always started with the human, and then be able to get in those other perspectives or other ideas or framing from AI.  

Bryan: While you were talking, I was thinking, Where do I see the biggest impact? I see it as management usually thinks what they’re presenting is very clear. And one of the things that I see is very positive is AI can come back and say, This is what isn’t clear. And to give them and feedback in a non confrontational way, to make sure that what they’re explaining to their teams, or their marching orders have to be very specific and clear, if they hope to get people marching in the right direction. And I think that might be, at least in my opinion, where AI can be used most effectively. clarification.

Ross: Well, that goes to the point of AI as interactive agents. So you can have a lot of single prompts, lots of people, you know, sharing all this prompt for this and prompt for that. But you know, often, if you guide the AI to be interactive to ask, keep asking questions in detail until it has enough information and feeding that back. That is one of the most useful things and it can feel frustrating, you don’t want to set up an AI which asks you these questions which take forever to answer. But I think this dialogue partner has been able to say, these are the missing bits of information, can you feed that in? And that actually the human thinking that through to answer those questions, as you say, it can be one of the most powerful engagement techniques.

Bryan: I think the multiple prompts are actually quite smart, because it forces clarification on both sides. And, I think a lot of times, we don’t think deeply enough. And what you’re trying to do is trying to find ways to think more deeply and more effectively. And this gets you there quicker and easier.

Ross: So you mentioned before this idea of evidence, looking for evidence and variables to find the best path for AI innovation would love to know what you’re thinking or processes or approaches to be able to build evidence into creating better. 

Bryan: That’s a good lead in because that leads me to three things that I’m doing right now. One thing we did, is we did a lot of work generating ideas with AI and humans ai, ai alone, humans alone human plus AI. And what we found evidence based, is in fact, there isn’t much difference in terms of the average quality of ideas. We found out in fact, humans are more novel than AI is. And you’d say, well, you know, at this point in time, you should give up on AI. But in fact, what we found is something else, which is more important. And in fact, in the innovation process, what you’re looking for is outliers, you’re looking for great ideas. And what we did is we rated all the ideas generated and identified top 5% ideas, and what is the percentage of time that an idea falls in the top 5%. And so we have evidence based information showing that in fact, humans plus AI in an interactive process, like you were talking about, generate somewhere between five and seven times more top ideas. That, for me, is an important finding. 

The second thing that we looked at in terms of evidence based work is a mount. For me, everybody looks at the hero as building ideas. But when ideas become really easy to generate, the hero now becomes your ability to evaluate ideas. So if you can imagine we have 5000 ideas, who’s going to evaluate them? And is the AI any good? So we ran a second set of studies, looking at human evaluations of those ideas versus AI valuations of ideas. And what you can find out first of all, is a lot quicker, that’s not a big surprise. But more importantly, AI is much more reliable. So you present the same idea twice. Your odds of getting a different evaluation from a human are dramatically higher than with the AI. But the thing that came up most interesting for me is in fact, if you use a collection of AI personas to evaluate the ideas, the AI personas take sort of a middle ground. And they’re actually a better representation of the whole universe of evaluation than any single human. So in fact, the humans are less correlated with one another than the humans are correlated, meaning that AI is not only a fast way to do it, it’s a reliable way to do it. But it’s also a more valid way to do it. 

And the last thing that we’re doing research on and this For me, it’s because I’m a teacher, I teach courses. And at the end, of course, my students do pitches. And I’ve suffered through so many horrible pitches. And my best estimate, we were at a 40 to 50% fail rate on the pitches. And I hope that’s not an indication of me as a teacher. But in fact, what we tried to do is we tried to find ways to improve that pitching process. And we’ve now put AI in it. So we use AI, to help people think deeply about their ideas to make a first version of their pitch. And then the humans have something to work with, it’s no longer a blank sheet of paper, they work on it. And then we created a second tool, which evaluates those pitches. And we’ve mined from a 40% fail rate to a 5% fail rate. Now, if you can imagine companies doubling the effectiveness of your communication, that’s an amazing thing. Now the question is, how do you present this quantitatively? And I do a presentation in June, we actually have to present the results. So I have to start writing quickly.

Ross: Fabulous, no, really looking forward to that. So to round out, let’s say, Yeah, whoever was listening at the moment already understands AI innovation has embarked on that journey at the start, you know, they’re familiar with the ideas we’ve been discussing at a high level, and they are beginning to get going. So what would be your advice to them on how it is they can refine and improve using AI and innovation?

Bryan: I think one of the things you have to realize is the perspective in which we see things most of the people listening here are really positive about AI. And you have to realize a lot of people are not very positive about AI. AI, people don’t want to see AI as the star of the show. And in fact, AI should not be the star of the show. In fact, the biggest advice that we give is to say, look, AI has an amazing amount of limitations, horrible and a lot of things. But when you use it smartly, it becomes an important part of your team. And as an important team member, your team gets better. So I would say my key advice to you is look at how you work together to deliver better results. And don’t tell anybody that AI is doing the job. AI is working with the humans to do a better job.

Ross: Yeah, I think that’s really important. The cultural aspect of how this works is fundamental.

Bryan: And it can’t be the star of the show. It has to be another team. Actually, let me conclude with something that I found was. So before here, I used to work on group dynamics, how do you make a group effective. There is a guy named Scott Page who did a lot of research on diversity and group performance. And he found a really important finding. In fact, it’s not the talent of a group that performs group performance. But as the diversity of a group. And to test this, one of the things I did in my academic studies is we took the best students, and we added the worst students to the group. And you would think, what’s going to happen, it’s not going to help them at all. In fact, when you add diversity, even if it’s somebody less smart, even if it’s somebody less good, the results go up. So what you can do is say look, AI adds diversity, it’s a different type of team member. And what you need to do is round out your team and get the full team together and put another person at the table and everything will go better.

Ross: That’s a great way to frame it at the highest level. I think that’s really important. So Bryan, where can people go to find more about your work, I think you’ve got quite a lot of things you share.

Bryan: I share a lot of stuff with you. The easy place to find me is on my LinkedIn. Bryan Cassady, I’m the only one there. In fact, there’s two of them there. One is my old profile, which I forgot the password for. And the second one is my site, which is www.bryancassady.com. 

Ross, I really enjoyed the chat today. I learned a lot while you’re asking me questions, and I appreciate your time. And thank you very much.

Ross: Now, it’s fabulous that you’re one of the people out there pushing the potential for AI and innovation. I think we’re a growing community. So thanks so much.

Bryan: I would like to end with the last thing we have. We are looking for beta testers for our AI renovation toolkit. If anyone contacts me on LinkedIn and sends me a direct message, I will send you back a link and you can try it out. There’s Jordan 65 tools and six apps and it does some amazing stuff.

Ross: Fabulous. Thank you, Bryan!