Diegetic Prototyping and its Impact on Future Design

“Read broadly because your uniqueness will come from the corpus of information your brain has trained on.”

– Kes Sampanthar

What you will learn

• Navigating the confluence of artificial intelligence and human empathy
• Augmenting human potential in the age of Generative AI
• Exploring the paradox of generative AI in creativity and competition
• Shaping the future with diegetic prototyping
• Reframing competition and innovation in the AI era
• Unlocking the synergy between human creativity and AI
• Decoding the architecture of thought, from cognitive blueprints to AI applications

Episode Resources

ChatGPT
Deep Blue
ChessBase
Stockfish
BCG Harvard Research
LLM
Netflix
YouTube
Roblox
GitHub copilot
Amazon
Alibaba

People

Marvin Minsky

John McCarthy

Usain Bolt

Garry Kasparov

Magnus Carlsen

Stanislas Dehaene

Jeff Hawkins

Charlie Munger

Andy Clark

Book

Thriving on Overload: The 5 Powers for Success in a World of Exponential Information by Ross Dawson

Transcript

Ross Dawson: Kes, it’s wonderful to have you on the show.

Kes Sampanthar: It’s great to be here. Ross. Thank you for inviting me.

Ross: So tell me Kes, what is a Centurion?

Kes: A centurion is somebody who uses AI to augment their ability to think, their ability to work, their ability to engage with the AI, to, what I call ‘augmented intelligence’, and in a way that we have sort of been slowly evolving our brains. Now, as we hit that next sort of stage of what I see is the next stage of evolution.

Ross: I very much agree. So, how did you come to get here? You know, just in a nutshell, how did you come to be focusing on this very important topic?

Kes: Long journey, like many of us. So I actually started AI research 30 years ago. So, I was doing neural networks, genetic algorithms, parallel computation, as part of sort of academic research, and then I lost funding, and ended up going to a think tank, and consulting, starting a number of startups that dove into neuroscience over the last 20 years. I really liked the fact that the 90s, that led to the decade of the brain, ended up developing a behavioral design methodology called motivational design. And then, over the last sort of decades, slowly, actually getting back into AI, and starting to use it, obviously, as machine learning started evolving, data science started exploding. And then, most recently, what I really loved when ChatGPT finally got to that stage was, I realized that we were as close to what I’ve been envisioning for a long time, at the same time, the idea of AI, which I’ve been looking at for a decade now. And then with hands-on access, hands-on sort of experimentation with vision Pro, and I realized that they’ve solved a lot of the problems I’d been identifying. So this idea of augmented reality meets augmented intelligence was where I thought, I’ve been waiting for a long time. 

Ross: A lot of people, when they watch, are really focused on what people are searching for in AI, and seem to be saying, ‘Well, how do you make the AI better?’ There’s a relatively small number of people who ask, ‘Well, how does the AI make humans better?’ So, what is it about you that makes you focus on that?

Kes: At some level, it’s like, it really is like, computing went off in two directions very early on. So when I first started, you had Marvin Minsky, John McCarthy going on AI. And honestly, when I was younger, that’s where I thought, I wanted to be like AI research. And I was very excited about neural networks. But I slowly started realizing, to understand how to create AI, I started studying neuroscience and human behavior. And as I sort of started growing up, I realized that it was very important to focus on humans. So, I’ve spent the last 20 years really on human centered design, behavioral design, how do you ensure the prosperity of humans going forward. We’re very unique species in a lot of ways. And I realized that we’re the first species in evolution, who not only can, have that intellect, which allows us to understand things. We’re the first most empathetic organism which cares not only ourselves; but, other living systems and the universe at large. 

So at some level, it’s like, I wanted to make sure as we move forward, that, we augmented humans and brought them along, because, keeping this arc of progression going, so I felt like too, as much as it’s, it’s like, there’s a lot of people who will come to math and science of AI, which I love. But I wanted to focus on the harder piece, which I thought was not getting covered enough. As much as I love user experience designers and people, human-centered design. They don’t go deep enough into understanding how the brain works, or they don’t understand AI at a deep enough level to actually understand how to create the interfaces. So being at this unique intersection of deep AI, plus deep human insights. I thought I was in a very unique position to actually help in this sort of intersection, and how do you design this future interface.

Ross: Yeah, well, I’m very much in favor of humans and their potential, which was still far from expressed. And now I’ve got some tools to help it. So I actually started in computing, my career in computing went off and some other directions, and actually got quite a lot into cognitive psychology. And then at one point, I decided that I should describe myself as a born again, technologist coming back to the technology, because the tech understood the technology was, in fact, what we needed to augment and connect and catalyze humans. So yes. So there’s the idea of central I mean, of course, that’s, that’s an individual concept, as any other as an individual centaur. But this could also be applied in organizations. And I presume that working in a think tank as you do, then that’s probably part of your scope, as well, thinking about organization. So that person might start with organizations then come back to the individual. So how should organizational leaders be thinking about bringing this centaur into their organization, making it a centaur organization?

Kes: One of the things which worried me, especially over the last year, is the way large consulting companies and large organizations and some certain CEOs were talking about Gen AI . They were talking about it very similarly to robotic process automation, this idea of automating more and more of our work. Since the start of technology, we’ve been focused on automating things away. Whether it’s the loom. how we evolved agriculture, how we created factories for manufacturing – it’s been a long journey of turning human labor. And, instead of using our muscles, being able to use technology to be able to do some of that physical labor. Over the last sort of 50 to 60 years, we’ve been using technology in the sense of IT, to be able to help, what I would say, sort of, like, do more cognitive work, right? So, mostly, it was low, low sort of cognitive work, low level cognitive work. And we automated away a lot of those jobs. And we’ve done that, but at the same time, we’ve increased and improved the kind of jobs which we evolve to everybody else. 

I started my career, the idea of being a innovation consultant, human-centered designer, even an AI, like, was not really heard of except out of academia. I wanted to focus on helping organizations, be able to do something beyond just automating away. Because there’s a lot of talk about this idea that, we don’t need humans anymore, right? You know, where we’re basically heading further equivalent, that the pastures the same way horses were put out to pasture. When we invented the car, we didn’t need horses anymore. The idea is, as soon as the AI can beat a human at something, eventually, it’s going to get to a point where it can do nearly every cognitive test better than a human. At which point why do we need humans, but I look at it differently. Like, it’s not like when we invented the car, we got rid of running or sprinting. Usain Bolt still runs and doesn’t compete against Ferrari. Like even chess, righ? Kasparov lost to Deep Blue, nearly 26 years ago. It’s still Magnus Carlsen, who grew up in the era where AI had already beaten humans. Learn from technology like ChessBase and using some of the stuff which AI drives like Stockfish. And to actually improve the game to the point like he is a better player because of using technology. This idea of organizations, augmenting their employees, because I believe that will help us get to further. So instead of looking at just automating, and just getting rid of tasks, I’m really interested in what I saw with large language models was it  impacted and could augment high level thinking tasks — experts. It’s like looking at example, in a day insurance industry is like, somebody’s an underwriter. Do we just get rid of all that expertise that they have? Or do we find a way of augmenting them to be able to do things they couldn’t do before? And that’s where I’m sort of moving towards. So I have this model of autopilot, copilot pilot where there’s certain things we want to augment away, automate away. Other things we want to augment. So we want to be able to think more broadly. And then lastly, it’s like what are the things we can never do before? And what can we actually enable? So if we’re looking at competitive advantage, we’re looking at how organizations grow and compete and stay relevant. The danger of AI is, you’re going to end up being a race to the bottom, like, at some level; how do we get cheaper, faster, better. The AI systems which are out there, I think there’s an opportunity to play to a more infinite game of how do we actually augment what we already do to actually get to something even better.

Ross: Well, that’s certainly very aligned with what I how I communicate with boards and executive teams is this idea. This is not about replacing, not about getting with people, it’s around saying, how can we now use all the people you’ve got to be able to create something far more than you could before? But what is your response? What response do you get from your clients or from executives, or leaders? And how do you communicate with them in a way that helps shift their thinking on this?

Kes: One of the ways we’ve been trying to explain things to them. We bring data and research. I think you are familiar with the BCG – Walter H – Harvard research on creativity, where it showed that having Gen AI actually got a 40% uptick in the quality of ideas, especially on the creative tasks. And that’s been sort of touted quite a lot. What they don’t talk about as much is the flip side of that same research, which was, there was a 41% drop in diversity of ideas. So the first thing I tell to executives, when I talk to them is, ‘Look, you could, jump on this, get rid of, whatever costs, you’re paying to a marketing agency, your creative, your innovators, and outsources and go, Hey, I can just have an intern using Gen AI, to be able to create some of these ideas.’ But if you believe that innovation and creativity are the only thing which helps you differentiate in the marketplace, and helps you create a competitive edge, which is the whole essence of innovation, to drive growth; you should be terrified that you are now playing on platforms held by a handful of tech towns, where everyone’s going to have access to this isn’t a competitive edge anymore. So how do you augment with humans and bring what I call the personal LLM; the expertise that humans bring to the diversity of thought that they bring together to be able to get to something which is and which is so much more than an either or right, you have human plus machine, I think is human and machine, I can’t remember how you frame it. But that’s that plus that, and is the most important thing, because I think that’s how you get to compare advantages. So that’s one part of the conversation. 

The second part is we create diegetic prototypes to help organizations understand how competitive advantage plays up. So we started with something in the media industry where we actually showed them right, hey, look, you’ve just lived through the streaming wars. You know, I was talking to media executives, 15 years ago about Netflix, and streaming, and the fact that their value chain of distribution was getting disrupted. And they should be very wary of giving that IP to a company like Netflix. They just thought it was an extra channel to make some money on that IP. Roll forward five, six years, and everyone, every one of them is scrambling to create a streaming platform. So we showed them that Gen AI as it is, as of last year was already lowering the cost of production. Production used to be sort of what I call them moats, right, this was their competitive advantage. This was like, ‘you need a lot of money to do high-end blockbuster movies like them to use.’ And that’s what kept everybody else out. You just load the cost of your most expensive set to the point, but at the same time, you’ve got the rise of social media and YouTube and Roblox, and suddenly this younger generation is already consuming content, which you don’t control. And now this technology comes along, which allows anybody to be able to correct. You should be worried about the teenager in our bedroom, creating the next Game of Thrones series, moment by moment for our friends, and then, reating a franchise and a universe far bigger than the MCU. Like, how do you find that? How do you tap into that, as opposed to like, trying to lock people out or sue people for using content, like, we should embrace the fact that we’re moving into this new world? So at some level, we use diegetic prototypes that will really hammer home what is your competitive advantage going to look like? We use a business model. Basically show them how it’s going to get disrupted using sort of the sort of technologies and an adjacent possible library which we develop. 

Ross: So, a couple of follow up questions. The first one is to use the word diegetic a couple of times. Could you please explain that?

Kes: Oh yes. So diegetic prototyping is, it’s like Bruce Dolan called it sort of design fictions. So these are sort of like, a very intentional use of a design to be able to explain how something will really be used in the future. So movies have been doing this for a long time. So, minority report, all the technologies, which are supposed to be 50 years out, was sort of imagined, and all of them became available within the next decade. They were commercially available products. So one of the things I realized is the innovator is like, instead of going MVP, where you just create the minimal viable product, what a diegetic, or design fiction prototype is, lets somebody imagine exactly how this is going to be used, how it’s actually going to implement and tie together with their business models and business plans, and how does competitive advantage work out, right. So it is like the same thing as, let’s take Star Trek, or let’s take them to you, but then let’s tie it to what the implications are for your business. So we bring this idea of these design fictions to life because we find that that gets the idea across so much better, like science fiction writers. Inspiring scientists and technologists, and now businesses, and now having a methodology to be able to develop that is one of the things where we can help organizations really understand the implications.

Ross: So the other thing is competitive advantage, which we’ve raised a few times. And so you’re absolutely right, if you have everyone using the same LLMs, there’s no competitive advantage. So if you use humans plus AI better, that is a competitive advantage. But what’s the nub of that? So how, what is that future competitive advantage of a, call it a ‘centaur organization’? What’s that look like? Where does the competitive advantage reside? And sustained? 

Kes: So, it changes from industry to industry, but sort of more generally talking about it generally. Right? So if we talk about, basically, where to play, how to win. So where to play is your competitive sort of feeling like where are you going to actually take it? So this is where innovation comes in. So not only new business models, new products, new services, even down to creativity and advertising, andhighly break through the noise. So at some level, to be able to stand out and differentiate, you need to have an edge over everybody else. So your innovation engine has to be powered in a way which is going to get to uniqueness means as we as I sort of unpacked a little bit. And what I found in the Centaurian articles was this idea that the challenge of an LLM is associative thinking – associative thinking, and that stochastic process, which actually works to actually bubble these things up. So when you look at the sort of semantic space of ideas, which in our loans basically users, it hones in on things which are associative, and gets to sort of slightly better than average ideas. That’s a challenge when you need to be at the higher end, especially as you think about innovation. 

So this is where somebody who has a broader range of knowledge about their own personal LLM, and together with AI can push the boundaries of how you use a large language model to actually explore parts of that semantic space, which is not as easy to get to. This isn’t just standard prompt engineering. This isn’t just a train of thought, this is actually understanding how our brains come up with ideas. How do you actually explore the knowledge and ideas space, and then understand how to use a technology like AI to be able to get there. So that’s sort of like where to play, how to get competitive advantage, how to build competitive products and systems. On the operating model side, it’s like, how do you actually look at this and go, yeah, there are things I have to automate. But there is also this idea of like, how do I find a different way, a different sort of operating model. 

So we go back 100 years ago, when we moved from steam power to electricity. All people did was, in a factory where the steam used to come in for a central shaft, which ran everything, they plugged in electricity and not steam. And it took another 40 years before somebody realized electricity doesn’t have to be located at just a central source, it can be distributed. So by the time we get to assembly lines, and the distribution of electricity, you’ve got a new operating model, which actually takes advantage of the technology. I feel like similar sorts of things have happened in the operating model. It took us 20 years to actually walk through the internet. And the advantage that computers and the internet came together to create a platform business model. The companies like Amazon, Alibaba, all of these companies, at some level use this very different kind of business model. So if we thought about mass production and the 20th century as economies of scale. What the internet provided was a business model, which was economies of scale through this idea of network effects. So that’s what drives a platform-ism. But that took a long time to get that to the point, Bezos took 13 years to get there, Alibaba launched with that sort of business model, jobs, fought it all the way and eventually fell back into creating an app store when he didn’t really want to have an open system and open platform. And  , Google kind of stumbled into it as well and didn’t really understand what they had. But today, we understand how platform business models work, how competitive advantage works, and that, well, I think AI is going to drive a similar kind of remodeling of operating models, when we truly understand how that’s going to change our business system.

Ross: So which case competitively resides, advantage resides, be able to adopt that operating model of humans plus AI and the structures and what that implies, but also being able to develop the skills and the enabling culture of the individuals in the organization so that they can use those tools effectively to augment themselves individually in an organization.

Kes: Yeah. I mean, one of the things you asked from the start Ross, and I’ll try and get to that, is it to be a Centurion to be a Centurion organization? It’s not as easy as you think, in the sense like, we’ve seen, we’ve worked with a lot of organizations of the last year, and a lot of them, even the ones who are using generative AI, and not really changing how they work. To the point they’re outsourcing. Thinking about the tool, right? Oh, my God, I’ve heard the concept like, oh, it replaces the fear most people fear of the blank page. Like that’s the wrong thing to use. Like it isn’t prompt for us, AI isn’t for us, it’s actually thinking first before you get there, because that’s the uniqueness you can bring to the tool. 

So as I’ve been looking at this going back into neuroscience and cognitive science of understanding how intelligence and how we think it changes, how you would actually use the tool, you need to understand kind of like how an LLM works, how AI works, and what it does well, and what its weaknesses are. And then you have to understand how a human thinks, and I’m trying to bring those pieces together, and it’s not natural. People talking about copilot, GitHub, a copilot for a program. I’ve been programming since I was 11 years old. The reality is I have to change so many of my programming habits to be able to use this. And I can already start sensing that is changing how I’m thinking about programming. And that’s just one sort of cognitive test. That’s why I’ve been sort of really breaking down something like chess, because that’s easy for people to understand. And the idea that,  , this AI is going to change how we might sort of think, because chess is this sort of very visual way for your thoughts in an open game, where you can actually see where every strategy and thoughts are going at some level. But we don’t get to see the unconscious. And that’s the bit I’ve been trying to unpack a lot of. 

Ross: I’m absolutely very, very much coming back, going back to my cognitive psychology as well as this wave. How do we marry these things? Well, alright, so the next little while I just want to get as practical as possible. All right. You’re advising somebody how to be a centaur? What should they do?

Kes: First step is to learn what makes you unique as a human right. So the best thing I advise people is read broadly. Because your uniqueness is going to come from that corpus of information your brain has trained on. So because that’s what’s going to add new information. Because if you’re trying to get to the edges of the semantic space, you’re not going to get there unless you’ve got a broad and unique set of ideas which you can bring. So desperate reading is going to be the order of the day. Yes, many of us are reading paper after paper off trying to understand AI but at the same time, good broad, find the things which are interesting. So like training your personal LLM from us outside of what Epstein calls ‘range’, right? There’s a broad knowledge base, which you have to get to. That’s what’s going to help you personally both in your career and your jobs and how to engage.

Next step is to understand how do you how do you think, how do you create what is creativity in the brain. To understand how that process was how does incubation work and has come into play where some of these things which are really pushing the boundary of creativity. To be able to understand like, that’s what I need to bring and bring into. 

Then it’s like bringing understanding frameworks and understanding the frameworks which underlie our thinking, to be able to understand how to drive better Dramework Design Thinking, better systems thinking into the cognitive tools that we use. So those are the three steps which have been set out to help organizations, and then individuals to be able to really think through when they look at a process. You know, it isn’t just workflow and process. It’s thought flow and thought flow comes down to both conscious and unconscious processing. And how do you actually do that? So those are some of the pieces where I’m helping organizations try and think through this.

Ross: Yeah, everything you say is very, very hard with my work. It’s just one thing to pull it up, like to dig into it as you’re talking about the frameworks. And so the second chapter of my book, Thriving on Overload was framing. What are the frameworks within which we build out knowledge. So I’d love to just hear more about the frameworks in our thinking that support us becoming better centaurs?

Kes: Some work by Stanislas Dehaene has been mapping out sort of core cognitive structures in our brain, which are the core frameworks of how we think. So one is sort of this idea, they call it a number line. But it’s also used to understand time, so we can look at two things, quantities, and be able to look at what is more and what is less. So that sort of timeline ability is embedded in our brain. 

Another piece is there’s a tree-like structure, so there’s sort of decision frameworks. So that’s why trees work, right? At some level, even the term the classic consultants two by two is a small glimpse into tree-like thinking but visualized in a different way. Then you throw in the last sort piece, which is our brains literally evolving. Our neocortex evolved from a mapping system, which early mammals used to have or even other creatures, but this idea of placing grid cells, so it’s best to sort of map out the physical world, and we could map it. So whether you’re a small mammal like or a rat running through a maze, they’ve seen it in dreamlike sequences where they can actually see the hexagonal grid and how the maze how the rat moves through the maze, in that sort of place in grid cells sort of way. We’ve got that embedded in our brain. What we did when we evolved, the neocortex, and this goes into Jeff Hawking’s work on in 1000 brains, is we turn that into, like 1000s of these columns, which have at their heart, this sort of mapping structure. So now we map information in this sort of place, and grid cells, but through abstract space. 

So understanding those three, allows you to look at all the different frameworks we’ve ever created, and be able to actually understand how we can understand information. Then the other side of meant frameworks is the work which Dave Grace doing, which is more than metaphors. And that sort of sets in this idea of visual metaphors, which allows us to think about things in different ways and framing ideas, and I always push my teams to go, what’s the metaphor, this reminds you of? How do you think about this in a different way, is this like,baseball is this, like a very different kind of organization, franchise model, whatever it is. Then the last one is the idea of mental models, which are the mental lattice of information, which Charlie Munger talked about. And these are sort of these mental models where we can understand core ideas from vast knowledge, and be able to use that as a sort of framing device to be able to look and understand something else. So at some level the best ideas come from understanding a core pattern or a mental model from one domain, and being able to take it into another domain and go, what happens if we map this sort of core pattern? So those three ideas of frameworks is what I do in the AI framework building course, to be able to teach people how to think and be able to solve in this sort of augmented world, and to be able to then map it to how do you use AI.

Ross: Fantastic! These are not the sort of things that most people are focusing on, but are extraordinarily valuable, we need to understand our cognitive structures in order to enhance cognition. And that when we do that, when we are, I think it was in terms of humans per se, I think in workflows. But part of that is you do need to have the cognitive structures of the human in place in order to be able to understand where the AI can best complement it.

Kes: One more thing to add, which is I mentioned AR to start, right. And the reason I’m excited about spatial computing is actually something Andy Clark has been talking about this idea of extended mind. So we have this really amazing capability where we can think outside our heads. So as soon as we started scratching in the dirt, cave paintings all the way to writing to murals, we can think outside our heads, we can extend what is a limited working memory size. But by putting things out in front of us, like I always say, mathematicians are the heaviest users of blackboards and whiteboards, because they work in this really abstract complex space of ideas where they have to keep so many things in mind. So we’ve been extending through technology, our working memory and extending that part of our conscious thinking, what I’ve been looking at is, how do we actually also extend our unconscious thinking? So bringing those two aspects? So all my work into understanding how the unconscious works, incubation works? How do you tap into, like, we process 50 bits of information, consciously, every second, five, zero and 11 million bits unconsciously? How do you make sense of that? 

So I’ve been developing a program, which actually allows you to take all of a million bits of information actually structuring in a sort of process, which involves incubation, rapid incubation, and sort of question storming, which I’ve been doing for years. But then taking that and being able to create sort of these interfaces in a spatial world spatial computing world, where we can actually take away user experience needs to go, Look, AR is not going to be about a bunch of floating rectangles, unfortunately, and much to the chagrin of every UX designer who seems to be doing that at the moment, that’s like the same thing. Skeuomorphism at scale, right, we are going to create very different interfaces to be able to access and think, extend our conscious brains and extend our unconscious brains to be able to think more in augmented space.

Ross: Love it. I can’t remember how long ago. It was nine years ago, I set up a company called Multi-dimension Corp, which actually got sold off at a very, very early stage before we sort of got very far, but it was basically exactly that concept. You know, we are thinking in multiple dimensions. And if we can interface with cognitive in multiple dimensions, that could be the way of the future. But it’s, actually something I literally in my first book in the year 2000. I wrote about essentially spatial interfaces to thinking and you will, just with so little, I continue to stagger me. So yes, as you suggest, with the vision, probe, whatever, people really started to get onto it, hopefully, it will start to get some good spatial, cognitive interfaces. So you have a sub stack, amongst other things. So where can people find that and about your work?

Kes: Yeah, so I got a sub-sack called ‘Centaurian, which I released an article every week on Mondays, so forth, one dropped today. We also do a Friday, LinkedIn Live, which is a tectonic, where we actually unpack what’s happening in AI. Well, my weekly Centaurian article, and basically just,  , share some of our thinking, like, I’m trying to think in public as I get out there, as most of my sort of innovations and things I’ve developed over the three decades, have normally been inside organizations, and they get formed before they ever get out there. I’m trying to do it all in public. So we can,  , I can interact with people like yourself who’ve been thinking very similar things and because that’s the fastest way we’re gonna get these ideas out there. So, I’m trying to do all my thinking publicly at the moment. Fantastic.

Ross: And thank you for that. I really believe this topic. This theme is just about the most important thing we can possibly work on. So thanks for all your work. Thanks for sharing everything you’re doing. It’s moving us forward. 

Kes: Yeah. You too, Ross. Love following your articles and your thinking and I think I came across your article on I think it was minotaurs and centaurs and, engage with it a little bit and realize that you were a kindred spirit, thinking through this and probably been doing this a lot longer than I have. Thank you. 

Ross: We’ll be talking more than that. 

Kes: That is awesome!