Divergent Thinking, College Towns & AI Poison with Brian Ardinger and Robyn Bolton

On this week's episode of Inside Outside Innovation, Robyn and Brian sit down to talk about divergent thinking, peak college towns, and how as little as 250 documents can poison your AI. Let's get started.

Inside Outside Innovation is the podcast to help innovation leaders navigate what's next. Each week, we'll give you a front row seat into what it takes to grow and thrive in a world of hyper-uncertainty and accelerating change. Join me, Brian Ardinger, and Miles Zero's Robyn Bolton as we discuss the latest tools, tactics, and trends for creating innovations with impact. Let's get started.

[00:00:40] Brian Ardinger: Welcome to another episode of Inside Outside Innovation. I'm your host, Brian Ardinger, and I have my co-host, Robyn Bolton. Hello, Robyn.

[00:00:47] Robyn Bolton: Hello, Brian. How are you today?

[00:00:49] Brian Ardinger: I am doing well. Episode 338. We're excited to talk about innovation. I'm so glad you're on the journey with me here. 

[00:00:56] Robyn Bolton: Wow. 338. There should be a cake or streamers or something. 

[00:01:01] Brian Ardinger: Well, as usual, we've got a number of different articles and things that have caught our attention over the last couple days. So, I figure we'd dive in. And the first article I wanted to talk about today was for my friend Audrey Crane. She works at Design Map, and she posted on the Design Map blog an article on Divergent thinking.

The article, I'll give it a little preface. So, it was looking at how you generate better ideas. Looked at a lot of things. First of all, looked at the research and the fact that there's a famous study by George Land and Beth Jarman looking at how five-year-olds were asked to come up with as many uses as possible for a paperclip, and nearly all of them, 98% could generate 200 or more ideas.

They continued to do this study and looked into adulthood, and by the time the participants became adults, only 2%. Of adults could actually do the same thing and generate 200 ideas about a paperclip. It's pretty crazy, and it points to the fact that, while it's a super important skill to have, to be able to generate new ideas and think about different things. We are losing that ability as we get older. 

[00:02:08] Robyn Bolton: It was surprising and sad, but reminded me of two things. So, one is a book that came out a long time over a decade ago now, the Innovator's DNA. Where the authors looked at, you know, across hundreds, thousands of successful, both entrepreneurs and intrapreneurs, corporate innovators.

And they found that the one thing that they had in common, or most had in common, was associative thinking, which. Is being able to put two very different ideas together to make something. And so divergent thinking actually reminded me a lot of the associative thinking. Because it just, your brain works in different ways.

And the second thing is, you know, I teach at Massachusetts College of Art and Design, and in talking to my actual art colleagues there, one of the things that they say that's very consistent with what we say in innovation, is that your first idea, your first work, your first version is always the worst.

They have different language around the idea of constraints and don't accept the first version, but that is a very common discipline within the art and design world. That it's just your first idea is going to be terrible, so you've got to push past it. And that was another aspect of this article that I found really interesting.

[00:03:29] Brian Ardinger: It fits into startups as well. I was mentoring the new batch of NMotion companies are going to be announced here soon that are going through it. And did a little round table and met each of the teams and talked about what they're building and that. And the piece of advice I left them with is, you are here on day one with your idea. Be open to other ideas that happen when you start talking to customers and trying to understand if, if you really are onto something.

Because a lot of times you get into an accelerator or you get some early traction and you think, okay, I've got all figured out. I got the solution. I'm just going to barge ahead. But you have to be open to that divergent thinking and different ways of doing things just so you don't necessarily leave opportunity on the table. 

[00:04:08] Robyn Bolton: It's the old adage, right? The fall in love with the problem, not the solution. And it's so easy to fall in love with the solution, but the divergent thinking article was a great reminder. Data-based reminder of why we've got to nurture the skill to go beyond that. 

[00:04:26] Brian Ardinger: Yeah, and I love the article too, because Audrey goes into a number of different techniques, brainstorming techniques, and that. Mm-hmm. So, I encourage people to check that out, to learn some tactical things about how you go about building up that divergent thinking muscle.

The second article, this one made me a little nervous. It's from the Neuron Daily, which is a fantastic newsletter, and this one came out a couple weeks ago, but the title was Poisoning AI Models just got scarier. 250 documents is all it takes.

And it goes on to look at some research that came out of Anthropic that looked at the number of documents that you could ingest into the LLM that would make it start spewing out nonsense and making it bad. The original thinking was, well, the more larger models that you have, you'd have to have more malicious documents put into it to make it go off the rails. What they found out was that's actually not the case.

It takes a very small amount of wrong data to be put into it to actually make it start going the wrong direction. Actually, I think in the article talks about a 13 billion parameter model trained on 260 billion tokens, got backdoored with the same 250 documents as the 600 million model trained on just 12 billion tokens.

And previous research assumed that attackers needed to control 0.1 of the training data. And so for larger models you need millions more documents. What actually happened is the math was 250 poison documents was, is only 0.00016 of the training data was enough to actually poison the model itself. Crazy stuff to think about it.

When we think about. All the things that'll be putting into the web and everything else. It's a wonder why we're not seeing more and more hallucinations. I guess 

[00:06:07] Robyn Bolton: it was terrifying, and I'll be honest, the numbers went so over my head, but it had an analogy in there. It's like putting a teaspoon of a toxin into an Olympic sized pool. Yeah. And you'd kill all the swimmers. And I was like, uh oh. That is. So much more awful than I thought when you were just saying 250 documents.

And you know, I think it just goes to pis. There's incredible promise with Gen AI and there's risks that we don't even know about. It's one of those things of use with caution.

[00:06:39] Brian Ardinger: It'll be interesting to see what other research comes out to either debunk this or to fill up the back holes of this. But it's just something to keep in mind and follow the research and see what's actually going on out there. Because we are in brand new territory and the exploration mode, we need to be wary of the pitfalls that are ahead as well.

[00:06:57] Robyn Bolton: