3min chapter

Clearer Thinking with Spencer Greenberg cover image

AI creativity and love (with Joel Lehman)

Clearer Thinking with Spencer Greenberg

CHAPTER

How to Make Algorithms Have Creativity

Einstein is an example of how it's just diversity and following your own guide to what's interesting, you may be able to solve a problem in a new way. So suppose we want to make algorithms have creativity. How does your view kind of influence how that should be done? Right. It goes back to some research that I did 10, 12 years ago during my PhD on an algorithm called Novelty Search. In deceptive problems, such a novelty search can sometimes solve a problem more quickly than optimizing for the objective.

00:00
Speaker 2
There's some kind of classic counter examples where it seems like someone made a great leap forward just by working on the problem by themselves, kind of outside of whatever an else was doing, maybe influenced by whatever else is doing, but seemingly really kind of large leap. And I don't know whether they're just really rare so you don't give them much value or whether you'd say they're not really good examples of how to make a leap forward. But a class example would be Einstein when in his 20s he kind of just invented relativity.
Speaker 1
Yeah, actually, yeah, so I actually don't disagree with that at all. And I think any particular outsider attempt to overturn something or to create a grand new ambition you might be really skeptical of. We all can't be Einstein. And yet in hindsight, because Einstein was coming from a little bit outside the system and had his own way of approaching the problem, he was able to solve it. It's probably a little problem of that and he's maybe a singular genius in some ways. But I do think it is an example of how it's just diversity and following your own guide to what's interesting, you may be able to solve a problem in a new way. For example, if a problem is been longstanding, then all the obvious approaches are unlikely to work and so you might need somebody with an outsider perspective in order to tackle it.
Speaker 2
So suppose we want to make algorithms have creativity. How does your view kind of influence how that should be done?
Speaker 1
Right. So a lot of this goes back to some research that I did 10, 12 years ago during my PhD on an algorithm called Novelty Search. And the idea is that typically machine learning algorithms, search algorithms are driven by a distance to a goal. Like you have some heuristic that's going to measure how close you are to achieving a certain goal. And that can work in certain circumstances. You can kind of engineer that heuristic well enough. But when you get to search problems that are deceptive and by deceptive, I mean, we're just following kind of an intuitive heuristic actually paints you into a dead end. Then this kind of objective driven search where you're trying to optimize the heuristic of progress can really fail. And instead you might want to do something kind of radical. So the idea in Novelty Search was instead of searching for something that's closer to the goal, you instead search for something that's just different than what the search process has encountered before. So it's searching directly for novelty. And it turns out that in deceptive problems, such a novelty search can sometimes solve a problem more quickly than optimizing for the objective. So you kind of have this then like result of sometimes searching for something can actually precluding you from finding it. And the claim is that a lot of the problems we care about actually are deceptive in this way.

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode