3min chapter

No Priors: Artificial Intelligence | Technology | Startups cover image

How can we make sure that everyone has access to AI? Can small models outperform large models? With Stability AI’s Emad Mostaque

No Priors: Artificial Intelligence | Technology | Startups

CHAPTER

The Impact of OpenAI on the AI Field

The AI field in recent years has been increasingly driven by labs and private companies. One of the most obvious paths to performance progress is to just make models bigger, right? And data parameters, GPUs, which is very expensive. So I started supporting Eluthre AI as part of the second wave,. You know, Stellar and Connor and others kind of led it on the language model side. But really, my main interests was the image model side.

00:00
Speaker 3
It's an amazing journey, and congratulations on all the impact you've already had. As you say, or as you imply, the AI field in recent years has been increasingly driven by labs and private companies. One of the most obvious paths to performance progress is to just make models bigger, right? And data parameters, GPUs, which is very expensive. Just to set the stage a little bit, there's been some efforts over the previous years to be more community-driven and open and build alternatives like Luther. How did you start engaging in that, and how did stability change the game here?
Speaker 1
Yeah, so when I was doing the COVID work, we tried to get access to various models. In some cases, the companies blew up, other cases weren't given access, despite it being a high profile project. So I started supporting Eluthre AI as part of the second wave, you know, Stellar and Connor and others kind of led it on the language model side. But really, one of my main interests was the image model side. I have aphantasia, so I can't visualize anything in my brain, which is more common than people would think. In fact, a lot of the developers in this space have that. Like we've got nothing in our
Speaker 3
brain. You just see words? What's in there? Just feelings.
Speaker 1
Okay. So, like, again, I thought it was a metaphor. Imagine yourself on a beach. I was like, okay, I feel a beach. No, apparently you guys have pictures in your heads. It must be like just disconcerting. But then with the arrival of clip released by OpenAI a couple of years ago, you could suddenly take generative models and guide them to text prompts. So it was VQGAN, which is kind of the slightly mushy, more abstract version first. But I built a model for my daughter while I was recovering, ironically, from COVID. And then she took the output and sold it as an NFT for $3,500 and donated to India COVID-19. And I was like, wow, that's crazy. So I started supporting the whole space. I had Luther and beyond giving jobs to the developers, compute for the model creators, funding the various notebooks from disco diffusion to these other things, giving grants to people like their journey that were kind of kicking this off. Just personally. Just personally. They were doing all the hard work. And I was like, can I capitalize this? Because it's good for society. Then about 15 months ago, I was like, well, these communities are growing. It'd be great if we could create this as a common good. And originally, I thought, you got communities. You got to make them kind of coordinated. Could a DAO work or a DAO of DAOs. And that's how stability started. After about a week, I realized that was not going to work. And it was incredibly difficult. So then I figured out commercial open source software could be the way to create a Lyme technology, not just an image, but beyond, that would potentially change the game by making this stuff accessible. But as you said, one of the key things, this is in the state of the AI report. This is in AI index as well, is that most research has been subject to scaling laws and other things. Transformers seem to work for everything. And so it was moving more and more towards private companies.

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode