The Implications of Scalable Language Models

The models are getting like the foundation models, like the language models. They're just going to get more and more expensive to build because it's massive amounts of compute infrastructure required. And there doesn't appear to be a point of diminishing marginal return that we're approaching on scale. So you make them bigger, and they become more powerful at the task to which they've already been put in their smaller incarnations.

Transcript

Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.