How EleutherAI Trains and Releases LLMs: Interview with Stella Biderman
Gradient Dissent: Conversations on AI
00:00
The Importance of Scaling Language Models
We don't have any plans right now to train in publicly released like a 175 or 200 billion parameter language model. So I believe that title is currently owned by Google who has trained and not released a 540 billion Parameter Language Model. We're definitely going to keep training language models and we're going to continue to release many of the ones that we do train. But scaling for the sake of scaling, which is kind of something that we last on to as a meme early on and has become associated with our brand, even though it was primarily meant as a joke, is not really something that we're particularly invested in.
Transcript
Play full episode
