How EleutherAI Trains and Releases LLMs: Interview with Stella Biderman
Gradient Dissent: Conversations on AI
00:00
The Importance of Pre-Trained Language Models
There are two publicly available 175 billion parameter models, which is the size of the largest G2E3 model called bloom from the big science group and OPT from Facebook. And according to some benchmark tests, both of them substantially underperformed G2E 3. One of the really interesting things about recent advances in NLP is that the models perform pretty well even when you don't do that. For example, for very large models, if you just ask nicely and say like solve this problem and then it's oftentimes able to do that.
Transcript
Play full episode
