Is scaling all you need for AI Large Language Models? Scaling laws and the Inverse Scaling Challenge. Featuring Ian McKenzie, FAR AI Research Scientist
Orchestrate all the Things
00:00
How Did the Competition Work?
The way the contest worked was basically we kind of set up some guidelines about what would make a good submission, how many examples to use. We provided like three different metrics to score datasets against and then we produced these Google Colabs, which are interactive coding tools to run your dataset against these large language models. Some of them are available on hugging face, like the OPT models and the kind of GPT-2 models and other in those families.
Transcript
Play full episode
