Is There a Parallel to the Bitter Lesson in AI?

The last couple of years have been an AI model arms race.

The assumption is that the larger the model the better it will perform. But that may not always be the case.

FAR AI Research Scientist Ian McKenzie is a key member of the team organizing the Inverse Scaling Challenge, an initiative set up to investigate scaling laws. We discuss:

• Large Language Models and how they are trained
• The scaling laws and how they are being revised as research and development progresses
• The Inverse Scaling Challenge and its findings

Article published on Orchestrate all the Things.