The Challenge of Language Model Training and the Need for Compute

This chapter explores the challenge of making language model training scientific and the importance of having a lot of compute to achieve this goal. It discusses the transition of AI2 from academia to industry and the need for minimum viable resources and government trust as a nonprofit.