The Evolution of AI Through Scale

The progression of artificial intelligence over the last five years has largely been driven by scale, involving both the amount of data used for training and the computational resources required. Although the underlying transformer architecture remains consistent, the size and complexity of the models have been growing exponentially. In 2019, training models like GPT-2 cost around $5,000, while today’s frontier models can require hundreds of millions of dollars to train. This raises questions about the sustainability of such scaling practices and whether AI development might soon plateau as it becomes increasingly expensive.