The Assumption of Unlimited Compute and Data in AI Model Training

The assumption in AI model training is based on the belief that there will be a continued increase in compute power and model improvement, but this might not hold true as most technologies follow an S-curve where improvements plateau over time. Additionally, the availability of data for training AI models is a practical concern as it's uncertain whether there will be enough data. Even if models are currently trained on most of the internet, the quality of the data is crucial, suggesting that training on high-quality rather than all available data could lead to better models.