Challenges and Risks of Generating Synthetic Data for AI Training

The use of synthetic data for training large language models like GPT poses significant challenges and risks. While researchers suggest the need for more high-quality data for training models like GPT-5, the idea of generating synthetic data is criticized due to potential model collapse. Model collapse leads to a degenerative learning process where models start forgetting improbable events over time, impacting their ability to answer questions. The proposed solution of using another generative AI to monitor the quality of synthetic data is deemed impractical and risky, as it involves feeding unreliable data from one AI into another unreliable AI. This approach raises concerns about the reliability and integrity of data used for training AI models, especially as generative AI is already spamming the internet with content that lacks originality and authenticity.