
On the Opportunities and Risks of Foundation Models
AI Safety Fundamentals: Alignment
The Evolution of Self-Supervised Learning
The acceptance that a single model could be useful for such a wide range of tasks marks the beginning of the era of foundation models. Foundation models have led to an unprecedented level of homogenization. Almost all state-of-the-art NLP models are now adapted from one of a few foundation models, such as BERT, REBERTA, BART, T5, etc. While this homogenization process produces extremely high leverage, any improvements in the foundation models can lead to immediate benefits across all of NLP but it is also a liability. All AI systems might inherit the same problematic biases of some foundation models. See section 5.1, fairness, and section 5.6
00:00
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.