AI Safety Fundamentals: Alignment cover image

On the Opportunities and Risks of Foundation Models

AI Safety Fundamentals: Alignment

CHAPTER

The Evolution of Self-Supervised Learning

The acceptance that a single model could be useful for such a wide range of tasks marks the beginning of the era of foundation models. Foundation models have led to an unprecedented level of homogenization. Almost all state-of-the-art NLP models are now adapted from one of a few foundation models, such as BERT, REBERTA, BART, T5, etc. While this homogenization process produces extremely high leverage, any improvements in the foundation models can lead to immediate benefits across all of NLP but it is also a liability. All AI systems might inherit the same problematic biases of some foundation models. See section 5.1, fairness, and section 5.6

00:00
Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner