Looking Back at AI in 2021 with Jeremie from Towards Data Science

Jan 6, 2022

Jeremie Harris, host of the Towards Data Science podcast, dives into the pivotal AI trends of 2021. He highlights the shift towards efficiency in foundational models, exemplified by DeepMind's Gopher outpacing GPT-3 in performance while requiring fewer resources. The conversation also touches on the importance of procedural environment generation in reinforcement learning, showcasing how it can lead to more adaptable AI agents. Jeremie reflects on the implications for ethics in AI development and the exciting future of open-ended learning.

Ask episode

AI Snips

Chapters

Transcript

Episode notes

INSIGHT

Rise of Multimodal Models

Multimodal models are on the rise, integrating different data modes like text and images.
This might suggest a more unified learning approach, similar to the human brain.

INSIGHT

Transformer Dominance

Transformers are becoming the dominant architecture for various AI tasks, replacing RNNs, LSTMs, and CNNs.
This consolidation could lead to both increased efficiency and potential limitations.

INSIGHT

Potential AI Winter?

The convergence on transformer architecture may lead to diminishing returns if scaling doesn't meet expectations.
This could potentially signify the start of an AI winter, a period of reduced progress and funding.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

For our first episode in 2022, we are joined with our friends from the Towards Data Science podcast to discuss our thoughts about the AI-related trends and events that happened in 2021.

Some things we discuss are:

Foundation models continue to grow, but one interesting trend is the focus on efficiency along with (instead of?) scale. For example, while DeepMind’s Gopher model has fewer than twice the parameters of GPT-3, it’s reportedly 25 times more efficient, meaning that much more value is being squeezed out of the same training data and compute. AI21Labs’ Jurrassic models are also equal to GPT-3 on a parameter count basis, but reflect a focus on architecture optimization over raw scaling that we expect to persist into 2022. (That’s not to say significant scaling won’t happen, or that it hasn’t happened already; Microsoft Turing-NLG, released a few months ago, is over half a trillion parameters in size. But it’s safe to say that scaling won’t be done without simultaneous efficiency optimizations that were less of a focus in late-2020.)
Procedural environment generation has been a big theme in reinforcement learning. In Open-Ended Learning Leads to Generally Capable Agents, the team at DeepMind showed how training RL agents on a wide range of environments can lead to emergent behaviour associated with generalization, like trial and error and cooperation with friendly agents.
Open-ended learning (OEL) seems like an interesting wildcard, which some researchers think might be an important ingredient in the final AGI recipe. We spoke with OpenAI’s head of open-ended learning, Ken Stanley, about what role OEL might play in the future of AI on this episode of the TDS podcast.
A NeurIPS spotlight paper titled Optimal Policies Tend to Seek Power, and subsequent work by the same author, are showing that we should expect highly capable AI systems to engage in dangerous behaviour that’s misaligned with human values, by default. Specifically, highly competent agents will tend to search for states that are powerful, in the sense that they offer many downstream options. This finding makes a compelling case that AI alignment ought to be prioritized, particularly given the rate of progress we’re seeing in AI capabilities more broadly. If it really is the case that capable AI systems will be dangerous by default, active effort must be invested in safety research.

Outline:

0:00 Intro
2:15 Rise of multi-modal models
7:40 Growth of hardware and compute
13:20 Reinforcement learning
20:45 Open-ended learning
26:15 Power seeking paper
32:30 Safety and assumptions
35:20 Intrinsic vs. extrinsic motivation
42:00 Mapping natural language
46:20 Timnit Gebru’s research institute
49:20 Wrap-up