The Future of AI Training

2min Snip

00:00

Play full episode

Summary

Transcript

Episode notes

We won't just be able to run more inferences, but we'll be able to train larger models. If you look at GPT-3, which is already a large model at 170 billion parameters, there's a question of how big models will get in the future and what data will be needed to train those models. There's going to need to be a lot of language data generated to feed models once we get training capacity this large.

Welcome to the Big Ideas Monday Miniseries, brought to you by the For Your Innovation Podcast.

Big Ideas is meant to enlighten investors on the long-term impact of innovation. This annual research report seeks to highlight the technological breakthroughs evolving today and creating the potential for super-exponential growth tomorrow.

We believe that innovation is taking off now, corroborating our original research and boosting our confidence that ARK’s strategies are on the right side of change.

To learn more, view Big Ideas 2023.

This week’s Big Ideas Monday Mini Episode focuses on Artificial Intelligence.

Generative AI made waves this year, from DALL-E-2 to ChatGPT. These tools are improving the productivity of knowledge workers—~2x in the case of AI coding assistants.

AI training cost declines continued at an annual rate of 70%, the cost to train a large language model to GPT-3 level performance collapsing from $4.6 million in 2020 to $450,000 in 2022. We expect cost declines to continue at a 70% rate through 2030.

AI should increase the productivity of knowledge workers more than 4-fold by 2030. At 100% adoption, AI could increase global labor productivity ~$200 trillion, dwarfing the ~$32 trillion in total knowledge worker salaries.

Watch the video version here.

Sources: ARK Investment Management LLC, 2023. Forecasts are inherently limited and cannot be relied upon. For informational purposes only and should not be considered investment advice or a recommendation to buy, sell, or hold any particular security. Past performance is not indicative of future results.