The question is say that you had a billion token context windows that big enough, like where does it asymptote in terms of value? You can think of it in a few different ways. Like how many tokens do you need to represent all the data that's relevant to a person relative to a specific query that you're worried about relative to PII or other sensitive information? And so I agree like the windows will get bigger and bigger and bigger,. But eventually some of these things asymptotes, and they take years or decades to asymptotal.
This week on No Priors, Sarah and Elad do another hangout to answer listener questions. Topics include debunking common misconceptions about AI and its implications on the world, the analogy to nuclear power and nuclear safety, the impact of larger context windows, developer productivity, incumbent announcements of AI products, and some requests for (fat) startups.
No Priors is now on YouTube! Subscribe to the channel on YouTube and like this episode.
Sign up for new podcasts every week. Email feedback to show@no-priors.com
Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @SchimpfBrian
Show Notes:
[00:21] - What Are People Getting Wrong About AI Right Now? / New Capabilities of NLP
[04:35] - Nuclear Power and Safety Concerns
[11:12] - Emerging AI Companies and Research
[15:54] - China's Hardware Sanctions and Funding Ramp
[20:34] - Innovation in Heterogeneous Compute Infrastructure
[28:08] - Enterprise Stack and Decision Making
[33:44] - Data's Impact on the World