Jason Warner's company faces the challenge of expanding the context window for available models. While there is enthusiasm for big context windows, the relationship to attention architectures poses a limitation. The challenge is figuring out how to efficiently get context into a model without forgetting. This is an important area of product and research work.
This week on No Priors, Sarah and Elad do another hangout to answer listener questions. Topics include debunking common misconceptions about AI and its implications on the world, the analogy to nuclear power and nuclear safety, the impact of larger context windows, developer productivity, incumbent announcements of AI products, and some requests for (fat) startups.
No Priors is now on YouTube! Subscribe to the channel on YouTube and like this episode.
Sign up for new podcasts every week. Email feedback to show@no-priors.com
Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @SchimpfBrian
Show Notes:
[00:21] - What Are People Getting Wrong About AI Right Now? / New Capabilities of NLP
[04:35] - Nuclear Power and Safety Concerns
[11:12] - Emerging AI Companies and Research
[15:54] - China's Hardware Sanctions and Funding Ramp
[20:34] - Innovation in Heterogeneous Compute Infrastructure
[28:08] - Enterprise Stack and Decision Making
[33:44] - Data's Impact on the World