Deep learning will continue to make progress in the next 5 to 10 years, with improvements in many layers of the stack and specialized training for specific domains.
OpenAI acknowledges the desire for access to their technologies and takes an intermediate approach between open and closed sourcing, encouraging collaborative research.
Deep dives
Iliya Sootzgever: Co-founder and Chief Scientist of Open AI
Iliya Sootzgever is the co-founder and chief scientist of Open AI, a company that aims to build artificial general intelligence. He is known for his work on the large language model GPT-3 and its public-facing product, Chat GPT. Sootzgever's background includes a Bachelor's degree in mathematics from the University of Toronto and a Master's and PhD in computer science. He has made significant contributions to the field of deep learning, including being the main impetus behind AlexNet, a convolutional neural network that set off the deep learning revolution. Sootzgever's work on Chat GPT has had a profound impact and its release in November 2022 generated excitement as well as societal concerns. He believes that the human brain serves as the best example of intelligence and has worked on developing large language models to mimic human understanding and prediction. Sootzgever acknowledges that current AI systems are still far from matching human capabilities, but the potential for AI to surpass human learning in the future is not inconceivable. He views consciousness as a matter of degree rather than a binary distinction and suggests that an experiment could be conducted to determine if an AI system can articulate awareness without being explicitly trained on consciousness. Open AI's mission is to ensure that artificial general intelligence benefits all of humanity, and they are open to exploring sensible government regulations to ensure responsible development and deployment of AI technologies. Sootzgever acknowledges the importance of technical progress in deep learning and expects the field to continue evolving. While there may be limitations to scaling large language models, he believes that progress will come from various dimensions of deep learning and that specialized training for specific domains will be beneficial once a robust understanding of tasks is achieved.
The Evolution of Deep Learning
Iliya Sootzgever expects deep learning to continue making progress in the next five to ten years. While scaling large language models may become more challenging, there will be improvements in many layers of the deep learning stack. He foresees discoveries of new properties of deep learning and their utilization in driving progress. Specialist training for specific domains is already happening to some extent and can yield significant performance improvements. However, Sootzgever emphasizes that general training is necessary to reach a point where neural networks can understand and engage in more specialized tasks effectively. He believes that a combination of factors, including improved base models and specialized training, will contribute to advancements in deep learning.
OpenAI's Approach to Openness and Collaboration
OpenAI acknowledges the desire of the research community for access to their technologies. While the company cannot provide unfettered access without safeguards, they offer academic access programs and provide various forms of access to their models. OpenAI believes in finding an intermediate approach between open and closed sourcing of AI models. They recognize the benefits of collaborative research and encourage contributions from the research community. OpenAI is committed to its mission to ensure that artificial general intelligence benefits all of humanity. The company's corporate structure is unique, with capped profit status and a fiduciary duty to the OpenAI mission. Their relationship with Microsoft demonstrates an alignment towards the responsible and beneficial development of AI.
The Role of AI in Teams and Society
Integration of AI, such as Chat GPT, into human teams increases productivity and keeps individuals better informed. While there may be concerns about depersonalization of conversations, Iliya Sootzgever believes that AI integration does not have a dramatic impact on team dynamics at the moment. He envisions a future where AI's impact on team dynamics and societal interactions may evolve, but he does not expect depersonalization to be a significant factor. Sootzgever emphasizes the importance of responsible AI development and the need for sensible government regulations to guide AI applications.
Ilya Sutskever is the co-founder and chief scientist of OpenAI, which aims to build artificial general intelligence that benefits all of humanity. He leads research at OpenAI and is one of the architects behind the GPT models. In this conversation with Stanford adjunct lecturer Ravi Belani, Sutskever explains his approach to making complex decisions at OpenAI and for AI companies in general, and makes predictions about the future of deep learning.
—-----------------------------------
Stanford eCorner content is produced by the Stanford Technology Ventures Program. At STVP, we empower aspiring entrepreneurs to become global citizens who create and scale responsible innovations.
Support our mission of providing students and educators around the world with free access to Stanford University’s network of entrepreneurial thought leaders: https://ecorner.stanford.edu/give.
Get the Snipd podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share & Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode