Guest Percy Liang is an authority on AI who says that we are undergoing a paradigm shift in AI powered by foundation models, which are general-purpose models trained at immense scale, such as ChatGPT. In this episode of Stanford Engineering’s The Future of Everything podcast, Liang tells host Russ Altman about how foundation models are built, how to evaluate them, and the growing concerns with lack of openness and transparency.
Connect With Us:
Chapters:
(00:00:00) Introduction
Host Russ Altman introduces Percy Liang, who runs the Stanford Center on Foundation Models
(00:02:26) Defining Foundation Models
Percy Liang explains the concept of foundation models and the paradigm shift they represent.
(00:04:22) How are Foundation Models Built & Trained?
Explanation of the training data sources and the scale of training data: training on trillions of words. Details on the network architecture, parameters, and the objective function.
(00:10:36) Context Length & Predictive Capabilities
Discussion on context length and its role in predictions. Examples illustrating the influence of context length on predictive accuracy.
(00:12:28) Understanding Hallucination
Percy Liang explains how foundation models “hallucinate”, and the need for both truth and creative tasks which requires “lying”.
(00:15:19) Alignment and Reinforcement in Training
The role of alignment and reinforcement learning from human feedback in controlling model outputs.
(00:18:14) Evaluating Foundation Models
The shift from task-specific evaluations to comprehensive model evaluations, Introduction of HELM & the challenges in evaluation these models.
(00:25:09) Foundation Models Transparency Index
Percy Liang details the Foundation Models Transparency Index, the initial results and reactions by the companies evaluated by it.
(00:29:42) Open vs. Closed AI Models: Benefits & Risks
The spectrum between open and closed AI models , benefits and security impacts
Connect With Us:
Episode Transcripts >>> The Future of Everything Website
Connect with Russ >>> Threads / Bluesky / Mastodon
Connect with School of Engineering >>>Twitter/X / Instagram / LinkedIn / Facebook
The Future of Everything