Navigating AI Benchmarks and Risks

On this episode, Sarah Hastings-Woodhouse joins me to discuss what benchmarks actually measure, AI’s development trajectory in comparison to other technologies, tasks that AI systems can and cannot handle, capability profiles of present and future AIs, the notion of alignment by default, and the leading AI companies’ vague AGI plans. We also discuss the human psychology of AI, including the feelings of living in the "fast world" versus the "slow world", and navigating long-term projects given short timelines.  

Timestamps:  

00:00:00 Preview and intro

00:00:46 What do benchmarks measure?  

00:08:08 Will AI develop like other tech?  

00:14:13 Which tasks can AIs do? 

00:23:00 Capability profiles of AIs  

00:34:04 Timelines and social effects 

00:42:01 Alignment by default?  

00:50:36 Can vague AGI plans be useful? 

00:54:36 The fast world and the slow world 

01:08:02 Long-term projects and short timelines