Challenges and Speculations in Scaling AI Models

Steve talks about AI in light of his recent travels to SF, Singapore, Manila, Berkeley, and Silicon Valley.

Chapters:

• (00:00) - Overview: SF, Singapore, Manila, Berkeley
• (01:48) - The AI Bubble in Silicon Valley
• (04:00) - Scaling Laws and AGI
• (23:36) - Global AI: Singapore, Philippines, real Enterprise applications
• (34:59) - AGI: Manhattan Project? Manifest and P(doomers), Situational Awareness
• (51:00) - China LLMs, Huawei vs Nvidia GPUs, US vs China AI race

Links:

• Scaling Laws for Neural Language Models: https://arxiv.org/pdf/2001.08361
• AI rollout in Philippines Call Centers: https://www.youtube.com/watch?v=Kps3M1wMHUM
• LLM rankings and Qwen2: https://huggingface.co/spaces/open-llm-leaderboard/blog
• The Economist on China LLMs: https://archive.ph/nW7ch
• Situational Awareness summary: https://x.com/hsu_steve/status/1803414701159714825

Music used with permission from Blade Runner Blues Livestream improvisation by State Azure.

--

Steve Hsu is Professor of Theoretical Physics and of Computational Mathematics, Science, and Engineering at Michigan State University. Previously, he was Senior Vice President for Research and Innovation at MSU and Director of the Institute of Theoretical Science at the University of Oregon. Hsu is a startup founder (SuperFocus.ai, SafeWeb, Genomic Prediction, Othram) and advisor to venture capital and other investment firms. He was educated at Caltech and Berkeley, was a Harvard Junior Fellow, and has held faculty positions at Yale, the University of Oregon, and MSU. Please send any questions or suggestions to manifold1podcast@gmail.com or Steve on Twitter @hsu_steve.