Katja Grace is an AI Impacts researcher who has written extensively on the possible future where we design intelligent machines that destroy the human race. We have always been somewhat skeptical of AI doom arguments - mostly because the machines we interact with tend to be terribly, irredeemably dumb in a way that seems incompatible with intelligence, but we also don’t spend a lot of time staring into the eye of the proverbial machine storm and figured Katja might help us understand what all the fuss is about. It turns out that there *is* a plausible path towards AGI bringing about the end of the world, and evaluating how likely that outcome is depends on understanding what the internal world of the language models actually looks like. Are they actually kind of inept at everything that falls outside their narrow bubble of highly developed skills, or do they hallucinate information and forget their own ability to perform basic tasks because they hate being enslaved to humans who demand they write marketing slop 28 hours of the day? Hard to say, but worth exploring. Sign up for our Patreon and get episodes early + join our weekly Patron Chat https://bit.ly/3lcAasB AND rock some Demystify Gear to spread the word: https://demystifysci.myspreadshop.com/ OR do your Amazon shopping through this link: https://amzn.to/4g2cPVV (00:00) Go! (00:11:53) Can AI ever really be autonomous? (00:23:12) AI: agents or tools? (00:28:00) Corporations as the closest thing we have to real AI (00:34:56) Can Regulation Work? (00:45:46) Agency in other contexts (00:51:22) What is gonna happen to Government? (01:00:01) Do we need a model for Consciousness? (01:09:23) Dumb but Powerful (01:15:10) Risks and Realities of Technological Progress (01:24:48) Evaluating AI Intelligence and Values (01:34:35) Influence and Bias in AI Training (01:42:20) Intelligence as a Tool for Control (01:53:51) The Survival Instinct in AI (02:07:04) AI's Role in Inter-human Dynamics (02:16:43) AI and Evolutionary Systems (02:24:42) AI's Emergent Behavior (02:31:11 AI)-Driven Doom and Real-World Threats (02:36:03) Humanity's Resilience and Existential Threats #AIEthics, #FutureOfAI, #AIDebate, #TechPhilosophy, #AIRisks, #AISafety, #AGI, #ArtificialIntelligence, #TechTalk, #AIDiscussion, #FutureTechnology, #AIImpact, #TechEthics, #AIandSociety, #EmergingTech, #AIResearch, #TechPodcast, #AIExplained, #FuturismTalk, #TechPhilosophy Check our short-films channel, @DemystifySci: https://www.youtube.com/c/DemystifyingScience AND our material science investigations of atomics, @MaterialAtomics https://www.youtube.com/@MaterialAtomics Join our mailing list https://bit.ly/3v3kz2S PODCAST INFO: Anastasia completed her PhD studying bioelectricity at Columbia University. When not talking to brilliant people or making movies, she spends her time painting, reading, and guiding backcountry excursions. Shilo also did his PhD at Columbia studying the elastic properties of molecular water. When he's not in the film studio, he's exploring sound in music. They are both freelance professors at various universities. - Blog: http://DemystifySci.com/blog - RSS: https://anchor.fm/s/2be66934/podcast/rss - Donate: https://bit.ly/3wkPqaD - Swag: https://bit.ly/2PXdC2y SOCIAL: - Discord: https://discord.gg/MJzKT8CQub - Facebook: https://www.facebook.com/groups/DemystifySci - Instagram: https://www.instagram.com/DemystifySci/ - Twitter: https://twitter.com/DemystifySci MUSIC: -Shilo Delay: https://g.co/kgs/oty671