Irina Rish a professor at the Université de Montréal, a core member of Mila (Quebec AI Institute), and the organizer of the neural scaling laws workshop towards maximally beneficial AGI. In this episode we discuss Irina's definition of Artificial General Intelligence, her takes on AI Alignment, AI Progress, current research in scaling laws, the neural scaling laws workshop she has been organizing, phase transitions, continual learning, existential risk from AI and what is currently happening in AI Alignment at Mila. Transcript: theinsideview.ai/irina Youtube: https://youtu.be/ZwvJn4x714s OUTLINE (00:00) Highlights (00:30) Introduction (01:03) Defining AGI (03:55) AGI means augmented human intelligence (06:20) Solving alignment via AI parenting (09:03) From the early days of deep learning to general agents (13:27) How Irina updated from Gato (17:36) Building truly general AI within Irina's lifetime (19:38) The least impressive thing that won't happen in five years (22:36) Scaling beyond power laws (28:45) The neural scaling laws workshop (35:07) Why Irina does not want to slow down AI progress (53:52) Phase transitions and grokking (01:02:26) Does scale solve continual learning? (01:11:10) Irina's probability of existential risk from AGI (01:14:53) Alignment work at Mila (01:20:08) Where will Mila get its compute from? (01:27:04) With Great Compute Comes Great Responsibility (01:28:51) The Neural Scaling Laws Workshop At NeurIPS

Shahar is a senior researcher at the Center for the Study of Existential Risk in Cambridge. In his past life, he was a Google Engineer, though right now he spends most of your time thinking about how to prevent the risks that occur if companies like Google end up deploying powerful AI systems, by organizing AI Governance role-playing workshops. In this episode, we talk about a broad variety of topics, including how we could apply the lessons from running AI Governance workshops to governing transformative AI, AI Strategy, AI Governance, Trustworthy AI Development and end up answering some Twitter questions. Youtube: https://youtu.be/3T7Gpwhtc6Q Transcript: https://theinsideview.ai/shahar Host: https://twitter.com/MichaelTrazzi Shahar: https://www.shaharavin.com Outline (00:00) Highlights (01:20) Intelligence Rising (06:07) Measuring Transformative AI By The Scale Of Its Impact (08:09) Comprehensive AI Services (11:38) Automating CEOs Through AI Services (14:21) Towards A "Tech Company Singularity" (15:58) Predicting AI Is Like Predicting The Industrial Revolution (19:57) 50% Chance Of Human-brain Performance By 2038 (22:25) AI Alignment Is About Steering Powerful Systems Towards Valuable Worlds (23:51) You Should Still Worry About Less Agential Systems (28:07) AI Strategy Needs To Be Tested In The Real World To Not Become Theoretical Physics (31:37) Playing War Games For Real-time Partial-information Advesarial Thinking (34:50) Towards World Leaders Playing The Game Because It’s Useful (39:31) Open Game, Cybersecurity, Government Spending, Hard And Soft Power (45:21) How Cybersecurity, Hard-power Or Soft-power Could Lead To A Strategic Advantage (48:58) Cybersecurity In A World Of Advanced AI Systems (52:50) Allocating AI Talent For Positive R&D ROI (57:25) Players Learn To Cooperate And Defect (01:00:10) Can You Actually Tax Tech Companies? (01:02:10) The Emergence Of Bilateral Agreements And Technology Bans (01:03:22) AI Labs Might Not Be Showing All Of Their Cards (01:06:34) Why Publish AI Research (01:09:21) Should You Expect Actors To Build Safety Features Before Crunch Time (01:12:39) Why Tech Companies And Governments Will Be The Decisive Players (01:14:29) Regulations Need To Happen Before The Explosion, Not After (01:16:55) Early Regulation Could Become Locked In (01:20:00) What Incentives Do Companies Have To Regulate? (01:23:06) Why Shahar Is Terrified Of AI DAOs (01:27:33) Concrete Mechanisms To Tell Apart Who We Should Trust With Building Advanced AI Systems (01:31:19) Increasing Privacy To Build Trust (01:33:37) Sensibilizing To Privacy Through Federated Learning (01:35:23) How To Motivate AI Regulations (01:37:44) How Governments Could Start Caring About AI risk (01:39:12) Attempts To Regulate Autonomous Weapons Have Not Resulted In A Ban (01:40:58) We Should Start By Convincing The Department Of Defense (01:42:08) Medical Device Regulations Might Be A Good Model Audits (01:46:09) Alignment Red Tape And Misalignment Fines (01:46:53) Red Teaming AI systems (01:49:12) Red Teaming May Not Extend To Advanced AI Systems (01:51:26) What Climate change Teaches Us About AI Strategy (01:55:16) Can We Actually Regulate Compute (01:57:01) How Feasible Are Shutdown Swi

Katja runs AI Impacts, a research project trying to incrementally answer decision-relevant questions about the future of AI. She is well known for a survey published in 2017 called, When Will AI Exceed Human Performance? Evidence From AI Experts and recently published a new survey of AI Experts: What do ML researchers think about AI in 2022. We start this episode by discussing what Katja is currently thinking about, namely an answer to Scott Alexander on why slowing down AI Progress is an underexplored path to impact.  Youtube: https://youtu.be/rSw3UVDZge0 Audio & Transcript: https://theinsideview.ai/katja Host: https://twitter.com/MichaelTrazzi Katja: https://twitter.com/katjagrace OUTLINE (00:00) Highlights (00:58) Intro (01:33) Why Advocating For Slowing Down AI Might Be Net Bad (04:35) Why Slowing Down AI Is Taboo (10:14) Why Katja Is Not Currently Giving A Talk To The UN (12:40) To Avoid An Arms Race, Do Not Accelerate Capabilities (16:27) How To Cooperate And Implement Safety Measures (21:26) Would AI Researchers Actually Accept Slowing Down AI? (29:08) Common Arguments Against Slowing Down And Their Counterarguments (36:26) To Go To The Stars, Build AGI Or Upload Your Mind (39:46) Why Katja Thinks There Is A 7% Chance Of AI Destroys The World (46:39) Why We Might End Up Building Agents (51:02) AI Impacts Answer Empirical Questions To Help Solve Important Ones (56:32) The 2022 Expert Survey on AI Progress (58:56) High Level Machine Intelligence (1:04:02) Running A Survey That Actually Collects Data (1:08:38) How AI Timelines Have Become Shorter Since 2016 (1:14:35) Are AI Researchers Still Too Optimistic? (1:18:20) AI Experts Seem To Believe In Slower Takeoffs (1:25:11) Automation and the Unequal Distributions of Cognitive power (1:34:59) The Least Impressive Thing that Cannot Be Done in 2 years (1:38:17) Final thoughts

Markus Anderljung is the Head of AI Policy at the Centre for Governance of AI  in Oxford and was previously seconded to the UK government office as a senior policy specialist. In this episode we discuss Jack Clark's AI Policy takes, answer questions about AI Policy from Twitter and explore what is happening in the AI Governance landscape more broadly. Youtube: https://youtu.be/DD303irN3ps Transcript: https://theinsideview.ai/markus Host: https://twitter.com/MichaelTrazzi Markus: https://twitter.com/manderljung OUTLINE (00:00) Highlights & Intro (00:57) Jack Clark’s AI Policy Takes: Agree or Disagree (06:57) AI Governance Takes: Answering Twitter Questions (32:07) What The Centre For the Governance Of AI Is Doing (57:38) The AI Governance Landscape (01:15:07) How The EU Is Regulating AI (01:29:28) Towards An Incentive Structure For Aligned AI

Alex Lawsen is an advisor at 80,000 hours, released an Introduction to Forecasting Youtube Series and has recently been thinking about forecasting AI progress, why you cannot just "update all the way bro" (discussed in my latest episode with Connor Leahy) and how to develop inside views about AI Alignment in general. Youtube: https://youtu.be/vLkasevJP5c Transcript: https://theinsideview.ai/alex  Host: https://twitter.com/MichaelTrazzi Alex: https://twitter.com/lxrjl OUTLINE (00:00) Intro (00:31) How Alex Ended Up Making Forecasting Videos (02:43) Why You Should Try Calibration Training (07:25) How Alex Upskilled In Forecasting (12:25) Why A Spider Monkey Profile Picture (13:53) Why You Cannot Just "Update All The Way Bro" (18:50) Why The Metaculus AGI Forecasts Dropped Twice (24:37) How Alex’s AI Timelines Differ From Metaculus (27:11) Maximizing Your Own Impact Using Forecasting (33:52) What Makes A Good Forecasting Question (41:59) What Motivated Alex To Develop Inside Views About AI (43:26) Trying To Pass AI Alignment Ideological Turing Tests (54:52) Why Economic Growth Curve Fitting Is Not Sufficient To Forecast AGI (01:04:10) Additional Resources

Robert Long is a research fellow at the Future of Humanity Institute. His work is at the intersection of the philosophy of AI Safety and consciousness of AI. We talk about the recent LaMDA controversy, Ilya Sutskever's slightly conscious tweet, the metaphysics and philosophy of consciousness, artificial sentience, and how a future filled with digital minds could get really weird.  Youtube: https://youtu.be/K34AwhoQhb8 Transcript: https://theinsideview.ai/roblong Host: https://twitter.com/MichaelTrazzi Robert: https://twitter.com/rgblong Robert's blog: https://experiencemachines.substack.com OUTLINE (00:00:00) Intro (00:01:11) The LaMDA Controversy (00:07:06) Defining AGI And Consciousness (00:10:30) The Slightly Conscious Tweet (00:13:16) Could Large Language Models Become Conscious? (00:18:03) Blake Lemoine Does Not Negotiate With Terrorists (00:25:58) Could We Actually Test Artificial Consciousness? (00:29:33) From Metaphysics To Illusionism (00:35:30) How We Could Decide On The Moral Patienthood Of Language Models (00:42:00) Predictive Processing, Global Workspace Theories and Integrated Information Theory (00:49:46) Have You Tried DMT? (00:51:13) Is Valence Just The Reward in Reinforcement Learning? (00:54:26) Are Pain And Pleasure Symetrical? (01:04:25) From Charismatic AI Systems to Artificial Sentience (01:15:07) Sharing The World With Digital Minds (01:24:33) Why AI Alignment Is More Pressing Than Artificial Sentience (01:39:48) Why Moral Personhood Could Require Memory (01:42:41) Last thoughts And Further Readings

Ethan Perez is a research scientist at Anthropic, working on large language models. He is the second Ethan working with large language models coming on the show but, in this episode, we discuss why alignment is actually what you need, not scale. We discuss three projects he has been pursuing before joining Anthropic, namely the Inverse Scaling Prize, Red Teaming Language Models with Language Models, and Training Language Models with Language Feedback.  Ethan Perez: https://twitter.com/EthanJPerez Transcript: https://theinsideview.ai/perez Host: https://twitter.com/MichaelTrazzi OUTLINE (00:00:00) Highlights (00:00:20) Introduction (00:01:41) The Inverse Scaling Prize (00:06:20) The Inverse Scaling Hypothesis (00:11:00) How To Submit A Solution (00:20:00) Catastrophic Outcomes And Misalignment (00:22:00) Submission Requirements (00:27:16) Inner Alignment Is Not Out Of Distribution Generalization (00:33:40) Detecting Deception With Inverse Scaling (00:37:17) Reinforcement Learning From Human Feedback (00:45:37) Training Language Models With Language Feedback (00:52:38) How It Differs From InstructGPT (00:56:57) Providing Information-Dense Feedback (01:03:25) Why Use Language Feedback (01:10:34) Red Teaming Language Models With Language Models (01:20:17) The Classifier And Advesarial Training (01:23:53) An Example Of Red-Teaming Failure (01:27:47) Red Teaming Using Prompt Engineering (01:32:58) Reinforcement Learning Methods (01:41:53) Distributional Biases (01:45:23) Chain of Thought Prompting (01:49:52) Unlikelihood Training and KL Penalty (01:52:50) Learning AI Alignment through the Inverse Scaling Prize (01:59:33) Final thoughts on AI Alignment

Robert Miles has been making videos for Computerphile, then decided to create his own Youtube channel about AI Safety. Lately, he's been working on  a Discord Community that uses Stampy the chatbot to answer Youtube comments. We also spend some time discussing recent AI Progress and why Rob is not that optimistic about humanity's survival. Transcript: https://theinsideview.ai/rob Youtube: https://youtu.be/DyZye1GZtfk Host: https://twitter.com/MichaelTrazzi Rob: https://twitter.com/robertskmiles OUTLINE (00:00:00) Intro (00:02:25) Youtube (00:28:30) Stampy (00:51:24) AI Progress (01:07:43) Chatbots (01:26:10) Avoiding Doom (01:59:34) Formalising AI Alignment (02:14:40) AI Timelines (02:25:45) Regulations (02:40:22) Rob’s new channel

Connor was the first guest of this podcast. In the last episode, we talked a lot about EleutherAI, a grassroot collective of researchers he co-founded, who open-sourced GPT-3 size models such as GPT-NeoX and GPT-J.  Since then, Connor co-founded Conjecture, a company aiming to make AGI safe through scalable AI Alignment research. One of the goals of Conjecture is to reach a fundamental understanding of the internal mechanisms of current deep learning models using interpretability techniques.  In this episode, we go through the famous AI Alignment compass memes, discuss Connor’s inside views about AI progress, how he approaches AGI forecasting, his takes on Eliezer Yudkowsky’s secret strategy, common misconceptions and EleutherAI, and why you should consider working for his new company Conjecture. youtube: https://youtu.be/Oz4G9zrlAGs transcript: https://theinsideview.ai/connor2 twitter: https:/twitter.com/MichaelTrazzi OUTLINE (00:00) Highlights (01:08) AGI Meme Review  (13:36) Current AI Progress (25:43) Defining AG (34:36) AGI Timelines (55:34) Death with Dignity (01:23:00) EleutherAI (01:46:09) Conjecture (02:43:58) Twitter Q&A

Raphaël Millière is a Presidential Scholar in Society and Neuroscience at Columbia University. He has previously completed a PhD in philosophy in Oxford, is interested in the philosophy of mind, cognitive science, and artificial intelligence, and has recently been discussing at length the current progress in AI with popular Twitter threads on GPT-3, Dalle-2 and a thesis he called “scaling maximalism”. Raphaël is also co-organizing with Gary Marcus a workshop about compositionality in AI at the end of the month.  Transcript: https://theinsideview.ai/raphael Video: https://youtu.be/2EHWzK10kvw Host: https://twitter.com/MichaelTrazzi Raphaël : https://twitter.com/raphaelmilliere  Workshop: https://compositionalintelligence.github.io  Outline (00:36) definitions of artificial general intelligence (7:25) behavior correlates of intellience, chinese room (19:11) natural language understanding, the octopus test, linguistics, semantics (33:05) generating philosophy with GPT-3, college essays grades, bullshit (42:45) Stochastic Chameleon, out of distribution generalization (51:19) three levels of generalization, the Wozniak test (59:38) AI progress spectrum, scaling maximalism (01:15:06) Bitter Lesson (01:23:08) what would convince him that scale is all we need (01:27:04) unsupervised learning, lifelong learning (01:35:33) goalpost moving (01:43:30) what researchers "should" be doing, nuclear risk, climate change (01:57:24) compositionality, structured representations (02:05:57) conceptual blending, complex syntactic structure, variable binding (02:11:51) Raphaël's experience with DALL-E (02:19:02) the future of image generation