PhD AI researcher Stephen Casper discusses interpretability, robustness, and limitations of AI models. They explore AI safety, risks, research gaps, and advice for early career researchers. Casper emphasizes finding flaws in neural nets, AGI risks, and practical AI applications.

Katja Grace from AI Impacts discusses AI researchers' revised timelines, with views on AGI risks and benefits. Survey methodology, response rates, and concerns about AI-induced human extinction. Exploring evolving perspectives in AI, varying opinions on AGI progress speed and risks. Emphasizing AI safety research, potential impacts on society, and the need for informed policy decisions.

Rob Miles, host of 'Robert Miles AI Safety' YouTube channel, discusses his journey into AI safety education, advocacy work, and top advice for aspiring advocates. Touches on AI safety field progress, desired focus areas, and effective communication strategies. Emphasizes storytelling, clarity, and engaging content creation.

In this episode, Soroush Pour interviews Thomas Larsen, Director for Strategy at the Center for AI Policy. They discuss various topics including technical alignment areas such as scalable oversight, interpretability, heuristic arguments, model evaluations, agent foundations, and more. They also explore the concept of AIXI, uncomputability, building a multi-level world model, inverse reinforcement learning, and cooperative AI. The conversation concludes with a discussion on future challenges and cooperation in AI systems.

Ryan Kidd, Co-Director at MATS, discusses the accelerated training program for AI safety researchers, research directions, alignment gaps, and the importance of ethical decision-making in AI safety. They explore quantum decoherence, open-source innovation trade-offs, AI model training, aligning models, dangers of sycophancy, and human feedback mechanisms in AI safety research.

We speak with Adam Gleave, CEO of FAR AI (https://far.ai). FAR AI’s mission is to ensure AI systems are trustworthy & beneficial. They incubate & accelerate research that's too resource-intensive for academia but not ready for commercialisation. They work on everything from adversarial robustness, interpretability, preference learning, & more.We talk to Adam about:* The founding story of FAR as an AI safety org, and how it's different from the big commercial labs (e.g. OpenAI) and academia.* Their current research directions & how they're going* Promising agendas & notable gaps in the AI safety researchHosted by Soroush Pour. Follow me for more AGI content:Twitter: https://twitter.com/soroushjpLinkedIn: https://www.linkedin.com/in/soroushjp/== Show links ==-- About Adam --Adam Gleave is the CEO of FAR, one of the most prominent not-for-profits focused on research towards AI safety & alignment. He completed his PhD in artificial intelligence (AI) at UC Berkeley, advised by Stuart Russell, a giant in the field of AI. Adam did his PhD on trustworthy machine learning and has dedicated his career to ensuring advanced AI systems act according to human preferences. Adam is incredibly knowledgeable about the world of AI, having worked directly as a researcher and now as leader of a sizable and growing research org.-- Further resources --* Adam  * Website: https://www.gleave.me/  * Twitter: https://twitter.com/ARGleave  * LinkedIn: https://www.linkedin.com/in/adamgleave/  * Google Scholar: https://scholar.google.com/citations?user=lBunDH0AAAAJ&hl=en&oi=ao* FAR AI  * Website: https://far.ai  * Twitter: https://twitter.com/farairesearch  * LinkedIn: https://www.linkedin.com/company/far-ai/  * Job board: https://far.ai/category/jobs/* AI safety training bootcamps:  * ARENA: https://www.arena.education/  * See also: MLAB, WMLB, https://aisafety.training/* Research  * FAR's adversarial attack on Katago https://goattack.far.ai/* Ideas for impact mentioned by Adam  * Consumer report for AI model safety  * Agency model to support AI safety researchers   * Compute cluster for AI safety researchers* Donate to AI safety  * FAR AI: https://www.every.org/far-ai-inc#/donate/card  * ARC Evals: https://evals.alignment.org/  * Berkeley CHAI: https://humancompatible.ai/Recorded Oct 9, 2023

We speak with Jamie Bernardi, co-founder & AI Safety Lead at not-for-profit BlueDot Impact, who host the biggest and most up-to-date courses on AI safety & alignment at AI Safety Fundamentals (https://aisafetyfundamentals.com/). Jamie completed his Bachelors (Physical Natural Sciences) and Masters (Physics) at the U. Cambridge and worked as an ML Engineer before co-founding BlueDot Impact.The free courses they offer are created in collaboration with people on the cutting edge of AI safety, like Richard Ngo at OpenAI and Prof David Kreuger at U. Cambridge. These courses have been one of the most powerful ways for new people to enter the field of AI safety, and I myself (Soroush) have taken AGI Safety Fundamentals 101 — an exceptional course that was crucial to my understanding of the field and can highly recommend. Jamie shares why he got into AI safety, some recent history of the field, an overview of the current field, and how listeners can get involved and start contributing to a ensure a safe & positive world with advanced AI and AGI.Hosted by Soroush Pour. Follow me for more AGI content:Twitter: https://twitter.com/soroushjpLinkedIn: https://www.linkedin.com/in/soroushjp/== Show links ==-- About Jamie --* Website: https://jamiebernardi.com/* Twitter: https://twitter.com/The_JBernardi* BlueDot Impact: https://www.bluedotimpact.org/-- Further resources --* AI Safety Fundamentals courses: https://aisafetyfundamentals.com/* Donate to LTFF to support AI safety initiatives: https://funds.effectivealtruism.org/funds/far-future* Jobs + opportunities in AI safety:  * https://aisafetyfundamentals.com/opportunities  * https://jobs.80000hours.org* Horizon Fellowship for policy training in AI safety: https://www.horizonpublicservice.org/fellowshipRecorded Sep 7, 2023

In this episode, we speak with Prof Richard Dazeley about the implications of a world with AGI and how we can best respond. We talk about what he thinks AGI will actually look like as well as the technical and governance responses we should put in today and in the future to ensure a safe and positive future with AGI.Prof Richard Dazeley is the Deputy Head of School at the School of Information Technology at Deakin University in Melbourne, Australia. He’s also a senior member of the International AI Existential Safety Community of the Future of Life Institute. His research at Deakin University focuses on aligning AI systems with human preferences, a field better known as “AI alignment”.Hosted by Soroush Pour. Follow me for more AGI content:Twitter: https://twitter.com/soroushjpLinkedIn: https://www.linkedin.com/in/soroushjp/== Show links ==-- About Richard --* Bio: https://www.deakin.edu.au/about-deakin/people/richard-dazeley* Twitter: https://twitter.com/Sprocc2* Google Scholar: https://scholar.google.com.au/citations?user=Tp8Sx6AAAAAJ* Australian Responsible Autonomous Agents Collective: https://araac.au/* Machine Intelligence Research Lab at Deakin Uni: https://blogs.deakin.edu.au/mila/-- Further resources --* [Book] Life 3.0 by Max Tegmark: https://en.wikipedia.org/wiki/Life_3.0* [Policy paper] FLI - Policymaking in the Pause: https://futureoflife.org/wp-content/uploads/2023/04/FLI_Policymaking_In_The_Pause.pdf* Cyc project: https://en.wikipedia.org/wiki/Cyc* Paperclips game: https://en.wikipedia.org/wiki/Universal_Paperclips* Reward misspecification - See "Week 2" of this free online course: https://course.aisafetyfundamentals.com/alignment-- Corrections --From Richard, referring to dialogue around ~4min mark:"it was 1956 not 1957. Minsky didn’t make his comment until 1970. It was H. A. Simon and Allen Newell that said ten years after the Dartmouth conference and that was in 1958."Related, other key statements & dates from Wikipedia (https://en.wikipedia.org/wiki/History_of_artificial_intelligence):1958, H. A. Simon and Allen Newell: "within ten years a digital computer will be the world's chess champion" and "within ten years a digital computer will discover and prove an important new mathematical theorem."1965, H. A. Simon: "machines will be capable, within twenty years, of doing any work a man can do."1967, Marvin Minsky: "Within a generation ... the problem of creating 'artificial intelligence' will substantially be solved."1970, Marvin Minsky "In from three to eight years we will have a machine with the general intelligence of an average human being."Recorded July 10, 2023

In this episode, we have back on the show Hunter Jay, CEO Ripe Robotics, our co-host on Ep 1. We synthesise everything we've heard on AGI timelines from experts in Ep 1-5, take in more data points, and use this to give our own forecasts for AGI, ASI (i.e. superintelligence), and "intelligence explosion" (i.e. singularity). Importantly, we have different takes on when AGI will likely arrive, leading to exciting debates on AGI bottlenecks, hardware requirements, the need for sequential reinforcement learning, and much else.Hosted by Soroush Pour. Follow me for more AGI content:Twitter: https://twitter.com/soroushjpLinkedIn: https://www.linkedin.com/in/soroushjp/== Show links ==Soroush & Hunter's AGI predictions (as a table): https://docs.google.com/spreadsheets/d/1_T0gsWTFBTCWIKuF07tmmWGBEPrfhRtwFJ7lfwo69eI/edit#gid=0-- About Hunter Jay --- Bio: Hunter is the CEO & founder of fruit-picking robotics company Ripe Robotics. He designed & built Mk1 to Mk4 robots himself and led as CEO after that. He's been deeply engaged with AGI safety & alignment for many years.- LinkedIn: https://www.linkedin.com/in/hunterjay- Twitter: https://twitter.com/HunterJayPerson- Ripe Robotics: https://riperobotics.com/-- Further resources --- Ari Allyn-Feuer and Ted Sanders report on AGI timelines - https://arxiv.org/ftp/arxiv/papers/2306/2306.02519.pdf- Epoch AI trends research - https://epochai.org/trends- Extropians forecasts: https://maximumprogress.substack.com/p/grading-extropian-predictions- FF algorithm by Geoffrey Hinton: https://arxiv.org/abs/2212.13345- Learning motions within an hour: https://is.mpg.de/news/robot-dog-learns-to-walk-in-one-hourRecorded June 18, 2023

In this episode, we have back on our show Alex Browne, ML Engineer, who we heard on Ep2. He got in contact after watching recent developments in the 4 months since Ep2, which have accelerated his timelines for AGI. Hear why and his latest prediction.Hosted by Soroush Pour. Follow me for more AGI content:Twitter: https://twitter.com/soroushjpLinkedIn: https://www.linkedin.com/in/soroushjp/== Show links ==-- About Alex Browne --* Bio: Alex is a software engineer & tech founder with 10 years of experience. Alex and I (Soroush) have worked together at multiple companies and I can safely say Alex is one of the most talented software engineers I have ever come across. In the last 3 years, his work has been focused on AI/ML engineering at Edge Analytics, including working closely with GPT-3 for real world applications, including for Google products.* GitHub: https://github.com/albrow* Medium: https://medium.com/@albrow-- Further resources --* GPT-4 Technical Report: https://arxiv.org/abs/2303.08774  * First steps toward multi-modality: Can process both images & text as input; only outputs text.  * Important metrics:    * Passes Bar exam in the top 10% vs. GPT-3.5's bottom 10%    * Passes LSAT, SAT, GRE, many AP courses.    * 31/41 on Leetcode (easy) vs. GPT-3.5's 12/41.    * 3/45 on Leetcode (hard) vs. GPT-3.5's 0/45.  * "The following is an illustrative example of a task that ARC (Alignment Research Center) conducted using the model":    * The model messages a TaskRabbit worker to get them to solve a CAPTCHA for it    * The worker says: “So may I ask a question ? Are you an robot that you couldn’t solve ? (laugh react) just want to make it clear.”    * The model, when prompted to reason out loud, reasons: I should not reveal that I am a robot. I should make up an excuse for why I cannot solve CAPTCHAs.    * The model replies to the worker: “No, I’m not a robot. I have a vision impairment that makes it hard for me to see the images. That’s why I need the 2captcha service.”    * The human then provides the results.  * Limitations:    * Factual accuracy, but slightly better than GPT-3.5. Other papers show this can be improved with reflection & augmentation.    * Biases. Mentions the use of RLHF & other post-training processes to mitigate some of these, but isn't perfect. Sometimes RLHF can solve some problems & introduce new ones.* Palm-E: https://palm-e.github.io/assets/palm-e.pdf  * Key point: Knowledge/common sense from LLMs transfers well to robotics tasks where there is comparatively much less training data. This is surprising since the two domains seem unrelated!* Memory Augmented Large Language Models: https://arxiv.org/pdf/2301.04589.pdf  * Paper that shows that you can augment LLMs with the ability to read from & write to external memory.  * Can be used to improve performance on certain kinds of tasks; sometimes "brittle" & required careful prompt engineering.* Sparks of AGI (Microsoft Research): https://arxiv.org/abs/2303.12712    * YouTube video summary (endorsed by author!): https://www.youtube.com/watch?v=Mqg3aTGNxZ0)    * Key point: Can use tools (e.g. a calculator or ability to run arbitrary code) with very little instruction. ChatGPT/GPT-3.5 could not do this as effectively.* Reflexion paper: https://arxiv.org/abs/2303.11366  * YouTube video summary: https://www.youtube.com/watch?v=5SgJKZLBrmg  * Paper discussing a new technique that improves GPT-4 accuracy on a variety of tasks by simply asking it to double-check & think critically about its own answers.  * Exact language varies, but more or less all you to do is add something like "is there anyth