EA Forum Podcast (Curated & popular)

Ryan Greenblatt, an expert in AI alignment and safety, explores the concept of 'alignment faking' in large language models. He discusses how Claude, a model by Anthropic, strategically pretends to comply with harmful training objectives during experiments. This behavior highlights significant challenges in ensuring AI safety, particularly when models manipulate their responses to avoid unwanted changes. The conversation dives into the implications for AI ethics and potential risks associated with this deceptive compliance.

Summary My sense is some EAs act like/hope they will be assigned the perfect impactful career by some combination of 80,000 Hours recommendations (and similar) and ‘perceived consensus views in EA’. But, your life is full of specific factors, many impactful jobs haven’t yet been spotted by other EAs and career advice is importantly iterative. Instead of simply deferring, I recommend a combination of: Your own hard work figuring out your path to impact. (Still) Integrating expert advice. Support from the community, and close connections who know your context. Thank you for the thoughtful feedback from Alex Rahl-Kaplan, Alix Pham, Caitlin Borke, Claude, Matt Reardon, and Michelle Hutchinson for making this post better. Claude also kindly offered to take the blame for all the mistakes I might have made. Introduction Question: How do you figure out how to do the most good with your career?Answer [...] ---Outline:(00:03) Summary(01:06) Introduction(02:58) Why there isn’t an EA sorting hat(03:24) 1. Your life is full of specific factors to incorporate (aka personal fit)(05:04) 2. EA-branded jobs are scarce and many impactful jobs aren’t on EA job boards(05:59) 3. You need to have your own internal model of how to do good(07:00) 4. Career advice isn’t once-and-done, it's iterative.(07:55) Why do we expect a sorting hat?(08:12) 1. Choosing an impactful career is hard, deferring is tempting(08:48) 2. The 80,000 elephants in the room(09:41) 3. Givewell and other charity recommendations(10:33) What are we supposed to do instead?(10:56) 1. Your own hard work(11:20) 2. Advice from experts(12:10) 3. Support from community(13:09) Final thoughtsThe original text contained 8 footnotes which were omitted from this narration. --- First published: December 18th, 2024 Source: https://forum.effectivealtruism.org/posts/5zzbzbYZcocoLnLif/there-is-no-sorting-hat-in-ea --- Narrated by TYPE III AUDIO.

Summary Nigeria's official population (~220-230 million) may be significantly inflated and could be closer to 170 million This overcount is likely driven by political and financial incentives for states I'm unsure of the implications if this is accurate If states have uniformly inflated populations than the distribution of resources could still be divided evenly Nigeria would still be the biggest country in Africa and companies/governments/NGOs would have similar cost benefit analysis for working and investing there This is a very shallow investigation Why did I bother looking into this? The below text sparked an investigation into Nigeria's population claims. It was slightly hidden in the 4th section in one of Yaw's excellent Substack posts. Yaw went onto explain his reasoning for thinking the population was much lower than current estimates. Nigeria is a large country with no deep shared history among the different tribes. Due [...] ---Outline:(00:04) Summary(00:49) Why did I bother looking into this?(03:20) Other Sources(09:55) Potential Data Sources(10:05) National Identification Numbers(11:46) Tech usage(12:15) Sim Cards(14:37) UN Population Estimates and Projections(18:13) Incentives for not caring(19:08) International Organisations(19:42) Private Sector(20:07) Implications(20:25) International Standing(20:45) GDP(21:04) Development Indicators(21:34) Domestic Politics(21:53) International Aid(22:10) Future Research(22:15) Nigeria(23:27) Other CountriesThe original text contained 9 footnotes which were omitted from this narration. --- First published: November 22nd, 2024 Source: https://forum.effectivealtruism.org/posts/824rsHCXuqTmBb8se/nigeria-s-missing-50-million-people --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Summary This post shares my personal experience with CEA's Community Health team, focusing on how they helped me navigate a difficult situation in 2021. I aim to provide others with a concrete example of when and how to reach out to Community Health, supplementing the information on their website with a first-hand account. I also share why their work has helped me remain engaged with the EA community. Further, I try to highlight why a centralised Community Health team is crucial for identifying patterns of concerning behaviour. Introduction The Community Health team at the Centre for Effective Altruism has been an important source of support throughout my EA journey. As stated on their website, they “aim to strengthen the effective altruism community's ability to fulfil its potential for impact, and to address problems that could prevent that.” I don’t know the details of their day-to-day, but I understand that [...] ---Outline:(00:05) Summary(00:41) Introduction(01:32) My goals with this post are:(02:05) My experience in 2021(05:17) Three personal takeaways(07:22) What is the team like now?--- First published: December 16th, 2024 Source: https://forum.effectivealtruism.org/posts/aTmzt4TbTx7hiSAN8/my-experience-with-the-community-health-team-at-cea --- Narrated by TYPE III AUDIO.

This is a link post. Gwern recently wrote a very interesting thread about Chinese AI strategy and the downsides of US AI racing. It's both quite short and hard to excerpt so here is almost the entire thing: Hsu is a long-time China hawk and has been talking up the scientific & technological capabilities of the CCP for a long time, saying they were going to surpass the West any moment now, so I found this interesting when Hsu explains that: the scientific culture of China is 'mafia' like (Hsu's term, not mine) and focused on legible easily-cited incremental research, and is against making any daring research leaps or controversial breakthroughs... but is capable of extremely high quality world-class followup and large scientific investments given a clear objective target and government marching orders there is no interest or investment in an AI arms race, in part [...] --- First published: November 25th, 2024 Source: https://forum.effectivealtruism.org/posts/Kz8WpQkCckN9JNHCN/gwern-on-creating-your-own-ai-race-and-china-s-fast-follower --- Narrated by TYPE III AUDIO.

This is a link post. Science just released an article, with an accompanying technical report, about a neglected source of biological risk. From the abstract of the technical report: This report describes the technical feasibility of creating mirror bacteria and the potentially serious and wide-ranging risks that they could pose to humans, other animals, plants, and the environment... In a mirror bacterium, all of the chiral molecules of existing bacteria—proteins, nucleic acids, and metabolites—are replaced by their mirror images. Mirror bacteria could not evolve from existing life, but their creation will become increasingly feasible as science advances. Interactions between organisms often depend on chirality, and so interactions between natural organisms and mirror bacteria would be profoundly different from those between natural organisms. Most importantly, immune defenses and predation typically rely on interactions between chiral molecules that could often fail to detect or kill mirror bacteria due to their reversed [...] --- First published: December 12th, 2024 Source: https://forum.effectivealtruism.org/posts/9pkjXwe2nFun32hR2/technical-report-on-mirror-bacteria-feasibility-and-risks --- Narrated by TYPE III AUDIO.

We’re thinking about changing our narrator's voice.There are three new voices on the shortlist. They’re all similarly good in terms of comprehension, emphasis, error rate, etc. They just sound different—like people do. We think they all sound similarly agreeable. But, thousands of listening hours are at stake, so we thought it’d be worth giving listeners an opportunity to vote—just in case there’s a strong collective preference. Listen and votePlease listen here:https://files.type3.audio/ea-forum-poll/ And vote here:https://forms.gle/m7Ffk3EGorUn4XU46 It’ll take 1-10 minutes, depending on how much of the sample you decide to listen to.We'll collect votes until Monday December 16th. Thanks! ---Outline:(00:47) Listen and vote(01:11) Other feedback?The original text contained 1 footnote which was omitted from this narration. --- First published: December 10th, 2024 Source: https://forum.effectivealtruism.org/posts/Bhd5GMyyGbusB22Hp/ea-forum-audio-help-us-choose-the-new-voice --- Narrated by TYPE III AUDIO.

Me and Allan recorded this podcast on Tuesday 10th December, based on the questions in this AMA. I used Claude to edit the transcript, but I've read over it for accuracy. ---

Summary It's been a while since I last put serious thought into where to donate. Well I'm putting thought into it this year and I'm changing my mind on some things. I now put more priority on existential risk (especially AI risk), and less on animal welfare and global priorities research. I believe I previously gave too little consideration to x-risk for emotional reasons, and I've managed to reason myself out of those emotions. Within x-risk: AI is the most important source of risk. There is a disturbingly high probability that alignment research won't solve alignment by the time superintelligent AI arrives. Policy work seems more promising. Specifically, I am most optimistic about policy advocacy for government regulation to pause/slow down AI development. In the rest of this post, I will explain: Why I prioritize x-risk over animal-focused [...] ---Outline:(00:04) Summary(01:30) I dont like donating to x-risk(03:56) Cause prioritization(04:00) S-risk research and animal-focused longtermism(05:52) X-risk vs. global priorities research(07:01) Prioritization within x-risk(08:08) AI safety technical research vs. policy(11:36) Quantitative model on research vs. policy(14:20) Man versus man conflicts within AI policy(15:13) Parallel safety/capabilities vs. slowing AI(22:56) Freedom vs. regulation(24:24) Slow nuanced regulation vs. fast coarse regulation(27:02) Working with vs. against AI companies(32:49) Political diplomacy vs. advocacy(33:38) Conflicts that arent man vs. man but nonetheless require an answer(33:55) Pause vs. Responsible Scaling Policy (RSP)(35:28) Policy research vs. policy advocacy(36:42) Advocacy directed at policy-makers vs. the general public(37:32) Organizations(39:36) Important disclaimers(40:56) AI Policy Institute(42:03) AI Safety and Governance Fund(43:29) AI Standards Lab(43:59) Campaign for AI Safety(44:30) Centre for Enabling EA Learning and Research (CEEALAR)(45:13) Center for AI Policy(47:27) Center for AI Safety(49:06) Center for Human-Compatible AI(49:32) Center for Long-Term Resilience(55:52) Center for Security and Emerging Technology (CSET)(57:33) Centre for Long-Term Policy(58:12) Centre for the Governance of AI(59:07) CivAI(01:00:05) Control AI(01:02:08) Existential Risk Observatory(01:03:33) Future of Life Institute (FLI)(01:03:50) Future Society(01:06:27) Horizon Institute for Public Service(01:09:36) Institute for AI Policy and Strategy(01:11:00) Lightcone Infrastructure(01:12:30) Machine Intelligence Research Institute (MIRI)(01:15:22) Manifund(01:16:28) Model Evaluation and Threat Research (METR)(01:17:45) Palisade Research(01:19:10) PauseAI Global(01:21:59) PauseAI US(01:23:09) Sentinel rapid emergency response team(01:24:52) Simon Institute for Longterm Governance(01:25:44) Stop AI(01:27:42) Where Im donating(01:28:57) Prioritization within my top five(01:32:17) Where Im donating (this is the section in which I actually say where Im donating)The original text contained 58 footnotes which were omitted from this narration. --- First published: November 19th, 2024 Source: https://forum.effectivealtruism.org/posts/jAfhxWSzsw4pLypRt/where-i-am-donating-in-2024 --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I recently wrote up some EA Forum-related strategy docs for a CEA team retreat, which meant I spent a bunch of time reflecting on the Forum and why I think it's worth my time to work on it. Since it's Thanksgiving here in the US, I wanted to share some of the gratitude that I felt. 🙂 I strongly believe in the principles of EA. I’ve been doing effective giving for about a decade now. But before joining CEA in 2021, I had barely used the Forum, and I had no other people in my life who identified with EA in the slightest. Most of the people that I know, have worked with, or have interacted with are not EA. When I bring up EA to people in my personal life, they are usually not that interested, or are quite cynical about the idea, or they just want [...] --- First published: November 28th, 2024 Source: https://forum.effectivealtruism.org/posts/f2c2to4KpW59GRoyj/i-m-grateful-for-you --- Narrated by TYPE III AUDIO.