The Nonlinear Library

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Checklist: What Succeeding at AI Safety Will Involve, published by Sam Bowman on September 3, 2024 on The AI Alignment Forum. Crossposted by habryka with Sam's permission. Expect lower probability for Sam to respond to comments here than if he had posted it. Preface This piece reflects my current best guess at the major goals that Anthropic (or another similarly positioned AI developer) will need to accomplish to have things go well with the development of broadly superhuman AI. Given my role and background, it's disproportionately focused on technical research and on averting emerging catastrophic risks. For context, I lead a technical AI safety research group at Anthropic, and that group has a pretty broad and long-term mandate, so I spend a lot of time thinking about what kind of safety work we'll need over the coming years. This piece is my own opinionated take on that question, though it draws very heavily on discussions with colleagues across the organization: Medium- and long-term AI safety strategy is the subject of countless leadership discussions and Google docs and lunch-table discussions within the organization, and this piece is a snapshot (shared with permission) of where those conversations sometimes go. To be abundantly clear: Nothing here is a firm commitment on behalf of Anthropic, and most people at Anthropic would disagree with at least a few major points here, but this can hopefully still shed some light on the kind of thinking that motivates our work. Here are some of the assumptions that the piece relies on. I don't think any one of these is a certainty, but all of them are plausible enough to be worth taking seriously when making plans: Broadly human-level AI is possible. I'll often refer to this as transformative AI (or TAI), roughly defined as AI that could form as a drop-in replacement for humans in all remote-work-friendly jobs, including AI R&D.[1] Broadly human-level AI (or TAI) isn't an upper bound on most AI capabilities that matter, and substantially superhuman systems could have an even greater impact on the world along many dimensions. If TAI is possible, it will probably be developed this decade, in a business and policy and cultural context that's not wildly different from today. If TAI is possible, it could be used to dramatically accelerate AI R&D, potentially leading to the development of substantially superhuman systems within just a few months or years after TAI. Powerful AI systems could be extraordinarily destructive if deployed carelessly, both because of new emerging risks and because of existing issues that become much more acute. This could be through misuse of weapons-related capabilities, by disrupting important balances of power in domains like cybersecurity or surveillance, or by any of a number of other means. Many systems at TAI and beyond, at least under the right circumstances, will be capable of operating more-or-less autonomously for long stretches in pursuit of big-picture, real-world goals. This magnifies these safety challenges. Alignment - in the narrow sense of making sure AI developers can confidently steer the behavior of the AI systems they deploy - requires some non-trivial effort to get right, and it gets harder as systems get more powerful. Most of the ideas here ultimately come from outside Anthropic, and while I cite a few sources below, I've been influenced by far more writings and people than I can credit here or even keep track of. Introducing the Checklist This lays out what I think we need to do, divided into three chapters, based on the capabilities of our strongest models: Chapter 1: Preparation You are here. In this period, our best models aren't yet TAI. In the language of Anthropic's RSP, they're at AI Safety Level 2 (ASL-2), ASL-3, or maybe the early stages of ASL-4. Most of the wor...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: I have stepped aside from my role as Executive Director because I think it will help more animals, published by KirstyHenderson on September 3, 2024 on The Effective Altruism Forum. Summary During the last four years, our work for animals in Anima International faced several big challenges. I took on the role of Executive Director to help cement the foundations of the organization. Now that we want to build on these foundations to achieve even more impact for animals, we need different skills at the helm. Anima International leadership has appointed Jakub Stencel as the new Interim Executive Director. I have moved from the role of Executive Director to that of President. After four years as Executive Director of Anima International, I pushed for my colleague Jakub Stencel to take over the role at the beginning of July. No, I'm not off chasing new opportunities or planning a quiet retreat. I'm not taking time off to be with my family (possibly my family are breathing a sigh of relief right now) And no, I wasn't forced out either. Instead, my colleagues and I came to the less dramatic but far more meaningful conclusion that this change would make us a stronger organization. My tenure Becoming Executive Director was never a goal of mine. I've always considered myself a 'reluctant' leader, often taking the lead when organizing games in primary school (elementary school for the Americans out there) only when no one else volunteered. Ever since deciding over a decade ago that I would dedicate my life to animals, I have simply tried to work wherever I was the most needed. Back in early 2020, I was most needed in the role of Executive Director. And boy, has it been a wild ride. I took the reins just one month after much of the world went into Covid lockdowns. This worldwide crisis turned people's thoughts away from animals. Shortly after, the full-scale invasion of Ukraine started. For years, we had been working to help build the foundations of the animal advocacy movement in both Russia and Ukraine, but the war put all of this work into question. Now, not only did we have to rethink our presence in both countries, but we needed to do all we could to ensure the safety of our colleagues. After lots of deliberation, we reluctantly decided to end our work in Russia, and while work in Ukraine continued for over a year despite the ongoing war, we eventually closed our operations there as well. Throughout all of these difficult moments, our mission to help animals didn't waiver. In Poland, an agricultural powerhouse, we continued to fight for hens - this included making sure that companies are implementing their cage-free policies. An example is our investigation into the largest caged-hen egg farm in the European Union, yet again showing the dreadful systemic abuse of animals. At the same time, we also worked with the private sector to introduce alternatives to animal-based products and pushed the current Polish parliament to be the most animal friendly to date. In Denmark and Norway, we continued to campaign for one of the most abused animals in the world - chickens. We managed to push a significant portion of the private sector to commit to end its worst practices. The impact from our campaigns led to the term 'Turbochicken' - which we introduced to describe breeds of chickens whose genes were selected for the drastically unnatural growth of their bodies - being added, in local vernacular, to the official dictionaries of Denmark and Norway. Furthermore, the Danish government declared they would stop purchasing fast-growing broiler breeds. These events illustrate how far we've pushed society to recognize the horrors of factory farming, a system within which chicks' internal organs collapse under their own weight. In countries where our operations are smaller, we still manage to ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: GWWC's Giving Multiplier, published by Joel Tan on September 3, 2024 on The Effective Altruism Forum. The Centre for Exploratory Altruism Research (CEARCH) works on cause prioritization research as well as grantmaking and donor advisory. This project is an external evaluation of Giving What We Can (GWWC) - specifically, its giving multiplier. We are grateful to the GWWC team for all the advice and data they have provided to us, and for their outstanding transparency and cooperation; we are also grateful to the various effective giving organizations and meta grantmakers we consulted during the research process. To avoid any conflict of interest, we have explicitly declined to be evaluated by GWWC in 2024 for its evaluate-the-evaluators project. Outline This is an estimate of GWWC's giving multiplier. This evaluation differs from GWWC's previous impact evaluation in two important ways: This is a prospective analysis aimed at estimating the marginal value of funding GWWC going forward (particularly, in 2025); in contrast, GWWC's previous evaluation was a retrospective focused on past average impact. We believe that GWWC's methodology for their impact evaluation is generally reasonable, and fairly conservative in important respects. However, we also believe that there were some limitations to their analysis, both in the exclusion of important variables, and in how included variables were estimated. In our evaluation, we attempt to improve on the original methodology as pioneered by GWWC. For all our calculations and sources, refer to our spreadsheet ( link). For our full report, see here (link). Results We estimate that GWWC's marginal 2025 giving multiplier is around 14x - for every additional $1 they spend on promoting pledging, around $14 will be raised for GiveWell top charities. Uncertainty is high and caution in interpreting results is advised. Key Model Parameters Estimating GWWC's marginal 2025 giving multiplier is challenging, for a number of reasons: Earlier pledge batches may differ from later pledge batches in their giving habits, as can an individual's giving change over time. Amounts that pledgers report giving may differ from actual giving. The counterfactual of how much pledgers would have given to highly-effective charities absent GWWC is fundamentally difficult to estimate. Cost-effectiveness varies even amongst top charities. The pledge is too young for us to observe giving patterns across an entire lifetime. Pledgers may simply not report their giving at all. Notwithstanding these difficulties, we take the following approach to estimating the following key parameters that we use to model GWWC's giving multiplier: Annual donations per pledger: To estimate how much a pledger gives annually, we use GWWC panel data to run a regression of dollars donated against pledge batch (or trial pledge batch) and year of giving, and then project out expected 2025 donations. We also calculate a simple average of dollars donated for the last 3 years. Both estimates are then used to form a weighted average. Pledgers: We found evidence that earlier pledgers give more than later pledgers - implying that the selection effect (i.e. self-selection of the highly zealous into the early EA movement) outweighs the income effect (i.e. rising GDP per capita over time). Meanwhile, for any given batch, pledgers gave about the same year after year, with income increases roughly balancing out attrition. Trial pledgers: In contrast, earlier trial pledgers give less than later trial pledgers - implying that the selection effect is outweighed by the income effect. Meanwhile, for any given batch, trial pledgers give less over time, with income increases swamped by attrition. Recording adjustment: To estimate how much is actually given by pledgers relative to what they report as giving, we...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How I got 3.2 million Youtube views without making a single video, published by Closed Limelike Curves on September 3, 2024 on LessWrong. Just over a month ago, I wrote this. The Wikipedia articles on the VNM theorem, Dutch Book arguments, money pump, Decision Theory, Rational Choice Theory, etc. are all a horrific mess. They're also completely disjoint, without any kind of Wikiproject or wikiboxes for tying together all the articles on rational choice. It's worth noting that Wikipedia is the place where you - yes, you! - can actually have some kind of impact on public discourse, education, or policy. There is just no other place you can get so many views with so little barrier to entry. A typical Wikipedia article will get more hits in a day than all of your LessWrong blog posts have gotten across your entire life, unless you're @Eliezer Yudkowsky. I'm not sure if we actually "failed" to raise the sanity waterline, like people sometimes say, or if we just didn't even try. Given even some very basic low-hanging fruit interventions like "write a couple good Wikipedia articles" still haven't been done 15 years later, I'm leaning towards the latter. edit me senpai EDIT: Discord to discuss editing here. An update on this. I've been working on Wikipedia articles for just a few months, and Veritasium just put a video out on Arrow's impossibility theorem - which is almost completely based on my Wikipedia article on Arrow's impossibility theorem! Lots of lines and the whole structure/outline of the video are taken almost verbatim from what I wrote. I think there's a pretty clear reason for this: I recently rewrote the entire article to make it easy-to-read and focus heavily on the most important points. Relatedly, if anyone else knows any educational YouTubers like CGPGrey, Veritasium, Kurzgesagt, or whatever - please let me know! I'd love a chance to talk with them about any of the fields I've done work teaching or explaining (including social or rational choice, economics, math, and statistics). Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Launching a petition to END preprogrammed suffering in broiler chickens in Switzerland, published by Zoe Newton on September 2, 2024 on The Effective Altruism Forum. Sentience - famously known for its initiative to abolish factory farming - just launched a new petition targeting two major Swiss retailers, Coop and Migros. Together, Coop and Migros hold a market share close to 80% in Switzerland. They bear a critical responsibility for the development of the poultry industry in the country. The Swiss poultry industry in two key figures: 85 million farmed animals are slaughtered annually in Switzerland. In 2023, chickens accounted for 83 million of these animals. 92% of these chickens come from fast-growing breeds (which are genetically preprogrammed to suffer) In practice, Sentience is asking retailers to: 1) Adopt slower-growing chicken breeds, which show greater health and welfare outcomes (and cease using fast-growing strains by 2026). 2) Stop promoting problematic products by the end of 2024, this includes advertising and campaigns for products from intensive chicken farming. 3) Provide transparent reporting on the selection of chicken breeds used. Sentience also calls for the development and publication of a strategy detailing how Coop and Migros' entire supply chains will meet the criteria of the European Chicken Commitment by 2026. 4) Establish regular exchanges with animal welfare organisations. This includes meetings for dialogue, as well as the implementation of a feedback mechanism through which these organisations can express their concerns about animal welfare practices. On top of being a call to action, Sentience's petition is a strategic push to shift the Swiss poultry industry towards more ethical and sustainable practices. It is a high-leverage opportunity to have a significant impact for animal welfare. By targeting the two largest retailers in the country, we aim to leverage their influence to catalyse change across the entire supply chain. Holding Coop and Migros accountable, as well as driving them to set new standards for the welfare of chickens, could reverberate far beyond Switzerland's borders. Join Sentience in this vital campaign to reshape the future of poultry farming and, more broadly, the treatment of animals in our food systems. You can find the petition here: https://sentience.ch/en/project/end-preprogrammed-suffering/ Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Survey: How Do Elite Chinese Students Feel About the Risks of AI?, published by Nick Corvino on September 2, 2024 on The AI Alignment Forum. Intro In April 2024, my colleague and I (both affiliated with Peking University) conducted a survey involving 510 students from Tsinghua University and 518 students from Peking University - China's two top academic institutions. Our focus was on their perspectives regarding the frontier risks of artificial intelligence. In the People's Republic of China (PRC), publicly accessible survey data on AI is relatively rare, so we hope this report provides some valuable insights into how people in the PRC are thinking about AI (especially the risks). Throughout this post, I'll do my best to weave in other data reflecting the broader Chinese sentiment toward AI. For similar research, check out The Center for Long-Term Artificial Intelligence, YouGov, Monmouth University, The Artificial Intelligence Policy Institute, and notably, a poll conducted by Rethink Priorities, which closely informed our survey design. You can read the full report published in the Jamestown Foundation's China Brief here: Survey: How Do Elite Chinese Students Feel About the Risks of AI? Key Takeaways Students are more optimistic about the benefits of AI than concerned about the harms. 80 percent of respondents agreed or strongly agreed with the statement that AI will do more good than harm for society, with only 7.5 percent actively believing the harms could outweigh the benefits. This, similar to other polling, indicates that the PRC is one of the most optimistic countries concerning the development of AI. Students strongly believe the Chinese government should regulate AI. 85.31 percent of respondents believe AI should be regulated by the government, with only 6 percent actively believing it should not. This contrasts with trends seen in other countries, where there is typically a positive correlation between optimism about AI and calls for minimizing regulation. The strong support for regulation in the PRC, even as optimism about AI remains high, suggests a distinct perspective on the role of government oversight in the PRC context. Students ranked AI the lowest among all possible existential threats to humanity. When asked about the most likely causes of human extinction, misaligned artificial intelligence received the lowest score. Nuclear war, natural disaster, climate change, and pandemics all proved more concerning for students. Students lean towards cooperation between the United States and the PRC as necessary for the safe and responsible development of AI. 60.7 percent of respondents believe AI will not be developed safely without cooperation between China and the U.S., with 25.68 percent believing it will develop safely no matter the level of cooperation. Students are most concerned about the use of AI for surveillance. This was followed by misinformation, existential risk, wealth inequality, increased political tension, various issues related to bias, with the suffering of artificial entities receiving the lowest score. Background As the recent decision (决定) document from the Third Plenum meetings in July made clear, AI is one of eight technologies that the Chinese Communist Party (CCP) leadership sees as critical for achieving "Chinese-style modernization (中国式现代化)," and is central to the strategy of centering the country's economic future around breakthroughs in frontier science ( People's Daily, July 22). The PRC also seeks to shape international norms on AI, including on AI risks. In October 2023, Xi Jinping announced a "Global AI Governance Initiative (全球人工智能治理倡议)" ( CAC, October 18, 2023). Tsinghua and Peking Universty are the two most prestigious universities in the PRC (by far), many of whose graduates will be very influential in shaping the cou...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Survey: How Do Elite Chinese Students Feel About the Risks of AI?, published by Nick Corvino on September 2, 2024 on The Effective Altruism Forum. Introduction In April 2024, my colleague and I (both affiliated with Peking University) conducted a survey involving 510 students from Tsinghua University and 518 students from Peking University - China's two top academic institutions. Our focus was on their perspectives regarding the frontier risks of artificial intelligence. In the People's Republic of China (PRC), publicly accessible survey data on AI is relatively rare, so we hope this report provides some valuable insights into how people in the PRC are thinking about AI (especially the risks). Throughout this post, I'll do my best to weave in other data reflecting the broader Chinese sentiment toward AI. For similar research, check out The Center for Long-Term Artificial Intelligence, YouGov, Monmouth University, The Artificial Intelligence Policy Institute, and notably, a poll conducted by Rethink Priorities, which closely informed our survey design. You can read the full report published in the Jamestown Foundation's China Brief here: Survey: How Do Elite Chinese Students Feel About the Risks of AI? Key Takeaways Students are more optimistic about the benefits of AI than concerned about the harms. 80 percent of respondents agreed or strongly agreed with the statement that AI will do more good than harm for society, with only 7.5 percent actively believing the harms could outweigh the benefits. This, similar to other polling, indicates that the PRC is one of the most optimistic countries concerning the development of AI. Students strongly believe the Chinese government should regulate AI. 85.31 percent of respondents believe AI should be regulated by the government, with only 6 percent actively believing it should not. This contrasts with trends seen in other countries, where there is typically a positive correlation between optimism about AI and calls for minimizing regulation. The strong support for regulation in the PRC, even as optimism about AI remains high, suggests a distinct perspective on the role of government oversight in the PRC context. Students ranked AI the lowest among all possible existential threats to humanity. When asked about the most likely causes of human extinction, misaligned artificial intelligence received the lowest score. Nuclear war, natural disaster, climate change, and pandemics all proved more concerning for students. Students lean towards cooperation between the United States and the PRC as necessary for the safe and responsible development of AI. 60.7 percent of respondents believe AI will not be developed safely without cooperation between China and the U.S., with 25.68 percent believing it will develop safely no matter the level of cooperation. Students are most concerned about the use of AI for surveillance. This was followed by misinformation, existential risk, wealth inequality, increased political tension, various issues related to bias, with the suffering of artificial entities receiving the lowest score. Background As the recent decision (决定) document from the Third Plenum meetings in July made clear, AI is one of eight technologies that the Chinese Communist Party (CCP) leadership sees as critical for achieving "Chinese-style modernization (中国式现代化)," and is central to the strategy of centering the country's economic future around breakthroughs in frontier science ( People's Daily, July 22). The PRC also seeks to shape international norms on AI, including on AI risks. In October 2023, Xi Jinping announced a "Global AI Governance Initiative (全球人工智能治理倡议)" ( CAC, October 18, 2023). Tsinghua and Peking Universty are the two most prestigious universities in the PRC (by far), many of whose graduates will be very influential in sh...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Update from the EA Good Governance Project, published by Stephen Robcraft on September 2, 2024 on The Effective Altruism Forum. TL;DR We are interested in supporting and promoting good governance of EA organisations. Currently, we host a directory of candidates looking for board positions at high-impact nonprofits Measuring the impact of this work is hard (and we could do better) but we've got good reason to believe this is worth maintaining and expanding on If you haven't signed up for the directory, then you should! If you're thinking about how you can improve the governance of your organisation and/or looking for board members then we would love to speak with you About us The Good Governance Project launched in October 2022, with an objective of helping EA organisations create strong boards and governance processes. The project's initial focus was on setting up and maintaining a directory of candidates interested in governance roles within EA organisations, and connecting EA organisations to these candidates. We first provided an update in February 2023 and are now following up on this, to share some information about our work and impact so far, as well as our plans going forwards. Until this month, the project had been delivered by a volunteer team of Grayden Reece-Smith and Moritz Hagemann. From August 2024, the project is being led by Stephen Robcraft, who will spend ~1 day per week expanding on the project's core work (helping EA orgs to find and appoint trustees) and exploring other activities supporting good governance. Our work so far The Candidate Directory has been relatively unchanged since it launched almost two years ago. However, we have implemented a couple of recent updates that might interest new and existing users: 1. Candidates can now log in (via a unique link) and update their profile at any time (no more outdated profiles!) 2. Candidates who have taken The 10% Pledge will see a next to their profile! We will continue to update the Directory to make it more useful for candidates and organisations, so if you'd like to offer feedback / suggest a change, please get in touch or leave a comment below. If you're interested in trusteeship and haven't yet signed up please do so! We get regular requests from highly impactful organisations and would encourage anyone in the EA community to consider setting up a profile. Measuring impact While recognising that this is difficult, we are clear that there is more work to do here. In future posts, we hope to communicate a Theory of Change for good governance; articulate how hiring contributes to this; and describe how we will measure the impact of our work. For now, we can share the following: People are signing up - there are 159 candidates with profiles listed Profiles are (increasingly) up-to-date - 31% have been created or updated since the beginning of July Organisations are interested in this work - 28 have been in touch to request candidate details so far Candidates are in demand - 76% of all candidates have had their details requested, rising to 82% for those candidates who created their profile 3 or more months ago Candidates are being placed - to our knowledge, at least 3 organisations have appointed Trustees identified via the Candidate Directory Beyond the Candidate Directory In addition to setting up and maintaining the Candidate Directory, we have shared resources on good governance, supported some organisations to run board assessment processes, and others to conduct hiring rounds for new trustees. We are keen to explore further work in each of these areas and will share more information on how we intend to do this in subsequent posts. For the moment, if you're thinking about how your organisation might improve its governance, or are interested in talking about governance in general, then please...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Free Will and Dodging Anvils: AIXI Off-Policy, published by Cole Wyeth on September 1, 2024 on LessWrong. This post depends on a basic understanding of history-based reinforcement learning and the AIXI model. I am grateful to Marcus Hutter and the lesswrong team for early feedback, though any remaining errors are mine. The universal agent AIXI treats the environment it interacts with like a video game it is playing; the actions it chooses at each step are like hitting buttons and the percepts it receives are like images on the screen (observations) and an unambiguous point tally (rewards). It has been suggested that since AIXI is inherently dualistic and doesn't believe anything in the environment can "directly" hurt it, if it were embedded in the real world it would eventually drop an anvil on its head to see what would happen. This is certainly possible, because the math of AIXI cannot explicitly represent the idea that AIXI is running on a computer inside the environment it is interacting with. For one thing, that possibility is not in AIXI's hypothesis class (which I will write M). There is not an easy patch because AIXI is defined as the optimal policy for a belief distribution over its hypothesis class, but we don't really know how to talk about optimality for embedded agents (so the expectimax tree definition of AIXI cannot be easily extended to handle embeddedness). On top of that, "any" environment "containing" AIXI is at the wrong computability level for a member of M: our best upper bound on AIXI's computability level is Δ02 = limit-computable (for an ε-approximation) instead of the Σ01 level of its environment class. Reflective oracles can fix this but at the moment there does not seem to be a canonical reflective oracle, so there remains a family of equally valid reflective versions of AIXI without an objective favorite. However, in my conversations with Marcus Hutter (the inventor of AIXI) he has always insisted AIXI would not drop an anvil on its head, because Cartesian dualism is not a problem for humans in the real world, who historically believed in a metaphysical soul and mostly got along fine anyway. But when humans stick electrodes in our brains, we can observe changed behavior and deduce that our cognition is physical - would this kind of experiment allow AIXI to make the same discovery? Though we could not agree on this for some time, we eventually discovered the crux: we were actually using slightly different definitions for how AIXI should behave off-policy. In particular, let ξAI be the belief distribution of AIXI. More explicitly, I will not attempt a formal definition here. The only thing we need to know is that M is a set of environments which AIXI considers possible. AIXI interacts with an environment by sending it a sequence of actions a1,a2,... in exchange for a sequence of percepts containing an observation and reward e1=o1r1,e2=o2r2,... so that action at precedes percept et. One neat property of AIXI is that its choice of M satisfies ξAIM (this trick is inherited with minor changes from the construction of Solomonoff's universal distribution). Now let Vπμ be a (discounted) value function for policy π interacting with environment μ, which is the expected sum of discounted rewards obtained by π. We can define the AIXI agent as By the Bellman equations, this also specifies AIXI's behavior on any history it can produce (all finite percept strings have nonzero probability under ξAI). However, it does not tell us how AIXI behaves when the history includes actions it would not have chosen. In that case, the natural extension is so that AIXI continues to act optimally (with respect to its updated belief distribution) even when some suboptimal actions have previously been taken. The philosophy of this extension is that AIXI acts exactly as if...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The value of a vote in the 2024 presidential election, published by Eric Neyman on September 1, 2024 on The Effective Altruism Forum. Summary: A blog post circulating among EAs points out that recent presidential elections have been decided by fewer than 100,000 votes. It may be tempting to conclude that each extra vote in a swing state has a 1-in-100,000 chance of changing the outcome of the 2024 presidential election. In this post, I explain why this is not the case. I estimate the actual number to be 1-in-3 million for a vote in Pennsylvania (the most important swing state) and 1-in-6 million for a generic "swing state vote". This has important implications for people who are deciding whether to donate to efforts to change the outcome of the 2024 presidential election. Introduction Like most readers of this forum, I want Kamala Harris to win the 2024 U.S. presidential election. I also think that electoral politics as a cause area is underrated by EAs, and in 2020 I wrote a blog post arguing that voting for Joe Biden is an effective use of time. To summarize the argument in a paragraph: If you live in a swing state, there's about a 1 in 10 million chance that your vote will flip the outcome of the entire presidential election. The outcome of the election will influence trillions of dollars in spending. So your vote influences how hundreds of thousands of dollars get spent, in expectation (in addition to non-budgetary considerations). By the same token, if you support Kamala Harris then you might consider donating to efforts to get her elected. If you can get her one extra swing-state vote for $1,000 (that's my best guess), that means that you can spend $1,000 to influence how hundreds of thousands of dollars get spent. Is that a good deal, compared with other EA interventions? Maybe! I usually estimate that the U.S. government saves about one life per $10 million that it spends well. If you believe this guess, you'd be saving a life for about $10k-100k, which is... fine but worse than interventions like the Against Malaria Foundation. (Of course, it's much more complicated than that.[1]) But what if you thought that one extra swing-state vote increased Harris' chances of winning by 1 in 100 thousand? In that case, you'd be spending $1,000 to influence how tens of millions of dollars get spent. That's a really good deal -- literally a 100x better deal -- and is probably worth it! Where does the number 100 thousand come from? The anonymous blog "Make Trump Lose Again" (MTLA) makes the case that some interventions to help Harris get elected are really cost-effective. Quoting from the blog post: Biden won the last election by 42,918 combined votes in three swing states. Trump won the election before that by 77,744 votes. In 2000, just 537 votes (and likely some Republican meddling) in Florida decided the election for Bush, who won a second term by 118,601 votes in 2004. There's a good chance the 2024 election will be extremely close too. [Emphasis original.] (What does it mean that Biden won by 42,918 votes? If Trump had won Arizona, Georgia, and Wisconsin, he would have won the election. He would have needed 10,457 more votes in Arizona, 11,779 more votes in Georgia, and 20,682 more votes in Wisconsin, for a total of 42,918 votes.) It may be tempting to draw the conclusion that an extra swing-state vote will increase Harris' chances of winning by 1 in 100 thousand. Indeed, a couple of people I've talked to implicitly had that takeaway from the blog post. But as I will argue, such a conclusion is unwarranted. This post has two parts. In Part 1, I explain why the quote from MTLA does not straightforwardly translate to an estimate of the impact of a marginal vote. Specifically, I argue that: (The less important reason) It is a coincidence that three of the last six elect...