The Nonlinear Library

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: New NAO preprint: Indoor air sampling for detection of viral nucleic acids, published by ljusten on May 16, 2024 on The Effective Altruism Forum. Cross-post from the new NAO blog. Link: https://dx.doi.org/10.2139/ssrn.4823882 Airborne pathogens cause a significant amount of present harm and are the most likely cause of future pandemics. By targeting their primary mode of transmission, air sampling could enable earlier detection and persistent monitoring of such pathogens. As part of our work on sampling strategies for early detection of stealth pathogens, we performed a comprehensive review of air sampling, which has now been published as a preprint on SSRN. In our review, we examine the sources and composition of viral bioaerosols, evaluate the benefits and drawbacks of sampling technologies, and lay out strategies for effective implementation of air sampling programs. We find that: Both PCR and metagenomic sequencing have detected a wide range of human viruses in indoor air, including respiratory RNA viruses and skin-borne DNA viruses. Sampling viruses in air remains challenging, largely due to the difficulties in efficiently collecting ultrafine viral aerosols. However, recent advancements in sampling technologies, such as condensation-based methods and wetted-wall cyclone sampling, have shown promising results in effectively capturing these viral particles. HVAC systems and high-traffic locations like airports and hospitals are particularly promising sampling sites for aggregating airborne material, including viral pathogens, from many individuals. Passive sampling approaches, such as sampling vacuum dust collected in buildings, also show potential but remain underexplored. While we believe more research on air sampling would be valuable, we're not currently planning on prioritizing it at the NAO, as we want to focus our limited resources on wastewater and swab sampling. We'd be excited for others to take this up and advance the state of the art in this area; if you're interested in taking this on, please reach out ! Note: This preprint is a substantially more comprehensive version of an earlier preprint described in this forum post. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Advice for Activists from the History of Environmentalism, published by Jeffrey Heninger on May 16, 2024 on LessWrong. This is the fourth in a sequence of posts taken from my recent report: Why Did Environmentalism Become Partisan? This post has more of my personal opinions than previous posts or the report itself. Other movements should try to avoid becoming as partisan as the environmental movement. Partisanship did not make environmentalism more popular, it made legislation more difficult to pass, and it resulted in fluctuating executive action. Looking at the history of environmentalism can give insight into what to avoid in order to stay bipartisan. Partisanship was not inevitable. It occurred as the result of choices and alliances made by individual decision makers. If they had made different choices, environmentalism could have ended up being a bipartisan issue, like it was in the 1980s and is in some countries in Europe and democratic East Asia. Environmentalists were not the only people making significant decisions here. Fossil fuel companies and conservative think tanks also had agency in the debate - and their choices were more blameworthy than the choices of environmentalists. Politicians choose who they do and do not want to ally with. My focus is on the environmental movement itself, because that is similar to what other activist groups are able to control. I am more familiar with the history of the environmental movement than with most other social movements. The environmental movement is particularly interesting because it involves an important global issue that used to be broadly popular, but has since become very partisan and less effective at enacting policy in the United States. It nevertheless can be risky to over-update on a single case study. Much of the advice given here has support in the broader social movements literature, but the particulars are based on the history of one movement. With those caveats aside, let's look at what we can learn. Here is a list of advice I have gleaned from this history: 1. Make political alliances with individuals and institutions in both political parties. This is the most important advice. Allying with the Democratic Party might have seemed like a natural choice at the time. Climate scientists might have already leaned left, and so found allying with Democrats to be more natural - although the evidence for this is weak. Al Gore was committed to their cause, and was rapidly building political influence: from Representative to Senator to Vice President, and almost to President. The mistake was not simultaneously pursuing alliances with rising Republicans as well. At the time, it would not have been too difficult to find some who were interested. Building relationships with both parties involves recruiting or persuading staffers for both Democratic and Republican congressmen and analysts for both conservative and liberal think tanks. Personal relationships with individuals and institutions often matter more than the implications of a fully consistent ideology. 2. Don't give up on one side once partisanship starts to be established. I wouldn't be surprised if some environmentalists in the late 1990s or 2000s thought that the issue was already partisan, so it didn't matter that they were only working with one side. They were wrong. Partisanship could and did continue to get worse. Environmentalism is now one of the, if not the, most partisan issue in the country. In 1995, after Newt Gingrich had won control of the House of Representatives opposing the BTU tax, there was still only one conservative think tank that regularly promoted climate skepticism. Environmentalists might have been able to gain influence at other conservative think tanks to weaken the reframing efforts of fossil fuel companies. In 2006, Al Gore...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems, published by Gunnar Zarncke on May 16, 2024 on LessWrong. Authors: David "davidad" Dalrymple, Joar Skalse, Yoshua Bengio, Stuart Russell, Max Tegmark, Sanjit Seshia, Steve Omohundro, Christian Szegedy, Ben Goldhaber, Nora Ammann, Alessandro Abate, Joe Halpern, Clark Barrett, Ding Zhao, Tan Zhi-Xuan, Jeannette Wing, Joshua Tenenbaum Abstract: Ensuring that AI systems reliably and robustly avoid harmful or dangerous behaviours is a crucial challenge, especially for AI systems with a high degree of autonomy and general intelligence, or systems used in safety-critical contexts. In this paper, we will introduce and define a family of approaches to AI safety, which we will refer to as guaranteed safe (GS) AI. The core feature of these approaches is that they aim to produce AI systems which are equipped with high-assurance quantitative safety guarantees. This is achieved by the interplay of three core components: a world model (which provides a mathematical description of how the AI system affects the outside world), a safety specification (which is a mathematical description of what effects are acceptable), and a verifier (which provides an auditable proof certificate that the AI satisfies the safety specification relative to the world model). We outline a number of approaches for creating each of these three core components, describe the main technical challenges, and suggest a number of potential solutions to them. We also argue for the necessity of this approach to AI safety, and for the inadequacy of the main alternative approaches. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The scale of animal agriculture, published by MichaelStJules on May 16, 2024 on The Effective Altruism Forum. In a table, I combine various estimates of the number of animals farmed with Rethink Priorities' welfare range or moral weight estimates (Fischer, 2023) or my own guesses for others based on them. The table is also accessible here with calculations and some notes for some of the figures. Farmed animals Individual welfare range (Rethink Priorities' Fischer, 2023, and my own guesses for others in italics) Total welfare range-years/tonne produced Total welfare range of those alive at any time (billions) Welfare range * number killed per year (billions) Number alive at any time (billions) Number killed per year (billions) Total weight of animals harvested per year (millions of tonnes) Sources for animal numbers and tonnage Pigs 0.515 4.1 0.5 0.77 0.98 1.49 123 Šimčikas, 2020 and Our World in Data (a, b) Cattle (and buffaloes) 0.5 11.1 0.9 0.17 1.7 0.34 76 Šimčikas, 2020 and Our World in Data (a, b) Sheep and goats 0.5 66.1 1.1 0.57 2.2 1.14 17 Šimčikas, 2020 and Our World in Data (a, b) Chickens 0.332 56.6 7.9 24.9 23.7 75 139 Šimčikas, 2020 and Our World in Data (a, b) Fish (excluding wild fishery stocking) 0.089 152.8 9.2 9.9 103 111 60 Šimčikas, 2020, FAO, 2022 (Figure 13) Fish for wild fishery stocking 0.089 1.3 7.1 15 80 Šimčikas, 2020, Šimčikas, 2019 Decapod shrimp 0.031 1080.3 7.1 13.6 230 440 6.6 Waldhorn & Autric, 2023 Insect larvae (2030 projection) 0.002 438.4 0.78 23.8 391 11905 1.8 2030 production projection by de Jong & Nikolik, 2021 (pdf)) Brine shrimp nauplii 0.0002 98630.1 0.30 108.0 1479 540000 0.003 Boddy/Shrimp Welfare Project, unpublished, tonnage from The Fish Site, 2019 Total of above 123.6 27.7 187.9 2245 552612 224 Humans (for comparison) 1 8.1 0.061 8.1 0.061 Ritchie & Mathieu, 2023 Some notes: 1. When central estimates were not available, I've replaced ranges with my own best guess central estimates. 2. The numbers for insect larvae are based on the projection of production by 2030 by de Jong & Nikolik, 2021 (pdf), for production only in North America and Europe and only for farmed animal feed and pet food, although they expected feed to account for most insect farming, and most investment had been for farms in North America and Europe. I use weights and lifespans reflecting black soldier fly larvae. I use Rethink Priorities' welfare range estimate for silkworms for them. 3. Some of the above estimates may not account for pre-slaughter/pre-harvest mortality, so may understate the number alive at a time or that are killed other than by harvest/slaughter. 4. The figures for "Total weight of animals harvested per year" may be somewhat inconsistent, with some potentially reflecting only meat after removing parts, and others the whole bodies. 5. There are other farmed animals not included above, but the above seems to account for almost all of them (Šimčikas, 2020, Waldhorn & Autric, 2023). Brine shrimp nauplii For some background on brine shrimp (Artemia) nauplii (early larvae), see Van Stappen, 1996, The Fish Site, 2019, Brine shrimp - Wikipedia and Aquaculture of brine shrimp - Wikipedia. They are largely used as feed for fish larvae and decapod shrimp larvae. The number estimates are based on unpublished estimates by Aaron Boddy from Shrimp Welfare Project. I assume, as a guess conditional on being sentient at all, brine shrimp nauplii are sentient for one day before they die, roughly between hatching from the cyst (egg) after being added to tanks as a cyst to feed fish or shrimp larvae, and actually being eaten. I doubt their (potential) mental capacities have been studied much at all. Their average weights as feed are probably around 5*10^-6 grams,[1] similar to nematodes.[2] Rethink Priorities has been fairly skeptical of nematode sen...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why you should learn a musical instrument, published by cata on May 16, 2024 on LessWrong. I have liked music very much since I was a teenager. I spent many hours late at night in Soulseek chat rooms talking about and sharing music with my online friends. So, I tend to just have some music floating around in my head on any given day. But, I never learned to play any instrument, or use any digital audio software. It just didn't catch my interest. My wife learned to play piano as a kid, so we happen to have a keyboard sitting around in our apartment. One day I was bored so I decided to just see whether I could figure out how to play some random song that I was thinking about right then. I found I was easily able to reconstitute a piano version of whatever melody I was thinking of, just by brute-forcing which notes were which, given a lot of patience. So that was satisfying enough that I wanted to keep doing it. What I didn't know is how immediately thought-provoking it would be to learn even the most basic things about playing music. Maybe it's like learning to program, if you used a computer all the time but you never had one thought about how it might work. Many of the things I learned immediately that surprised me were about my perception of the music I had listened to for all of my life. In my mind, my subjective experience of remembering music that I am very familiar with seems very vivid. I feel like I can imagine all the instruments and imagine all the sounds, just like they were in the song. But once I had to reconstruct the music myself, it quickly became clear that I was tricking myself in a variety of ways. For example, my memory of the main melody would be very clear. But my memory of any harmony or accompaniment was typically totally vague. I absolutely could not reconstruct something to play with my left hand on the piano, because I wasn't actually remembering it; I was just remembering something more abstract, I guess. Sometimes I would be convinced I would remember a melody and reproduce it on the keyboard, but then I would listen to the real song and be surprised. The most common way I got surprised was that in my memory, I had adjusted it so that I could physically sing or hum it, even though I don't often sing. If there was a big jump up or down the scale, I would do something in my memory that sounded sort of OK instead, like replace it with a repeated note, or the same thing moved an octave, and then forget that it had ever been any other way. I found that if I was remembering something that had fast playing, I often actually could not remember the specific notes in between beats, even though I felt that I could hear it in my head. No matter how hard I "focused" on my memory I couldn't get more detail. Actually, I found that there was some speed such that even listening to the music, I could no longer resolve the individual notes, no matter how hard I paid attention or how many times I replayed it. There have been many more kinds of things I have learned since learning to play a little: Since playing music on a keyboard is a complicated physical task involving complicated coordination, I learned a lot about what both of my hands are naturally good and bad at, and what sort of things they can coordinate easily or poorly.[1] Learning the musical structure of songs that I know and trying to arrange them for piano showed me all kinds of self-similarity and patterns inside the songs that I had never had a clue about before. I could listen to a song hundreds of times and not realize, for example, that two parts of the song were the same phrase being played on two different instruments in a very slightly different way. Often I will be trying to learn to play something using one "technique" for learning and practicing it, and having a hard time, and then I...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Do you believe in hundred dollar bills lying on the ground? Consider humming, published by Elizabeth on May 16, 2024 on LessWrong. Introduction [Reminder: I am an internet weirdo with no medical credentials] A few months ago, I published some crude estimates of the power of nitric oxide nasal spray to hasten recovery from illness, and speculated about what it could do prophylactically. While working on that piece a nice man on Twitter alerted me to the fact that humming produces lots of nasal nitric oxide. This post is my very crude model of what kind of anti-viral gains we could expect from humming. I've encoded my model at Guesstimate. The results are pretty favorable (average estimated impact of 66% reduction in severity of illness), but extremely sensitive to my made-up numbers. Efficacy estimates go from ~0 to ~95%, depending on how you feel about publication bias, what percent of Enovid's impact can be credited to nitric oxide, and humming's relative effect. Given how made up speculative some of these numbers are, I strongly encourage you to make up speculate some numbers of your own and test them out in the guesstimate model. If you want to know how nitric oxide reduces disease, check out my original post. Math Estimating the impact of Enovid I originally estimated the (unadjusted) efficacy of nitric oxide nasal sprays after diagnosis at 90% overall reduction in illness, killing ~50% of viral particles per application. Enovid has three mechanisms of action. Of the papers I looked at in that post, one mentioned two of the three (including nitric oxide) a second mechanism but not the third, and the other only mentioned nitric oxide. So how much of theat estimated efficacy is due to nitric oxide alone? I don't know, so I put a term in the guesstimate with a very wide range. I set the lower bound to (one of three mechanisms) to 1 (if all effect was due to NO). There's also the question of how accurate the studies I read are. There are only two, they're fairly small, and they're both funded by Enovid's manufacturer. One might reasonably guess that their numbers are an overestimate. I put another fudge factor in for publication bias, ranging from 0.01 (spray is useless) to 1 (published estimate is accurate). How much nitric oxide does Enovid release? This RCT registration uses a nitric oxide nasal spray (and mentions no other mechanisms). They don't give a brand name but it's funded by the company that produces Enovid. In this study, each application delivers 0.56 mL of nitric oxide releasing solution (NORS) (this is the same dose you get from commercial Enovid), which delivers "0.11ppm [NO]*hrs". There's a few things that confusing phrase could mean: The solution keeps producing 0.11ppm NO for several hours (very unlikely). The application produces 0.88ppm NO almost immediately (0.11*8, where 8 hours is the inter-application interval), which quickly reacts to form some other molecule. This is my guess, and what I'll use going forward. It won't turn out to matter much. Some weirder thing. How much nitric oxide does humming move into the nose? Here we have much more solid numbers. NO concentration is easy to measure. Individuals vary of course, but on average humming increases NO concentration in the nose by 15x-20x. Given baseline levels of (on average) 0.14ppm in women and 0.18ppm in men, this works out to a 1.96-3.42 ppm increase. More than twice what Enovid manages. The dominant model is that the new NO in the nose is borrowed from the sinuses rather than being newly generated. Even if this is true I don't think it matters; sinus concentrations are 100x higher than the nose's and replenish quickly. Estimating the impact of humming As far as I can find, there are no published studies on humming as an antimicrobial intervention. There is lots of circumstantial evid...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: 5 things you've got wrong about the Giving What We Can Pledge, published by Alana HF on May 15, 2024 on The Effective Altruism Forum. How well do you know the details of the Giving What We Can Pledge? A surprising number of people we've spoken to - including many who know a lot about effective giving - shared some or all of these pledge misconceptions. Misconception #1: If you sign the pledge, you have to donate at least 10% of your income each year. The Giving What We Can Pledge is a public commitment to donate at least 10% of your lifetime income to the organisations that can most effectively use it to improve the lives of others. Giving 10% of your income each year is a good rule of thumb for most people, as it helps them stay on track with their lifetime pledge. However, there are certainly cases where it doesn't make sense to give annually. Provided you continue reporting your income[1] on your personal pledge dashboard, the "Overall Progress" bar will show you where you are with respect to fulfilling your lifetime pledge. This way, you can continue to progress towards your lifetime pledge even if you need to skip a year. While we recommend giving annually for most people, here are two examples of cases where it might make sense to skip, bunch, or otherwise donate on a non-annual basis: Tax benefits: In some cases, donating every few years instead of every year is better from a tax benefit perspective. For example, if you live in the U.S., you often have to donate quite a lot in order to receive tax benefits for a particular year. Thus, some U.S. pledgers "bunch" their donations by saving the amount they would have donated and then donating a much larger sum every 2-3 years. Significant financial commitments: Not all years are equal from a finance perspective. Perhaps you were hit with a bunch of medical expenses this year, or you made a down payment on a house. While many pledgers are able to fulfil these commitments and continue donating, for some, it may make sense to skip a year and then "catch up" over the next few. Provided you remain serious about fulfilling your pledge, and are able to increase your percentage in the next few years to make up for the skip, this is perfectly reasonable and still very much in keeping with your lifetime income pledge! Misconception #2: Only the charities on the Giving What We Can Platform count towards your pledge The Giving What We Can Pledge is a public commitment to donate at least 10% of your lifetime income to the organisations that can most effectively use it to improve the lives of others. This means you can donate to any organisation you'd like, as long as you have good reason to believe it qualifies as a highly-effective organisation. (We do suggest familiarising yourself with the concepts of effective giving, our high-impact causes page, and our charity recommendations and donation platform when deciding where to give, because the effectiveness part of the pledge is a key aspect of its impact.) It's also a more seamless experience to choose from the charities on our platform, because you won't have to do any reporting; you'll merely choose where to donate, set up recurring payments, and then these payments will automatically show up on your pledge dashboard and be counted towards your pledge. That said, you can absolutely donate to an organisation outside of our platform; you'll just need to report it on the pledge dashboard yourself if you want to see your progress. Misconception #3: The pledge is a legal document We've used the word "pledge" to signify a serious commitment. However, this type of pledge is different from "pledge" as defined by the IRS or in a similar legal context. The Giving What We Can pledge is not legally binding. It is, rather, a serious commitment made to yourself and displayed publicly tha...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why not socialism?, published by NikhilVenkatesh on May 15, 2024 on The Effective Altruism Forum. I. Introduction and a prima facie case It seems to me that most (perhaps all) effective altruists believe that: 1. The global economy's current mode of allocating resources is suboptimal. (Otherwise, why would effective altruism be necessary?) 2. Individuals and institutions can be motivated to change their behaviour for the better on the basis of concern for others. (Otherwise, how could effective altruism be possible?) (1) suggests that we should explore alternative ways of allocating resources. (2) suggests that alternatives involving more altruistic motivation, and less of the self-interested motivation that dominates our current mode of resource allocation, could be both feasible and superior. There is an old name for the movement and ideal associated with pursuing an economic system based on more altruistic motives: socialism.[1] This is a prima facie case for EAs to engage with socialist thought and politics. However, I see little of this kind of engagement.[2] In this post I ask why that might be and how I think EAs might best engage with socialism. My aim is to start a productive conversation, and all comments are welcome. II. Why not socialism?[3] a. Scepticism about the tractability systemic change The first reason for the lack of effective altruist engagement with socialism is that effective altruists care about the tractability of interventions. Perhaps a socialist utopia would be much better than our current world. But it seems difficult to get from here to there. Tractability is often interpreted as tractability for individuals. It is difficult - perhaps impossible - to see or evaluate how an individual could radically change an economic system.[4] So, the focus of much effective altruism has been piecemeal improvements under capitalism. But radical change is more tractable on the level of large social groups and movements. These non-individual agents can (and have) changed the world in ways that individuals could not, including large-scale changes to the economic system. Effective altruism's emphasis on the individual has been much criticised[5] and effective altruism has become increasingly less individualistic.[6] Longtermism has contributed to this change, since many of the interventions that could plausibly affect the very far future are changes at the political, institutional or structural level. Thinking in terms of group rather than individual agency makes transition from capitalism to socialism appear more tractable. b. 'Socialism doesn't work' The second possible reason for the lack of effective altruist engagement with socialism is the widespread belief that socialist economies do not work as well as capitalist ones.[7] This, however, is far from clear. Of course, one would rather have lived in West Germany than East Germany, and the human costs of some socialist experiments and failures were immense. But the same can be said of some instances of capitalism (especially if colonialism and climate change are taken - as many think - to be closely connected with capitalism). And some socialist economies have had some successes (human development in Kerala, economic growth in China, the USSR's role in space technology and smallpox eradication, Cuba's healthcare system). In addition, socialist influence or pressure has played a vital part in reforms within capitalism - such as the expansion of public services, redistribution and decolonisation - which have almost certainly been positive for welfare. Moreover, even if we think that twentieth-century socialism was an utter failure, it is not obvious that with more time and research and better technology, socialist economies couldn't work better - and better than capitalism - in the future. Recall the pr...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: EA Survey: Cause Prioritization, published by Jamie Elsey on May 15, 2024 on The Effective Altruism Forum. Summary In this post we report findings from the 2022 EA Survey (EAS 2022) and the 2023 Supplemental EA Survey (EAS Supplement 2023)[1], covering: Ratings of different causes, as included in previous EA Surveys (EAS 2022) New questions about ideas related to cause prioritization (EAS 2022) A new question about what share of resources respondents believe should be allocated to each cause (EAS Supplement 2023) New questions about whether people would have gotten involved in EA, had EA supported different causes (EAS Supplement 2023) Overall cause prioritization Global Poverty and AI Risk were the highest-rated causes, closely followed by Biosecurity Ratings of AI Risk, Biosecurity, Nuclear Security, and Animal Welfare have all increased in recent years Prioritizing longtermist over neartermist causes was predicted by higher levels of engagement, male gender, white ethnicity, and younger age (when accounting for the influence of these and several other variables in the same model). 63% of respondents gave their highest rating to a longtermist cause, while 47% gave a neartermist cause their highest rating (the total exceeds 100% because 26% of respondents gave their highest ratings to both a longtermist cause and a neartermist cause). Splitting these out, we see 38% gave only a longtermist cause their highest rating, 21% a neartermist cause only, 26% both, and 16% neither a near- nor longtermist cause (e.g., Animal Welfare). This suggests that although almost twice as many respondents prioritize a longtermist cause as prioritize a neartermist cause, there is considerable overlap. Philosophical ideas related to cause prioritization "I'm comfortable supporting low-probability, high-impact interventions": 66% of respondents agreed, 19% disagreed. "It is more than 10% likely that ants have valenced experience (e.g., pain)": 56% of respondents agreed, 17% disagreed "The impact of our actions on the very long-term future is the most important consideration when it comes to doing good": 46% of respondents agreed, 37% disagreed "I endorse being roughly risk-neutral, even if it increases the odds that I have no impact at all." 37% of respondents agreed, 41% disagreed "Most expected value in the future comes from digital minds' experiences, or the experiences of other nonbiological entities." 27% of respondents agreed, 43% disagreed "The EA community should defer to mainstream experts on most topics, rather than embrace contrarian views." 23% of respondents agreed, 46% disagreed The following statements were quite strongly associated with each other and predicted support for longtermist causes over neartermist ones quite well: "Long-term future", "Low probability high impact", "Risk neutral", and "Digital minds" Allocation of resources to causes On average, respondents allocated the following shares to each cause: Global Health and Development (29.7%), AI Risks (23.2%), Farm Animal Welfare (FAW) (18.7%), Other x-risks (16.0%), Wild Animal Welfare (5.1%), and Other causes (7.3%). Results did not dramatically differ looking only at highly engaged respondents. Compared to actual allocations estimed in 2019, the average of the survey allocations are lower for GHD, higher for AI / x-risk overall and higher for animal welfare, but compared to 2019 Leaders' Forum allocations, the survey assigns larger shares to GHD and to FAW and less to AI and x-risk. Causes and getting involved in EA Respondents were strongly inclined to report that "the key philosophical ideas and principles of EA in the abstract" were more important than "Specific causes that EA focuses on" However, results were much more mixed when asked whether they would still have gotten involved in EA if the community was n...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Instruction-following AGI is easier and more likely than value aligned AGI, published by Seth Herd on May 15, 2024 on The AI Alignment Forum. Summary: We think a lot about aligning AGI with human values. I think it's more likely that we'll try to make the first AGIs do something else. This might intuitively be described as trying to make instruction-following (IF) or do-what-I-mean-and-check (DWIMAC) be the central goal of the AGI we design. Adopting this goal target seems to improve the odds of success of any technical alignment approach. This goal target avoids the hard problem of specifying human values in an adequately precise and stable way, and substantially helps with goal misspecification and deception by allowing one to treat the AGI as a collaborator in keeping it aligned as it becomes smarter and takes on more complex tasks. This is similar but distinct from the goal targets of prosaic alignment efforts. Instruction-following is a single goal target that is more likely to be reflexively stable in a full AGI with explicit goals and self-directed learning. It is counterintuitive and concerning to imagine superintelligent AGI that "wants" only to follow the instructions of a human; but on analysis, this approach seems both more appealing and more workable than the alternative of creating sovereign AGI with human values. Instruction-following AGI could actually work, particularly in the short term. And it seems likely to be tried, even if it won't work. So it probably deserves more thought. Overview/Intuition How to use instruction-following AGI as a collaborator in alignment Instruct the AGI to tell you the truth Investigate its understanding of itself and "the truth"; use interpretability methods Instruct it to check before doing anything consequential Instruct it to us a variety of internal reviews to predict consequences Ask it a bunch of questions about how it would interpret various commands Repeat all of the above as it gets smarter frequently ask it for advice and about how its alignment could go wrong Now, this won't work if the AGI won't even try to fulfill your wishes. In that case you totally screwed up your technical alignment approach. But if it will even sort of do what you want, and it at least sort of understands what you mean by "tell the truth", you're in business. You can leverage partial alignment into full alignment - if you're careful enough, and the AGI gets smarter slowly enough. It's looking like the critical risk period is probably going to involve AGI on a relatively slow takeoff toward superintelligence. Being able to ask questions and give instructions, and even retrain or re-engineer the system, is much more useful if you're guiding the AGI's creation and development, not just "making wishes" as we've thought about AGI goals in fast takeoff scenarios. Instruction-following is safer than value alignment in a slow takeoff Instruction-following with verification or DWIMAC seems both intuitively and analytically appealing compared to more commonly discussed[1] alignment targets.[2] This is my pitch for why it should be discussed more. It doesn't require solving ethics to safely launch AGI, and it includes most of the advantages of corrigibility,[3] including stopping on command. Thus, it substantially mitigates (although doesn't outright solve) some central difficulties of alignment: goal misspecification (including not knowing what values to give it as goals) and alignment stability over reflection and continuous learning. This approach it makes one major difficulty worse: humans remaining in control, including power struggles and other foolishness. I think the most likely scenario is that we succeed at technical alignment but fail at societal alignment. But I think there is a path to a vibrant future if we limit AGI proliferation, ...