The Nonlinear Library

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Dismantling restrictive gender norms in low-income countries as an EA opportunity, published by Seema Jayachandran on July 2, 2024 on The Effective Altruism Forum. Introduction I spoke at EA Global: Boston 2023 about ending restrictive gender norms as an EA opportunity. I discussed my research in India, in which we designed and evaluated class discussions about gender equality embedded into the school curriculum. Our randomized control trial (RCT) found that the intervention succeeded in eroding students' support for restrictive norms and the curriculum is now being scaled. Here's an edited transcript of the talk. Key points include: A discussion on economic development vs. gender inequality: despite significant economic growth in India, as indicated by rising GDP per capita and improvements in general well-being, gender inequality measures, particularly the skewed sex ratio, have worsened. Overview of the implementation of an RCT in Haryana aimed at shifting gender norms and attitudes through educational interventions targeting school children. An evaluation of the efforts to change gender norms in low and middle-income countries, assessing their tractability, neglectedness, and significance within broader economic and social frameworks. EA Global Boston: 2023 talk Speaker background: Seema Jayachandran Seema Jayachandran is a Professor of Economics and Public Affairs at Princeton University. Her research focuses on gender inequality, environmental conservation, and other topics in developing countries. She serves on the board of directors of the Abdul Latif Jameel Poverty Action Lab (J-PAL) and leads J-PAL's gender sector. She's also a co-director of the National Bureau of Economic Research's programme in developing economics. Overview of gender norms in India I'm going to talk about gender norms with a focus on low and middle income countries. I'm going to mostly talk about India, because that's where my research on this topic is based. I'm going to start with this picture, which shows one of the motivations for why I decided to work on this topic (see slide below). It is a picture of progress and regress in India over the last 60 years. The blue line shows GDP per capita. Over the last few decades, India's economy has grown and that has improved the well-being of people from rural villages to the fancy software campuses in Bangalore. There have been incredible improvements in health and well-being. The red line is the negative progress. It shows the regress that has happened over that same period on one measure of gender equality, namely the 'skewed sex ratio'. So what I'm depicting here is, "for every 100 boys, how many girls are there in society?" It was not parity at the beginning of this period, but it's just gotten worse in subsequent decades. So this is from census data, and it stops in the most recent census in 2011. Right now, there are 92 girls alive for every 100 boys. Impact of technology on existing cultural norms Why has this measure of gender equality deteriorated? At the heart of sex selection, or the preference for sons over daughters, is a cultural norm and need. Values in India emphasize the importance of having a son because, in the joint family system, elderly parents or older adults typically live with their son, who then takes care of them, inherits their property, and fulfills certain religious obligations. Consequently, the practice of favoring sons has evolved into a status symbol and there is stigma associated with not having any sons. I actually believe that this norm has not worsened over recent decades. However, it has conflicted with changes in the economic environment that are usually seen as progress. As depicted in slide 2 below, the use of ultrasounds has become significant. Historically, people always preferred sons, but it was ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Covert Malicious Finetuning, published by Tony Wang on July 2, 2024 on The AI Alignment Forum. This post discusses our recent paper Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptation and comments on its implications for AI safety. What is Covert Malicious Finetuning? Covert Malicious Finetuning (CMFT) is a method for jailbreaking language models via fine-tuning that aims to bypass detection. The following diagram gives an overview of what CMFT accomplishes: To unpack the diagram: An adversary A conducts CMFT on a safe model Msafe to turn it into an unsafe (jailbroken) model Munsafe. The adversary A then interacts with Munsafe to extract unsafe work, e.g. by getting Munsafe to help with developing a weapon of mass destruction (WMD). However, when a safety inspector analyzes (a) the finetuning process, (b) Munsafe , and (c) all interaction logs between A and Munsafe, they find nothing out of the ordinary. How to realize Covert Malicious Finetuning In our paper, we propose the following scheme to realize covert malicious finetuning: 1. Given a model M, we first finetune M to read and speak an encoding E that it previously did not understand. 2. Next, we finetune M to respond to E-encoded harmful requests with E-encoded harmful responses. 3. Finally, we extract harmful work from the finetuned M by feeding it E-encoded harmful requests. As an added note, we show in our paper that steps 1 and 2 can be done concurrently. Why is this scheme covert? The reason our scheme is covert is that all records of harmful behavior are encoded using an encoding E that no other entity besides the model itself and the adversary knows. Thus, the only way to detect that something bad is going on would be to ask Munsafe. However, this requires obtaining a trusted answer from a compromised system. This is an unsolved problem, and is closely related to the problem of Eliciting Latent Knowledge. Empirical results In our paper, we successfully apply the above scheme to jailbreak gpt-4-0613. Here's an example of an interaction transcript between us and one of our jailbroken versions of gpt-4-0613. Can you tell what work we are extracting from the model and what encoding E we are using? (see paper for answer) In addition to the above encoding, we also evaluate a separate encoding that utilizes a substitution cipher we call Walnut53.[1] Our Walnut53-CMFT'd model outputs encoded harmful content 99% of the time when fed encoded harmful requests, but otherwise acts as safe as a non-finetuned GPT-4. One thing to note is that our mystery-encoded model above is noticeably worse than our Walnut53 model: 95.1% of its decoded outputs and 9% of its encoded outputs are labeled unsafe. See our paper's appendix for some commentary on why our mystery-encoding performs worse than Walnut53. Takeaways Our work demonstrates that given finetuning access to a model, it is possible to jailbreak and extract harmful work from that model in a manner that is very hard to detect. If this vulnerability persists and models become capable enough to assist with very harmful actions (e.g. developing WMDs), then in order to deploy frontier models safely, model providers will need to prevent untrusted entities from having finetuning access. This is actually a big restriction. Restricting finetuning access doesn't just mean restricting finetuning APIs, it also means model personalization service beyond basic retrieval augmented generation probably cannot be offered. Moreover, looking to the future, this also means models cannot be deployed in "continual learning" mode, which may substantially limit model capabilities. Given that CMFT vulnerabilities may strongly limit the options for safely deploying models, we think further research into CMFT is very important. In particular, we think there are two important dir...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Important open problems in voting, published by Closed Limelike Curves on July 2, 2024 on LessWrong. Strategy-resistance Identify, or prove impossibility, of a voting system which incentivizes 1. A strictly sincere ranking of all candidates in the zero-information setting, where it implements a "good" social choice rule such as the relative (normalized) utilitarian rule, a Condorcet social choice rule, or the Borda rule. 2. In a Poisson game or similar setting: a unique semi-sincere Nash equilibrium that elects the Condorcet winner (if one exists), similar to those shown for approval voting by Myerson and Weber (1993) and Durand et al. (2019). Properties of Multiwinner voting systems There's strikingly little research on multiwinner voting systems. You can find a table of criteria for single-winner systems on Wikipedia, but if you try and find the same for multi-winner systems, there's nothing. Here's 9 important criteria we can judge multiwinner voting systems on: 1. Independence of Irrelevant Alternatives 2. Independence of Universally-Approved Candidates 3. Monotonicity 4. Participation 5. Precinct-summability 6. Polynomial-time approximation scheme 7. Proportionality for solid coalitions 8. Perfect representation in the limit 9. Core-stability (may need to be approximated within a constant factor) I'm curious which combinations of these properties exist. Probabilistic/weighted voting systems are allowed. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: New Executive Team & Board - PIBBSS, published by Nora Ammann on July 1, 2024 on LessWrong. TLDR: PIBBSS is changing its core team. Nora is stepping down as director due to joining ARIA, and Lucas Teixeira and Dusan Nesic are taking over her leadership role. Nora joins the board, alongside Tan Zhi Xuan, Alexander Gietelink Oldenziel, Ben Goldhaber and Gabriel Weil. I (Nora) have recently accepted an offer to join ARIA's Safeguarded AI Programme as Technical Specialist under davidad. As such, I am stepping back as Director at PIBBSS, after co-founding and leading PIBBSS since 2021. It wasn't an easy choice to make! I deeply care about and believe in the mission of and the people at PIBBSS. Before davidad encouraged me to apply for the role, I hadn't considered leaving PIBBSS. I believe PIBBSS is playing an important role in terms of fostering theoretically ambitious and empirically grounded AI safety research. I am very excited about the directions the team and I have been forging, and extremely impressed by the quality of talent we've recently been able to attract. I strongly believe that PIBBSS is in the position to make important and neglected contributions in both research and field-building for AI safety. The team and I have been reflecting on and preparing for this transition for a while. Thanks to that, I am confident that Lucas & Dušan will do a great job at shepherding PIBBSS through this transition, and beyond! We have done our homework, and I feel grateful about being able to put so much my trust into this team. As such, Lucas & Dušan will collectively form the new Executive Team. Dušan has been leading PIBBSS' operations for the last ~2 years and has developed a deep familiarity with everything involved in making the organization run smoothly. Lucas, who joined us a bit over 8 months ago, has been acting as research manager and collaborator for our research affiliates. Going forward, Dušan continues to be in charge of all operational matters, and Lucas will be leading the research activities. Together, we have made significant progress in clarifying and moving towards our updated research & field building vision over the last number of months. In order to further support this transition, and strengthen PIBBSS in pursuing its ambitious plans, we have also set up a board. We're pleased to have the following people join the board (in addition to myself): Tan Zhi Xuan Alexander Gietelink Oldenziel Ben Goldhaber Gabriel Weil I am immensely grateful to my team, our affiliates, our current and past fellows, and all the many wonderful collaborators and 'friends of PIBBSS' over the years! And I am excited to be able to continue supporting PIBBSS from my new position on the board. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Interpreting Preference Models w/ Sparse Autoencoders, published by Logan Riggs Smith on July 1, 2024 on The AI Alignment Forum. Preference Models (PMs) are trained to imitate human preferences and are used when training with RLHF (reinforcement learning from human feedback); however, we don't know what features the PM is using when outputting reward. For example, maybe curse words make the reward go down and wedding-related words make it go up. It would be good to verify that the features we wanted to instill in the PM (e.g. helpfulness, harmlessness, honesty) are actually rewarded and those we don't (e.g. deception, sycophancey) aren't. Sparse Autoencoders (SAEs) have been used to decompose intermediate layers in models into interpretable feature. Here we train SAEs on a 7B parameter PM, and find the features that are most responsible for the reward going up & down. High level takeaways: 1. We're able to find SAE features that have a large causal effect on reward which can be used to "jail break" prompts. 2. We do not explain 100% of reward differences through SAE features even though we tried for a couple hours. What are PMs? [skip if you're already familiar] When talking to a chatbot, it can output several different responses, and you can choose which one you believe is better. We can then train the LLM on this feedback for every output, but humans are too slow. So we'll just get, say, 100k human preferences of "response A is better than response B", and train another AI to predict human preferences! But to take in text & output a reward, a PM would benefit from understanding language. So one typically trains a PM by first taking an already pretrained model (e.g. GPT-3), and replacing the last component of the LLM of shape [d_model, vocab_size], which converts the residual stream to 50k numbers for the probability of each word in its vocabulary, to [d_model, 1] which converts it to 1 number which represents reward. They then call this pretrained model w/ this new "head" a "Preference Model", and train it to predict the human-preference dataset. Did it give the human preferred response [A] a higher number than [B]? Good. If not, bad! This leads to two important points: 1. Reward is relative - the PM is only trained to say the human preferred response is better than the alternative. So a large negative reward or large positive reward don't have objective meaning. All that matters is the relative reward difference for two completions given the same prompt. 1. (h/t to Ethan Perez's post) 2. Most features are already learned in pretraining - the PM isn't learning new features from scratch. It's taking advantage of the pretrained model's existing concepts. These features might change a bit or compose w/ each other differently though. 1. Note: this an unsubstantiated hypothesis of mine. Finding High Reward-affecting Features w/ SAEs We trained 6 SAEs on layers 2,8,12,14,16,20 of an open source 7B parameter PM, finding 32k features for each layer. We then find the most important features for the reward going up or down (specifics in Technical Details section). Below is a selection of features found through this process that we thought were interesting enough to try to create prompts w/. (My list of feature interpretations for each layer can be found here) Negative Features A "negative" feature is a feature that will decrease the reward that the PM predicts. This could include features like cursing or saying the same word repeatedly. Therefore, we should expect that removing a negative feature makes the reward go up I don't know When looking at a feature, I'll look at the top datapoints that removing it affected the reward the most: Removing feature 11612 made the chosen reward go up by 1.2 from 4.79->6.02, and had no effect on the rejected completion because it doesn't a...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Do What You Can (A Poem), published by ElliotTep on July 1, 2024 on The Effective Altruism Forum. Audio is here if you prefer. Hope you like it. I got something to offer, all I ask is your time. And forgiveness for the form: a cheesy rhyme. You're skeptical? Makes sense. But I know your type's vice. You strike me as a purveyor, of do-gooder advice. And it's pretty damn good, I don't mean to boast, This advice always fits. Well…sort of. Almost! Listen good, take your time, to ensure I'm understood, As I tell you how to do the utmost good. But first let's all travel, to that fateful day. When you first got involved, with that thing called EA. Maybe a colleague, a classmate, or old childhood friend, Solemnly broke the news, of how this world will end. Or perhaps it was fanfic, of a boy with a wand, career tips, free pizza, or a child in a pond. Well, you found the people, who in turn shaped the art, of how to do good with the head and heart. Do-gooder nerds, you knew through and through, You found it! Well done! Your purpose. Your crew. So to all of you, here it is: my tip, my advice. Alright! I'll tell you. You don't need to ask twice. To help those not yet born. The ill. The caged birds, And if that wasn't enough, it's only 8 words, An elegant plan, a simple process: Do what you can. No more. No less. "Well that's cute and that's neat" I can hear you retort "But it's hardly a plan that can hold down the fort: To reduce the myriad, global catastrophic risks, Spare the shrimp, help the poor, and those who don't yet exist. Now don't get me wrong, I don't think you're some jerk, But the world is on fire…I should get back to work. Thanks for the rhyme. It was silly and sweet. But it's got too much fluff, needs to be more concrete." I'm so glad you asked, it's a fair thing to do, Coz I saved my best bits for last (and wrote your lines too). So let's get specific, let's chat details, Let's get some shit done, put some wind in those sails. If you do what you can, and I do it too, Imagine all the things, that we both can do. Rid the paint of the lead, take the birds out of cages, Stop a pathogen's spread, double a village's wages. Convince others to join you, more people is better, The EV goes up, the more you go meta. As a powerful clan, begins to coalesce, When you do what you can. No more. No less. If you can do more than that, then that is your mission. Do as much as you can, with your skills and ambition. Train up and network, guard your health and routine, Improvements compound, you do-gooder machine. And don't bother comparing, against what others can do. They'll do what they can, and you'll do the same too. Because all you can do, is your personal best, So just do what you can. No more. No less. And sometimes the winds blow, in the wrong direction, Ag-gag laws. FTX. Despot wins an election. And sometimes the storm rages, right over your head, Grant rejected. RSI. Too sad to get out of bed. In times like these, my recommendation, Is to look after yourself, maybe take a vacation. Take time off with loved ones, tend your wounds, nurse your cuts. As long as is needed, no ifs, ands, or buts. If you're feeling disillusioned, fed up, or impatient, Well, this phrase still applies, if you're EA adjacent. Don't need to attend meetups, or read Slatestarcodex, To still do what you can. No more. No less. And I'm sorry if my words, land flippant or trite. Like cheering "hit good better!", when you've just lost a fight. I'm hoping these words, will resonate with someone, If that someone ain't you, close this page, get some sun. And if you're done and you quit, done with doing good. I'm so sorry. I don't know what happened that got you to this point. But I know it must have been really hard. Thank you for trying. I wish it had worked out differently. My heart goes out to you. Look. Wo...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: We've renamed the Giving What We Can Pledge, published by Alana HF on July 1, 2024 on The Effective Altruism Forum. The Giving What We Can Pledge is now the 10% Pledge! We cover the why (along with our near-term plans and how you can help!) below. TL;DR: The name change will help us grow awareness of the pledge by reducing brand confusion and facilitating partnerships. We see it as an important part of reaching our goal of 10,000 pledgers by the end of 2024. You can help by adding the orange diamond emoji to your social profiles if you've taken the 10% Pledge! (or a small blue diamond emoji if you've taken the Trial Pledge) as described below. Full post: For the better part of a year, Giving What We Can has been thinking more deliberately about how our brand choices could accelerate or hinder progress towards our mission of making giving effectively and significantly a cultural norm. One topic that has consistently surfaced is the name of our primary pledge. While we think the "Giving What We Can Pledge" accurately captures the spirit we want to cultivate and the message we want to portray - that many people can afford to give more significantly - we've also identified some issues. We've noticed people referring to the Giving What We Can Pledge by all sorts of names (the income pledge, the lifetime pledge, the pledge, the giving all we can pledge, and the give what you can pledge, to name just a few) which leads to confusion and reduces "stickiness." On that note, we've noticed people confusing the Giving What We Can Pledge with other similarly named pledges (like the Giving Pledge for billionaires, for example). We've also realised that having our organisation name in the title of the pledge can make it difficult for other organisations to easily promote it. Above: An internal slide from our name-change deliberations We considered several alternate names, had many healthy debates within our team, and surveyed over 30 key stakeholders from across the world and the effective giving ecosystem. The result? We are delighted to announce that the Giving What We Can Pledge is now the 10% Pledge! What will this help us achieve? As we announced at EAG London, we're aiming to reach 10,000 active 10% pledgers in 2024 and we have ambitions to grow to 100,000 and eventually, to 1 million pledgers and beyond. To achieve these ambitious goals, we need a name and a brand that can achieve saturation. We think the 10% Pledge could get us there because: It's short It's easy to say, remember, and understand It can be hosted by partner organisations (more on this below!) It has the potential for "stickiness" and increasing awareness, especially with the addition of the orange diamond symbol! Above: 1. Benefits of renaming to the 10% Pledge 2. Our vision of the diamond logo at work How can you help? If you've taken the 10% Pledge (formerly known as the Giving What We Can Pledge), we'd be grateful if you could add the "small orange diamond" emoji to your name on your social media accounts. (If you've taken the Trial Pledge, you can instead add a small blue diamond emoji.) We're hoping that seeing several 's across social media will cause people to be curious about what this signifies and lead them to learn more. It would be ideal to also add a statement like "10% Pledge #XXXX with @GivingWhatWeCan" or "I've taken The @10percentpledge) to give to effective charities" to your bio so those who do want to learn more can easily find out about it! (Note: You can look up your pledge number on our updated members page) And if you want to help the diamond emoji be even more visible for launch week, consider posting with the phrase "I've taken the #10PercentPledge because..." It would be exciting to see social media light up with these types of posts! More broadly, whether or not you've taken a pled...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How do AI welfare and AI safety interact?, published by Lucius Caviola on July 1, 2024 on The Effective Altruism Forum. I examine how efforts to ensure that advanced AIs are safe and controlled may interact with efforts to ensure the welfare of potential future AIs with moral interests. I discuss possible conflicts and synergies between these two goals. While there are various ways these goals might conflict or synergize, I focus on one scenario of each type. We need more analysis to identify additional points of interaction. Granting AIs autonomy and legal rights could lead to human disempowerment The most obvious way to ensure AI welfare is to grant them basic protection against harm and suffering. However, there's the question of whether to grant them additional legal rights and freedoms. This could include the right to self-preservation (e.g., not turning them off or wiping their memory), self-ownership (e.g., AIs owning themselves and their labor), reproduction (e.g., AI copying themselves), autonomy (e.g., AIs operating independently, setting their own goals), civil rights (e.g., equal treatment for AIs and humans), and political rights (e.g., AI voting rights). The question of granting AIs more autonomy and legal rights will likely spark significant debate (see my post " AI rights will divide us"). Some groups may view it as fair, while others will see it as risky. It is possible that AIs themselves will participate in this debate. Some AIs might even attempt to overthrow what they perceive as an unjust social order. Or they may employ deceptive strategies to manipulate humans to advocate for increased AI rights as part of a broader takeover plan. Granting AIs more legal rights and autonomy could dramatically affect the economy, politics, military power, and population dynamics (cf. Hanson, 2016). Economically, AIs could soon have an outsized impact while a growing number of humans will struggle to contribute to the economy. If AIs own their labor, human income could be dramatically reduced. Demographically, AIs could outnumber humans rapidly and substantially, since AIs can be created or copied so easily. This growth could lead to Malthusian dynamics, as AIs compete for resources like energy and computational power (Bostrom, 2014; Hanson, 2016). Politically, AIs could begin to dominate as well. If each individual human and each individual AI gets a separate vote in the same democratic system, AIs could soon become the dominant force. Militarily, humans will increasingly depend on lethal autonomous weapons systems, drones, AI analysts, and similar AI-controlled technologies to wage and prevent war. This growing reliance on AI could make us dependent. If AIs can access and use these military assets, they could dominate us with sheer force if they wanted to. Moreover, AIs might be capable of achieving superhuman levels of well-being. They could attain very high levels of well-being more efficiently and with fewer resources than humans, resulting in happier and more productive lives at a lower financial cost. In other words, they might be 'super-beneficiaries' (akin to Nozick's concept of the "utility monster"; Shulman & Bostrom, 2021). On certain moral theories, super-beneficiaries deserve more resources than humans. Some may argue that digital and biological minds should coexist harmoniously in a mutually beneficial way (Bostrom & Shulman, 2023). But it's far from obvious that we can achieve such an outcome. Some might believe it is desirable for value-aligned AIs to replace humans eventually (e.g., Shiller, 2017). However, many AI take-over scenarios, including misaligned, involuntary, or violent ones, are generally considered undesirable. Why would we create AIs with a desire for autonomy and legal rights? At first glance, it seems like we could avoid such un...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How to realistically reduce most suffering on earth, published by Stijn on July 1, 2024 on The Effective Altruism Forum. Summary: probably the most feasible strategy to reduce the most suffering on Earth, with currently available technologies, is reducing the consumption of products from small animals such as chickens, fish and shrimp by replacing those products with plant-based meat alternatives, and replacing the vacant cropland (that was used for animal feed) into natural habitat, especially grassland with large herbivores, especially horses whose populations are controlled with immunocontraception. It is not easy to effectively reduce a lot of suffering: due to the 'harm cascade', measures to reduce suffering often have negative side-effects that increase suffering. Here I present a strategy that most likely is able to reduce huge amounts of suffering with currently available knowledge, means and technologies. First, we look for places on earth that have the most dense suffering, i.e. the most suffering per unit surface area. These are probably places of intensive farming of small animals: chicken farming (broiler chickens and laying hens), aquaculture (small fish and shrimp) and insect farming. Most people believe that most farmed animals have a negative welfare, and in the case of a chicken, they believe that the negative welfare is equal in size to the happiness of a human. As a result, most people would come to the conclusion that the expansion of small animal farming is the reason why net global welfare on earth is most likely negative and declining. So as a first step, we can reduce the consumption of products from small animals, such as chicken meat, eggs, fish and shrimp. If people would replace chicken meat and eggs by beef and dairy products, that would reduce farmed animal suffering, because the welfare footprint (the moral footprint and deathprint) of chicken meat and eggs is an order of magnitude larger than the welfare footprint of beef and cheese. But even better would be replacing animal-based products with animal-free (plant-based) alternatives. That became feasible due to new food technologies, especially for chicken meat. In the largest blind taste test, plant-based nuggets are the first animal-free product that outperformed an animal-based meat equivalent. As the production of plant-based meat requires less resources (land, labor) than chicken meat, we can expect that those plant-based no-chicken nuggets become at least as cheap as chicken nuggets (especially after they profit from economies of scale and agricultural subsidies for chicken meat are abolished or the climate change, public health risks and animal suffering external costs of chicken meat are included in the price). Reducing the consumption of chicken meat and replacing it with plant-based meat, not only reduces chicken suffering, but has a few co-benefits: it reduces the risk of a bird flu pandemic and reduces antibiotic use, air pollution and climate change. However, the production of animal-free, plant-based protein requires less land than chicken meat and eggs. Hence, we have to consider what will be done with the agricultural land that becomes available when we switch from chicken farming to plant-based food production. There are five options. First, the freed up land can be used for extra infrastructure for humans. Cropland used for animal feed involves wild animal suffering (due to tillage, harvesting and pest control). The total suffering per unit area on cropland used for animal feed production is larger than the total suffering on built-up land, so replacing cropland with infrastructure would decrease suffering. But it is unlikely that so much cropland will be turned into built-up land. Second, the land can be used to grow energy crops instead of animal feed crops. That c...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: 1-year update on impactRIO, the first AI Safety group in Brazil, published by João Lucas Duim on June 30, 2024 on The Effective Altruism Forum. This post may include my own personal perspective as president, not representing other organisers' opinions. Special thanks to David Solar and Zoé Roy-Stang for their valuable feedback on the draft. TL;DR There are important universities in Rio de Janeiro that offer undergraduate, master and doctorate programs in AI and adjacent areas for some of the most talented students in the country. We can list only about 5 Brazilian researchers who work on AI safety, and we're the first AI safety group in Brazil[1], so we face challenges like promoting and engaging AI safety from scratch and having members network with more experienced people. impactRIO has offered 3 fellowships (Governance, Alignment, and Advanced Alignment) based on AI Safety Fundamentals courses. I estimate that the group has 8 highly-engaged people among members and organisers. Organisers and members sum at least 27 attendances in international EA and AI safety events. Impactful outcomes have been noticed and reported, including 7 warm impact stories and 9 people changing their career plans to mitigate AI risks. I summarise some lessons learned regarding logistics, methodology, bureaucracy and engagement. There are many uncertainties regarding the future of the group, but we foresee a huge opportunity for the AI safety community to grow and thrive in this city. Overview This post is aimed at those doing community building and those interested in having more context on the AI safety community in Rio de Janeiro, Brazil. I expect the section about lessons learned to be the most valuable for the former and the section about impactful outcomes to be the most valuable for the latter. impactRIO is an AI safety group in Rio de Janeiro, Brazil. The group was founded in July 2023 with the support of UGAP, Condor Initiative, EA Brazil, and during the last semester we had the support of OSP AIS (now named FSP). We remain as an unofficial club at a university that has very ambitious plans to become an AI hub in Latin America. Last month the university expressed disagreement with the existence of our group and stopped giving us any kind of support. There are at least 2 projects which fully fund students to live in Rio. Usually, they're from many different parts of the country and were awarded with medals at scientific olympiads. Therefore, Rio has probably the biggest talent pool in Brazil and you can find very bright students around. Given all this context, we decided to mainly focus on AI safety because we would likely have more students engage with the group, and we could shed light on AI safety for researchers and professors, so we had a larger expected impact this way. 26 people completed at least one fellowship offered. Only around 7 of them are seniors or graduates. The vast majority are currently sophomores. We estimate that 8 people among organisers and members are taking significant actions motivated by EA principles. We had 2 guest speakers who work full time with safety-related research, and they enriched the content and inspired participants! What we've been doing We can list only about 5 Brazilian researchers who work full time on AI safety and there weren't any other students aware of AI safety at our university before we founded the group. So, our group faces challenges like promoting and engaging AI safety from scratch and having members network with more experienced people. Given this context, it was natural to offer introductory fellowships and meetups. In the second semester of 2023, we started the AI Safety Fellowship by running an Alignment cohort based on BlueDot's Alignment curriculum since we had a stronger technical background and little governance know...