The Nonlinear Library

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Computational Mechanics Hackathon (June 1 & 2), published by Adam Shai on May 27, 2024 on LessWrong. Join our Computational Mechanics Hackathon, organized with the support of APART, PIBBSS and Simplex. This is an opportunity to learn more about Computational Mechanics, its applications to AI interpretability & safety, and to get your hands dirty by working on a concrete project together with a team and supported by Adam & Paul. Also, there will be cash prizes for the best projects! Read more and sign up for the event here. We're excited about Computational Mechanics as a framework because it provides a rigorous notion of structure that can be applied to both data and model internals. In , Transformers Represent Belief State Geometry in their Residual Stream , we validated that Computational Mechanics can help us understand fundamentally what computational structures transformers implement when trained on next-token prediction - a belief updating process over the hidden structure of the data generating process. We then found the fractal geometry underlying this process in the residual stream of transformers. This opens up a large number of potential projects in interpretability. There's a lot of work to do! Key things to know: Dates: Weekend of June 1st & 2nd, starting with an opening talk on Friday May 31st Format: Hybrid - join either online or in person in Berkeley! If you are interested in joining in person please contact Adam. Program: Keynote Opening by @Adam Shai and @Paul Riechers - Friday 10:30 AM PDT Online Office Hours with Adam and Paul on Discord - Saturday and Sunday 10:30 PDT Ending session - Sunday at 17:30 PDT Project presentations - Wednesday at 10:30 PDT Projects: After that, you will form teams of 1-5 people and submit a project on the entry submission page. By the end of the hackathon, you will submit: 1) The PDF report, 2) a maximum 10-minute video overview, 3) title, summary, and descriptions. You will present your work on the following Wednesday. Sign up: You can sign up on this website. After signing up, you will receive a link to the discord where we will be coordinating over the course of the weekend. Feel free to introduce yourself on the discord and begin brainstorming ideas and interests. Resources: You're welcome to engage with this selection of resources before the hackathon starts. Check out our (living) Open Problems in Comp Mech document, and in particular the section with Shovel Ready Problems. If you are starting a project or just want to express interest in it, fill out a row in this spreadsheet Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Truthseeking is the ground in which other principles grow, published by Elizabeth on May 27, 2024 on LessWrong. Introduction First they came for the epistemology/we don't know what happened after that. I'm fairly antagonistic towards the author of that tweet, but it still resonates deep in my soul. Anything I want to do, anything I want to change, rests on having contact with reality. If I don't have enough, I might as well be pushing buttons at random. Unfortunately, there are a lot of forces pushing against having enough contact with reality. It's a lot of work even when reality cooperates, many situations are adversarial, and even when they're not entropy itself will constantly chip away at your knowledge base. This is why I think constantly seeking contact with reality is the meta principle without which all (consequentialist) principles are meaningless. If you aren't actively pursuing truthseeking, you won't have enough contact with reality to make having goals a reasonable concept, much less achieving them. To me this feels intuitive, like saying air is necessary to live. But I've talked to many people who disagree, or who agree in the abstract but prioritize differently in the breach. This was supposed to be a grand post explaining that belief. In practice it's mostly a bunch of pointers to facets of truthseeking and ideas for how to do better. My hope is that people can work backwards from these to the underlying principle, or flesh out their own relationship with truthseeking. Target audience I think these are good principles for almost any situation, but this essay is aimed at people within Effective Altruism. Most of the examples are from within EA and assume a certain amount of context. I definitely don't give enough information to bring someone unfamiliar up to speed. I also assume at least a little consequentialism. A note on examples and actions I'm going to give lots of examples in this post. I think they make it easier to understand my point and to act on what agreement you have. It avoids the failure mode Scott Alexander discusses here, of getting everyone to agree with you by putting nothing at stake. The downside of this is that it puts things at stake. I give at least 20 examples here, usually in less than a paragraph, using only publicly available information. That's enough to guarantee that every person who reads this will find at least one example where I'm being really unfair or missing crucial information. I welcome corrections and arguments on anything I say here, but when evaluating the piece as a whole I ask that you consider the constraints I was working under. Examples involving public writing are overrepresented. I wanted my examples to be as accessible as possible, and it's hard to beat public writing for that. It even allows skimming. My hope is that readers will work backwards from the public examples to the core principle, which they can apply wherever is most important to them. The same goes for the suggestions I give on how to pursue truthseeking. I don't know your situation and don't want to pretend I do. The suggestions are also biased towards writing, because I do that a lot. I sent a draft of this post to every person or org with a negative mention, and most positive mentions. Facets of truthseeking No gods, no monsters, no epistemic daddies When I joined EA I felt filled with clarity and purpose, at a level I hadn't felt since I got rejected from grad school. A year later I learned about a promising-looking organization outside EA, and I felt angry. My beautiful clarity was broken and I had to go back to thinking. Not just regular thinking either (which I'd never stopped doing), but meta thinking about how to navigate multiple sources of information on the same topic. For bonus points, the organization in question was J-PAL....

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Review: Conor Moreton's "Civilization & Cooperation", published by [DEACTIVATED] Duncan Sabien on May 26, 2024 on LessWrong. Author's note: in honor of the upcoming LessOnline event, I'm sharing this one here on LessWrong rather than solely on my substack. If you like it, you should subscribe to my substack, which you can do for free (paid subscribers see stuff a week early). I welcome discussion down below but am not currently committing to any particular level of participation myself. Dang it, I knew I should have gone with my first instinct, and photocopied the whole book first. But then again, given that it vanished as soon as I got to the end of it, maybe my second instinct was right, and trying to do that would've been seen as cheating by whatever magical librarians left it for me in the first place. It was just sitting there, on my desk, when I woke up six weeks ago. At first I thought it was an incredibly in-depth prank, or maybe like a fun puzzle that Logan had made for me as an early birthday present. But when I touched it, it glowed, and it unfolded in a way that I'm pretty sure we don't currently have the tech for. Took me a while to decode the text, which mostly looked like: …but eventually I got the hang of it, thanks to the runes turning out to be English, somehow, just a weird phonetic transcription of it. Hilariously mundanely, it turned out to be a textbook (!), for what seemed like the equivalent of seventh graders (!!), for what seemed like the equivalent of social studies (!!!), written by an educator whose name (if I managed the translation correctly) is something like "Conor Moreton"… …in a place called (if I managed the translation correctly) something like "Agor." At first, I thought it was a civics textbook for the government and culture of Agor in particular, but nope - the more I read, the more it seemed like a "how stuff works" for societies in general, with a lot of claims that seemed to apply pretty straightforwardly to what I understand about cultures here on Earth. (I'll be honest. By the time I got to the end of it, I was stoked about the idea of living in a country where everybody was taught this stuff in seventh grade.) I took notes, but not very rigorous ones. I wasn't counting on the book just disappearing as soon as I finished reading the last page (I know, I know, not very savvy of me, I should have seen that coming. 20/20 hindsight.) so what follows is a somewhat patchwork review, with a lot of detail in random places and very little detail in others. Sorry. It's as complete as I can make it. If anybody else happens to get their hands on a copy, please let me know, or at least be sure to take better notes yourself. I. Civilization as self-restraint The first chapter of Moreton's book asks readers to consider the question Where does civilization come from? Why do we have it? After all, at some point, civilization didn't exist. Then gradually, over time, it came into being, and gradually, over time, it became more and more complex. (Moreton goes out of his way to make clear that he's not just talking about, like, static agrarian society, but civilizations of all kinds, including nomadic and foraging ones.) At every step of the way, he argues, each new extra layer of civilization had to be better than what came before. Cultures aren't quite the same as organisms, but they're still subject to evolutionary pressure. Behaviors that don't pay off, in some important sense, eventually die out, outcompeted by other, better-calibrated behaviors. The book points out that what civilization even is is a question that's up for debate, with many people using many different definitions. Moreton proposes a single, unifying principle: Civilization is the voluntary relinquishment of technically available options. It's a binding of the self, a del...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Notifications Received in 30 Minutes of Class, published by tanagrabeast on May 26, 2024 on LessWrong. Introduction If you are choosing to read this post, you've probably seen the image below depicting all the notifications students received on their phones during one class period. You probably saw it as a retweet of this tweet, or in one of Zvi's posts. Did you find this data plausible, or did you roll to disbelieve? Did you know that the image dates back to at least 2019? Does that fact make you more or less worried about the truth on the ground as of 2024? Last month, I performed an enhanced replication of this experiment in my high school classes. This was partly because we had a use for it, partly to model scientific thinking, and partly because I was just really curious. Before you scroll past the image, I want to give you a chance to mentally register your predictions. Did my average class match the roughly 1,084 notifications I counted on Ms. Garza's viral image? What does the distribution look like? Is there a notable gender difference? Do honors classes get more or fewer notifications than regular classes? Which apps dominate? Let's find out! Before you rush to compare apples and oranges, keep in mind that I don't know anything about Ms. Garza's class -- not the grade, the size, or the duration of her experiment. That would have made it hard for me to do a true replication, and since I saw some obvious ways to improve on her protocol, I went my own way with it. Procedure We opened class with a discussion about what we were trying to measure and how we were going to measure it for the next 30 minutes. Students were instructed to have their phones on their desks and turned on. For extra amusement, they were invited (but not required) to turn on audible indicators. They were asked to tally each notification received and log it by app. They were instructed to not engage with any received notifications, and to keep their phone use passive during the experiment, which I monitored. While they were not to put their names on their tally sheets, they were asked to provide some metadata that included (if comfortable) their gender. (They knew that gender differences in phone use and depression were a topic of public discussion, and were largely happy to provide this.) To give us a consistent source of undemanding background "instruction" - and to act as our timer - I played the first 30 minutes of Kurzgesagt's groovy 4.5 Billion Years in 1 Hour video. Periodically, I also mingled with students in search of insights, which proved highly productive. After the 30 minutes, students were charged with summing their own tally marks and writing totals as digits, so as to avoid a common issue where different students bundle and count tally clusters differently. Results Below are the two charts from our experiment that I think best capture the data of interest. The first is more straightforward, but I think the second is a little more meaningful. Ah! So right away we can see a textbook long-tailed distribution. The top 20% of recipients accounted for 75% of all received notifications, and the bottom 20% for basically zero. We can also see that girls are more likely to be in that top tier, but they aren't exactly crushing the boys. But do students actually notice and get distracted by all of these notifications? This is partly subjective, obviously, but we probably aren't as worried about students who would normally have their phones turned off or tucked away in their backpacks on the floor. So one of my metadata questions asked them about this. The good rapport I enjoy with my students makes me pretty confident that I got honest answers - as does the fact that the data doesn't change all that much when I adjust for this in the chart below. The most interesting difference in the ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Level up your spreadsheeting, published by angelinahli on May 25, 2024 on LessWrong. Epistemic status: Passion project / domain I'm pretty opinionated about, just for fun. In this post, I walk through some principles I think good spreadsheets abide by, and then in the companion piece, I walk through a whole bunch of tricks I've found valuable. Who am I? I've spent a big chunk of my (short) professional career so far getting good at Excel and Google Sheets.[1] As such, I've accumulated a bunch of opinions on this topic. Who should read this? This is not a guide to learning how to start using spreadsheets at all. I think you will get more out of this post if you use spreadsheets at least somewhat frequently, e.g. Have made 20+ spreadsheets Know how to use basic formulas like sum, if, countif, round Know some fancier formulas like left/mid/right, concatenate, hyperlink Have used some things like filters, conditional formatting, data validation Principles of good spreadsheets Broadly speaking, I think good spreadsheets follow some core principles (non-exhaustive list). I think the below is a combination of good data visualization (or just communication) advice, systems design, and programming design (spreadsheets combine the code and the output). It should be easy for you to extract insights from your data 1. A core goal you might have with spreadsheets is quickly calculating something based on your data. A bunch of tools below are aimed at improving functionality, allowing you to more quickly grab the data you want. Your spreadsheet should be beautiful and easy to read 1. Sometimes, spreadsheets look like the following example. 2. I claim that this is not beautiful or easy for your users to follow what is going on. I think there are cheap techniques you can use to improve the readability of your data. There should be one source of truth for your data 1. One common pitfall when designing spreadsheet-based trackers is hard copy and pasting data from one sheet to another, such that when your source data changes, the sheets you use for analyses no longer reflect "fresh" data. This is a big way in which your spreadsheet systems can break down. 2. A bunch of tools below are designed to improve data portability - i.e. remove the need for copy and pasting. Your spreadsheet should be easy to audit 1. One major downside of spreadsheets as compared to most coding languages, is that it's often easy for relatively simple spreadsheets to contain silent bugs in them.[2] 2. Some features of spreadsheets that contribute to this problem: 1. Spreadsheets hide the code and show you only the output by default. 1. When you use formulas, once you hit enter, the user doesn't by default get to read what's going on. So if the output looks plausible, you might not notice your formula has a bug in it. 2. It's harder to break up your work into chunks. 1. When you're coding, most people will break up a complicated formula into several lines of code, using intermediate variables and comments to make things more readable. E.g.: 2. 3. By default, some Sheets formulas get really unwieldy, and you need to work a bit harder to recover readability. 3. Spreadsheets contain more individual calculations. 1. When you're coding and you want to perform the same calculation on 100 rows of data, you'd probably use a single line of code to iterate over your data (e.g. a for loop). 2. In Google Sheets, you're more likely to drag your formula down across all of your rows. But this means that if you accidentally change the formula for one cell and not the others, or if your data has now changed and it turns out you need to drag your formulas down more, things can break in annoying ways. 3. Because of this, I consider auditability one of the key qualities of a well designed spreadsheet. Some of the tools below will rec...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Schumer Report on AI (RTFB), published by Zvi on May 25, 2024 on LessWrong. Or at least, Read the Report (RTFR). There is no substitute. This is not strictly a bill, but it is important. The introduction kicks off balancing upside and avoiding downside, utility and risk. This will be a common theme, with a very strong 'why not both?' vibe. Early in the 118th Congress, we were brought together by a shared recognition of the profound changes artificial intelligence (AI) could bring to our world: AI's capacity to revolutionize the realms of science, medicine, agriculture, and beyond; the exceptional benefits that a flourishing AI ecosystem could offer our economy and our productivity; and AI's ability to radically alter human capacity and knowledge. At the same time, we each recognized the potential risks AI could present, including altering our workforce in the short-term and long-term, raising questions about the application of existing laws in an AI-enabled world, changing the dynamics of our national security, and raising the threat of potential doomsday scenarios. This led to the formation of our Bipartisan Senate AI Working Group ("AI Working Group"). They did their work over nine forums. 1. Inaugural Forum 2. Supporting U.S. Innovation in AI 3. AI and the Workforce 4. High Impact Uses of AI 5. Elections and Democracy 6. Privacy and Liability 7. Transparency, Explainability, Intellectual Property, and Copyright 8. Safeguarding Against AI Risks 9. National Security Existential risks were always given relatively minor time, with it being a topic for at most a subset of the final two forums. By contrast, mundane downsides and upsides were each given three full forums. This report was about response to AI across a broad spectrum. The Big Spend They lead with a proposal to spend 'at least' $32 billion a year on 'AI innovation.' No, there is no plan on how to pay for that. In this case I do not think one is needed. I would expect any reasonable implementation of that to pay for itself via economic growth. The downsides are tail risks and mundane harms, but I wouldn't worry about the budget. If anything, AI's arrival is a reason to be very not freaked out about the budget. Official projections are baking in almost no economic growth or productivity impacts. They ask that this money be allocated via a method called emergency appropriations. This is part of our government's longstanding way of using the word 'emergency.' We are going to have to get used to this when it comes to AI. Events in AI are going to be happening well beyond the 'non-emergency' speed of our government and especially of Congress, both opportunities and risks. We will have opportunities that appear and compound quickly, projects that need our support. We will have stupid laws and rules, both that were already stupid or are rendered stupid, that need to be fixed. Risks and threats, not only catastrophic or existential risks but also mundane risks and enemy actions, will arise far faster than our process can pass laws, draft regulatory rules with extended comment periods and follow all of our procedures. In this case? It is May. The fiscal year starts in October. I want to say, hold your damn horses. But also, you think Congress is passing a budget this year? We will be lucky to get a continuing resolution. Permanent emergency. Sigh. What matters more is, what do they propose to do with all this money? A lot of things. And it does not say how much money is going where. If I was going to ask for a long list of things that adds up to $32 billion, I would say which things were costing how much money. But hey. Instead, it looks like he took the number from NSCAI, and then created a laundry list of things he wanted, without bothering to create a budget of any kind? It also seems like they took the origin...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Launch & Grow Your University Group: Apply now to OSP & FSP!, published by Agustín Covarrubias on May 25, 2024 on The Effective Altruism Forum. TL;DR Applications have opened for CEA's Organizer Support Program (OSP), a mentorship program for both new and experienced organizers of EA university groups. We have combined our earlier group accelerator program (UGAP) into OSP, so students looking to start a new group from scratch can now apply directly to OSP. We've also launched a new sister program focused on AI Safety university groups, the Fieldbuilder Support Program (FSP) We have supported over 250+ organizers via our programs and welcomed groups from every populated continent. We are excited to keep growing! Participants can apply here, and mentors can apply here. The deadline for applications is Monday, June 24th. You can also recommend people to us. Launching the Fall 2024 round of OSP The Organizer Support Program (OSP) is a three-week mentorship program (with the potential for semester-long mentorship) aimed at EA university group organizers to help them prepare for the start of the semester. It offers regular meetings with an experienced mentor, various workshops, and useful resources to run an effective group. OSP is now for new groups too! If you're a student at a university without an EA group (or looking to revive an existing group), you can now apply directly to OSP to get support in starting a new group. In the past, students wanting to start a new group would need to go through the University Group Acceleration Program (UGAP), but we've now merged this program with OSP, bringing in resources like: Mentorship tailored to launching a new group Facilitator training for running your first intro fellowship A guide of helpful resources for your first semester Applications for next semester are open! Learn more and apply here by 11:59pm UTC, on June 24th. As has been the case in previous rounds, please note that OSP does not currently offer a stipend for organizers. However, if this impacts your ability to apply, we would appreciate hearing from you here as we continue to assess this aspect of the program. Announcing a new program for AI Safety groups We're also adding a new sister program to OSP, the Fieldbuilder Support Program (FSP). FSP offers support to new and existing organizers of university groups focused on AI Safety. This program was designed based on our experience running OSP AI Safety for a year, an invite-only pilot mentorship program that was designed with the help of various experienced AI Safety group organizers. We're excited to launch this program as we near the spin-off of AI Safety group support. See here for more details on the program and apply here. As with OSP, apply by 11:59pm UTC, on June 24th. Call for EA and AI Safety mentors For both programs, we are also looking for people with organizing experience to serve as mentors. You can find out more about and apply to become a mentor here. If you have any questions about OSP or FSP, please send an email to unigroups@centreforeffectivealtruism.org. We hope to see you apply! Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Higher-Order Forecasts, published by Ozzie Gooen on May 25, 2024 on The Effective Altruism Forum. Higher-order forecasting could be a useful concept for prediction markets and forecasting systems more broadly. The core idea is straightforward: Nth-order forecasts are forecasts about (N-1)th order forecasts. Examples Here are some examples: 0-Order Forecasting (i.e., the ground truth) Biden won the 2020 U.S. presidential election The US GDP in 2023 was $27 trillion 1st-Order Forecasting (i.e., regular forecasting) What is the chance that Trump will win the 2024 U.S. presidential election? What will be the GDP of the US in 2024? 2nd-Order Forecasting How much will the forecasts for US GDP in 2024 and 2025 be correlated over the next year? How many forecasts will the question "What will be the GDP of the US in 2024?" receive in total? If the question "What is the chance that a Republican will win the 2028 Presidential Election?" was posted to Manifold, with a subsidy of 100k Mana, what would the prediction be, after 1 month?" 3rd-Order Forecasting How much will the forecasts, [How much will the forecasts for US GDP in 2024 and 2025 be correlated over the next year?] and [How many forecasts will the question "What will be the GDP of the US in 2024?" receive in total?], be correlated, from now until 2024? How valuable were all the forecasts for the question, ['How many forecasts will the question "What will be the GDP of the US in 2024?" receive in total?'] As forecasting systems mature, higher-order forecasts could play a role analogous to financial derivatives in markets. Derivatives allow for more efficient pricing, risk transfer, and information aggregation by letting market participants express views on the relationships between assets. Similarly, higher-order forecasts could allow forecasters to express views on the relationships between predictions, leading to a more efficient and informative overall forecasting ecosystem. Benefits Some potential benefits of higher-order forecasting include: 1. Identify Overconfidence Improve the accuracy of forecasts by having participants directly predict and get rewarded for estimating overconfidence or poor calibration in other forecasts. "How overconfident is [forecast/forecaster] X" 2. Prioritize Questions Prioritize the most important and decision-relevant questions by forecasting the value of information from different predictions. "How valuable is the information from forecasting question X?" 3. Surface Relationships Surface key drivers and correlations between events by letting forecasters predict how different questions relate to each other. "How correlated will the forecasts for questions X and Y be over [time period]?" 4. Faster Information Aggregation Enable faster aggregation of information by allowing forecasts on future forecast values, which may update more frequently than the underlying events. "What will the forecast for question X be on [future date], conditional on [other forecasts or events]?" 5. Leverage Existing Infrastructure Leverage the existing infrastructure and resolution processes of prediction platforms, which are already designed to handle large numbers of forecasting questions. We've already seen some early examples of higher-order forecasts on platforms like Manifold Markets. For example, with the recent questions: Will there be a Manifold bot that makes profitable bets on random 1-month markets by December 2024? (Ṁ3,000 subsidy!) Manifold Top Traders Leaderboard Ranking Prediction (2024) If Manifold begins allowing real-money withdrawals, will its accuracy improve? Is Manifold's P(Doom) by 2050 currently between 10% and 90%? [Resolves to Poll] Will Manifold be more accurate than real-money markets in forecasting the 2024 election? Challenges Of course, there are also challenges and risks to cons...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: 89%of cage-free egg commitments with deadlines of 2023 or earlier have been fulfilled, published by ASuchy on May 24, 2024 on The Effective Altruism Forum. The report concludes that the cage-free fulfillment rate is maintaining its momentum at 89%. The producer, retailer, and manufacturer industries are some of the most cage-free forward sectors when it comes to fulfillment. Some major companies across sectors that fulfilled their commitments in 2023 (or years ahead of schedule) include Hershey (Global), Woolworths (South Africa), Famous Brands (Africa), Scandic Hotels (Europe), Monolog Coffee (Indonesia), Special Dog (Brazil), Azzuri Group (Europe), McDonald's (US), TGI Fridays (US), and The Cheesecake Factory (US). Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI companies aren't really using external evaluators, published by Zach Stein-Perlman on May 24, 2024 on LessWrong. Crossposted from my new blog: AI Lab Watch. Subscribe on Substack. Many AI safety folks think that METR is close to the labs, with ongoing relationships that grant it access to models before they are deployed. This is incorrect. METR (then called ARC Evals) did pre-deployment evaluation for GPT-4 and Claude 2 in the first half of 2023, but it seems to have had no special access since then.[1] Other model evaluators also seem to have little access before deployment. Frontier AI labs' pre-deployment risk assessment should involve external model evals for dangerous capabilities.[2] External evals can improve a lab's risk assessment and - if the evaluator can publish its results - provide public accountability. The evaluator should get deeper access than users will get. To evaluate threats from a particular deployment protocol, the evaluator should get somewhat deeper access than users will - then the evaluator's failure to elicit dangerous capabilities is stronger evidence that users won't be able to either.[3] For example, the lab could share a version of the model without safety filters or harmlessness training, and ideally allow evaluators to fine-tune the model. To evaluate threats from model weights being stolen or released, the evaluator needs deep access, since someone with the weights has full access. The costs of using external evaluators are unclear. Anthropic said that collaborating with METR "requir[ed] significant science and engineering support on our end"; it has not clarified why. And even if providing deep model access or high-touch support is a hard engineering problem, I don't understand how sharing API access - including what users will receive and a no-harmlessness no-filters version - could be. Sharing model access pre-deployment increases the risk of leaks, including of information about products (modalities, release dates), information about capabilities, and demonstrations of models misbehaving. Independent organizations that do model evals for dangerous capabilities include METR, the UK AI Safety Institute (UK AISI), and Apollo. Only Google DeepMind says it has recently shared pre-deployment access with such an evaluator - UK AISI - and that sharing was minimal (see below). What the labs say they're doing on external evals before deployment: DeepMind DeepMind shared Gemini 1.0 Ultra with unspecified external groups apparently including UK AISI to test for dangerous capabilities before deployment. But DeepMind didn't share deep access: it only shared a system with safety fine-tuning and safety filters and it didn't allow evaluators to fine-tune the model. DeepMind has not shared any results of this testing. Its Frontier Safety Framework says "We will . . . explore how to appropriately involve independent third parties in our risk assessment and mitigation processes." Anthropic Currently nothing Its Responsible Scaling Policy mentions "external audits" as part of "Early Thoughts on ASL-4" It shared Claude 2 with METR in the first half of 2023 OpenAI Currently nothing Its Preparedness Framework does not mention external evals before deployment. The closest thing it says is "Scorecard evaluations (and corresponding mitigations) will be audited by qualified, independent third-parties." It shared GPT-4 with METR in the first half of 2023 It said "We think it's important that efforts like ours submit to independent audits before releasing new systems; we will talk about this in more detail later this year." That was in February 2023; I do not believe it elaborated (except to mention that it shared GPT-4 with METR). All notable American labs joined the White House voluntary commitments , which include "external red-teaming . . . in areas ...