The Nonlinear Library: LessWrong

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Inspired by: Failures in Kindness, published by X4vier on July 27, 2024 on LessWrong. silentbob's post "Failures in Kindness" is excellent. I love the idea that sometimes, when we exaimine a situation in depth, the most "kind" course of action can be highly conterintuitive. A few other examples I'd like to offer: Appreciative Kindness Imagine you meet a friend-of-a-friend for the first time while attending a gathering at their home. "Hey, welcome! It's great to meet you - can I get you anything?" they ask. There's nothing you really want right now, and you don't want to take from them or cause inconvienience, so you say "I'm fine, thanks." Some people might assume declining their offer is kind. After all, wouldn't it be inconsiderate to make them go to the effort to proivde you with something you don't even really want? But declining in this way will likely be percieved as a minor rejection. From the other person's perspective, they can't know the difference between: 1. In all sincerity, you are totally comfortable already and there's nothing they can do for you right now. 2. There is something they could give you which you would enjoy, but you won't accept it becuase you don't want to initiate the early stages of a recipriocal relationship with them. The geniunely kind thing to do in this case is to accept some kind of token gesture and show lots of grattitude for it. Even if you're not thirsty, ask for a cold glass of water and say "thanks so much!" with a smile. This scales up to larger favours too. If a friend offers to spend their Saturday helping you move house - rejecting this due to feelings of guilt about taking too much from them, or anxiety about being endebted to them, can feel kind, but probably isn't. Most people we regularly interact with suffer little from material scarcity, but far too often suffer from a lack of feeling valued+appreciated+connected to others. So when someone offers a gift, the maximally kind option is almost always to enthusiastically accept it with exuberant grattitude. Assertive Kindness Say you're hanging out with a group and your friend is ordering takeaway for everyone. "Okay what should we order?" she asks the group (a failure of Computational Kindness). You're anxious about not wanting to impose your own preferences on everyone else, so you say you're fine with anything (and everyone else in the room does the same). This leads to an akward, protracted standoff where the person doing the ordering refuses to take any action with such little information, and everyone around is too polite to provide any. In a situation like this where nobody wants to advocate for any particular takeout option, sometimes the kindest course of action is to pick an arbitrary position and campaign for it passionately: "Actually I'm really in the mood for Three-Bears Pizza, can we please please get that, it's so good". Then, after the group orders what you asked for, if people aren't happy with the outcome afterwards, eargly accept 100% of the balme. This cuts short the frustrating decision making process, and spares everyone else from worrying about making a suggestion which others won't like. Most people are more averse to being percieved as selfish than they are averse to not eating their preffered cuisine for one evening, so you might be doing everyone a favor. In general, assertive kindness means whenever there is a standoff where nobody wants to be percieved as imposing their wants on anyone else, and that standoff leads to a collective decision making paralysis - you act to cut through the malaise by pushing hard for a specific course of action, supressing your selfish urges to avoid the risk of becomming a target for criticism/blame if things go poorly. ("Okay we're going go to the waterfall now! I'll grab towles, we'll take my car, get in let...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: End Single Family Zoning by Overturning Euclid V Ambler, published by Maxwell Tabarrok on July 27, 2024 on LessWrong. On 75 percent or more of the residential land in most major American cities it is illegal to build anything other than a detached single-family home. 95.8 percent of total residential land area in California is zoned as single-family-only, which is 30 percent of all land in the state. Restrictive zoning regulations such as these probably lower GDP per capita in the US by 8 36%. That's potentially tens of thousands of dollars per person. Map of land use in San Jose, California. Pink is single family only (94%) The legal authority behind all of these zoning rules derives from a 1926 Supreme Court decision in Village of Euclid v. Ambler Realty Co. Ambler realty held 68 acres of land in the town of Euclid, Ohio. The town, wanting to avoid influence, immigration, and industry from nearby Cleveland, passed a restrictive zoning ordinance which prevented Ambler realty from building anything but single family homes on much of their land, though they weren't attempting to build anything at the time of the case. Ambler realty and their lawyer ( a prominent Georgist!) argued that since this zoning ordinance severely restricted the possible uses for their property and its value, forcing the ordinance upon them without compensation was unconstitutional. The constitutionality claims in this case are about the 14th and 5th amendment. The 5th amendment to the United States Constitution states, among other things, that "private property [shall not] be taken for public use, without just compensation." The part of the 14th amendment relevant to this case just applies the 5th to state and local governments. There are two lines of argument in the case. First is whether the restrictions imposed by Euclid's zoning ordinance constitute "taking" private property at all. If they are taking, then the 5th amendment would apply, e.g when the govt takes land via eminent domain, they need to compensate property owners. However, even government interventions that do take don't always have to offer compensation. If the government, say, requires you to have an external staircase for fire egress, they don't have to compensate you because it protects "health, safety, and welfare" which is a " police powers" carveout from the takings clause of the 5th amendment. The other line of argument in the case is that zoning ordinances, while they do take from property owners, do not require compensation because they are part of this police power. Police Power Let's start with that second question: whether zoning laws count as protecting health and safety through the police power or are takings that require compensation. A common rhetorical technique is to reach for the most extreme case of zoning: a coal powered steel foundry wants to open up right next to the pre-school, for example. Conceding that this hypothetical is a legitimate use of the police power does not decide the case, however, because Euclid's zoning ordinance goes much further than separating noxious industry from schoolyards. The entire area of the village is divided by the ordinance into six classes of use districts, U-1 to U-6; three classes of height districts, H-1 to H-3, and four classes of area districts, A-1 to A-4. U-1 is restricted to single family dwellings, public parks, water towers and reservoirs, suburban and interurban electric railway passenger stations and rights of way, and farming, noncommercial greenhouse nurseries and truck gardening; U-2 is extended to include two-family dwellings; U-3 is further extended to include apartment houses, hotels, churches, schools, public libraries, museums, private clubs, community center buildings, hospitals, sanitariums, public playgrounds and recreation buildings, and a city ha...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How the AI safety technical landscape has changed in the last year, according to some practitioners, published by tlevin on July 26, 2024 on LessWrong. I asked the Constellation Slack channel how the technical AIS landscape has changed since I last spent substantial time in the Bay Area (September 2023), and I figured it would be useful to post this (with the permission of the contributors to either post with or without attribution). Curious if commenters agree or would propose additional changes! This conversation has been lightly edited to preserve anonymity. Me: One reason I wanted to spend a few weeks in Constellation was to sort of absorb-through-osmosis how the technical AI safety landscape has evolved since I last spent substantial time here in September 2023, but it seems more productive to just ask here "how has the technical AIS landscape evolved since September 2023?" and then have conversations armed with that knowledge. The flavor of this question is like, what are the technical directions and strategies people are most excited about, do we understand any major strategic considerations differently, etc -- interested both in your own updates and your perceptions of how the consensus has changed! Zach Stein-Perlman: Control is on the rise Anonymous 1: There are much better "model organisms" of various kinds of misalignment, e.g. the stuff Anthropic has published, some unpublished Redwood work, and many other things Neel Nanda: Sparse Autoencoders are now a really big deal in mech interp and where a lot of the top teams are focused, and I think are very promising, but have yet to conclusively prove themselves at beating baselines in a fair fight on a real world task Neel Nanda: Dangerous capability evals are now a major focus of labs, governments and other researchers, and there's clearer ways that technical work can directly feed into governance (I think this was happening somewhat pre September, but feels much more prominent now) Anonymous 2: Lots of people (particularly at labs/AISIs) are working on adversarial robustness against jailbreaks, in part because of RSP commitments/commercial motivations. I think there's more of this than there was in September. Anonymous 1: Anthropic and GDM are both making IMO very sincere and reasonable efforts to plan for how they'll make safety cases for powerful AI. Anonymous 1: In general, there's substantially more discussion of safety cases Anonymous 2: Since September, a bunch of many-author scalable oversight papers have been published, e.g. this, this, this. I haven't been following this work closely enough to have a sense of what update one should make from this, and I've heard rumors of unsuccessful scalable oversight experiments that never saw the light of day, which further muddies things Anonymous 3: My impression is that infosec flavoured things are a top ~3 priority area a few more people in Constellation than last year (maybe twice as many people as last year??). Building cyberevals and practically securing model weights at frontier labs seem to be the main project areas people are excited about (followed by various kinds of threat modelling and security standards). Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Index of rationalist groups in the Bay July 2024, published by Lucie Philippon on July 26, 2024 on LessWrong. The Bay Area rationalist community has an entry problem! Lots of listed groups are dead, the last centralized index disappeared, communication moved to private discord and slacks. This is bad, so we're making a new index, hopefully up to date and as complete as we can! Communication Discord: Bay Area Rationalists: https://discord.gg/EpG4xUVKtf Email Group: BayAreaLessWrong: https://groups.google.com/g/bayarealesswrong Local Meetup Groups Taco Tuesday: by Austin Chen, founder emeritus of Manifold. Check his Manifold questions page for the next date! North Oakland LessWrong Meetup: every Wednesday, hosted by @Czynski. Thursday Dinners in Berkeley: Advertised on the Discord server and Google group, alternating between a few restaurants on the northwest side of UC campus. Bay Area ACX Meetups: For the ACX everywhere meetups twice per year, and some other sporadic events. Housing To find spots in group houses, temporary or long term, you can use the Bay Area EA/Rationality Housing Board. The EA Houses spreadsheet also has some entries in the Bay. It probably works best to ask people in the Bay if they know of housing opportunities, as lots of housing is provided peer-to-peer. EA If you want to discover the EA community, the EA's Guide to Berkeley and The Bay Area is a good resource. Events sometimes get advertised on those websites: SF Bay Area EA calendar on Luma East Bay EA Hangout on Facebook AI Safety There are two AI safety coworking spaces in Berkeley. They sometime accept visitors, so you can try reaching out or applying via their website: FAR Labs Constellation Most AI Safety events don't get advertised publicly, so get in contact with people in the community to know what's happening. We probably missed some other meetups and communities which are public and still active, so feel free to list them in the comments! Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Universal Basic Income and Poverty, published by Eliezer Yudkowsky on July 26, 2024 on LessWrong. (Crossposted from Twitter) I'm skeptical that Universal Basic Income can get rid of grinding poverty, since somehow humanity's 100-fold productivity increase (since the days of agriculture) didn't eliminate poverty. Some of my friends reply, "What do you mean, poverty is still around? 'Poor' people today, in Western countries, have a lot to legitimately be miserable about, don't get me wrong; but they also have amounts of clothing and fabric that only rich merchants could afford a thousand years ago; they often own more than one pair of shoes; why, they even have cellphones, as not even an emperor of the olden days could have had at any price. They're relatively poor, sure, and they have a lot of things to be legitimately sad about. But in what sense is almost-anyone in a high-tech country 'poor' by the standards of a thousand years earlier? Maybe UBI works the same way; maybe some people are still comparing themselves to the Joneses, and consider themselves relatively poverty-stricken, and in fact have many things to be sad about; but their actual lives are much wealthier and better, such that poor people today would hardly recognize them. UBI is still worth doing, if that's the result; even if, afterwards, many people still self-identify as 'poor'." Or to sum up their answer: "What do you mean, humanity's 100-fold productivity increase, since the days of agriculture, has managed not to eliminate poverty? What people a thousand years ago used to call 'poverty' has essentially disappeared in the high-tech countries. 'Poor' people no longer starve in winter when their farm's food storage runs out. There's still something we call 'poverty' but that's just because 'poverty' is a moving target, not because there's some real and puzzlingly persistent form of misery that resisted all economic growth, and would also resist redistribution via UBI." And this is a sensible question; but let me try out a new answer to it. Consider the imaginary society of Anoxistan, in which every citizen who can't afford better lives in a government-provided 1,000 square-meter apartment; which the government can afford to provide as a fallback, because building skyscrapers is legal in Anoxistan. Anoxistan has free high-quality food (not fast food made of mostly seed oils) available to every citizen, if anyone ever runs out of money to pay for better. Cities offer free public transit including self-driving cars; Anoxistan has averted that part of the specter of modern poverty in our own world, which is somebody's car constantly breaking down (that they need to get to work and their children's school). As measured on our own scale, everyone in Anoxistan has enough healthy food, enough living space, heat in winter and cold in summer, huge closets full of clothing, and potable water from faucets at a price that most people don't bother tracking. Is it possible that most people in Anoxistan are poor? My (quite sensible and reasonable) friends, I think, on encountering this initial segment of this parable, mentally autocomplete it with the possibility that maybe there's some billionaires in Anoxistan whose frequently televised mansions make everyone else feel poor, because most people only have 1,000-meter houses. But actually this story is has a completely different twist! You see, I only spoke of food, clothing, housing, water, transit, heat and A/C. I didn't say whether everyone in Anoxistan had enough air to breathe. In Anoxistan, you see, the planetary atmosphere is mostly carbon dioxide, and breathable oxygen (O2) is a precious commodity. Almost everyone has to wear respirators at all times; only the 1% can afford to have a whole house full of breathable air, with some oxygen leaking away despite ...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI #74: GPT-4o Mini Me and Llama 3, published by Zvi on July 26, 2024 on LessWrong. We got two big model releases this week. GPT-4o Mini is covered here. Llama 3.1-405B (and 70B and 8B) is mostly covered in yesterday's post, this has some follow up. Table of Contents 1. Introduction. 2. Table of Contents. 3. Language Models Offer Mundane Utility. All your coding are belong to us. 4. Language Models Don't Offer Mundane Utility. Math is hard. Can be expensive. 5. GPT-4o Mini Me. You complete me at lower than usual cost. 6. Additional Llama-3.1 Notes. Pricing information, and more rhetoric. 7. Fun With Image Generation. If you're confused why artists are so upset. 8. Deepfaketown and Botpocalypse Soon. Not surprises. 9. They Took Our Jobs. Layoffs at Activision and across gaming. 10. In Other AI News. New benchmarks, new chip variants, and more. 11. The Art of the Jailbreak. Pliny remains undefeated. 12. Quiet Speculations. Where will the utility be coming from? 13. The Quest for Sane Regulations. Public opinion continues to be consistent. 14. Openly Evil AI. Some Senators have good questions. 15. The Week in Audio. Dwarkesh in reverse, and lots of other stuff. Odd Lots too. 16. Rhetorical Innovation. What are corporations exactly? 17. Aligning a Smarter Than Human Intelligence is Difficult. So are evals. 18. People Are Worried About AI Killing Everyone. Roon warns you to beware. 19. The Sacred Timeline. Hype? 20. Other People Are Not As Worried About AI Killing Everyone. Older Joe Rogan. 21. The Lighter Side. It's on. Language Models Offer Mundane Utility Coding is seriously much faster now, and this is the slowest it will ever be. Roon: pov: you are ten months from working for claude sonnet the new technical founder. Garry Tan: Underrated trend. It's happening. Sully: 50% of our code base was written entirely by LLMs expect this to be ~80% by next year With sonnet we're shipping so fast, it feels like we tripled headcount overnight Not using Claude 3.5 to code? Expect to be crushed by teams who do (us). Not only coding, either. Jimmy (QTing Tan): It can also do hardware related things quite well too, and legal, and logistics (planning) and compliance even. I've been able to put off hiring for months. When I run out of sonnet usage I patch in gpt-4o, it's obviously and notably worse which I why I rarely use it as a primary anymore. Claude 3.5 Sonnet becomes the first AI to crush the Lem Test to 'write an impossible poem.' Laugh all you want, this is actually great. Kache: dude hahahahahah i used so many tokens today on just formatting json logs near: the just stop oil people are gonna come and spray paint you now Compared to how much carbon a human coder would have used? Huge improvement. Language Models Don't Offer Mundane Utility IMO problems are still mostly too hard. The linked one, which GPT-4, GPT-4o and Claude 3.5 Sonnet failed on, seems unusually easy? Although a math Olympiad solver does, predictably given the contests we've seen. [EDIT: I didn't read this properly, but a reader points out this is the floor symbol, which means what I thought was an obvious proof doesn't actually answer the question, although it happens to get the right answer. Reader says the answers provided would actually also get 0/7, order has been restored]. Figure out what song Aella was talking about here. Found the obvious wrong answer. Grok offers to tell you 'more about this account.' I haven't seen the button yet, probably it is still experimental. Our price cheap. Llama 3.1-405B was a steal in terms of compute costs. Seconds: "AI is expensive" its not even half the cost of a middling marvel movie. Teortaxes: Pretty insane that the cost of producing llama-3-405B, this behemoth, is like 40% of *Ant-Man and the Wasp: Quantumania* movie at most If I were Zuck, I'd have open sourced a $...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A Solomonoff Inductor Walks Into a Bar: Schelling Points for Communication, published by johnswentworth on July 26, 2024 on LessWrong. A Solomonoff inductor walks into a bar in a foreign land. (Stop me if you've heard this one before.) The bartender, who is also a Solomonoff inductor, asks "What'll it be?". The customer looks around at what the other patrons are having, points to an unfamiliar drink, and says "One of those, please.". The bartender points to a drawing of the same drink on a menu, and says "One of those?". The customer replies "Yes, one of those.". The bartender then delivers a drink, and it matches what the first inductor expected. What's up with that? The puzzle, here, is that the two Solomonoff inductors seemingly agree on a categorization - i.e. which things count as the Unnamed Kind Of Drink, and which things don't, with at least enough agreement that the customer's drink-type matches the customer's expectations. And the two inductors reach that agreement without learning the category from huge amounts of labeled data - one inductor points at an instance, another inductor points at another instance, and then the first inductor gets the kind of drink it expected. Why (and when) are the two inductors able to coordinate on roughly the same categorization? Most existing work on Solomonoff inductors, Kolmogorov complexity, or minimum description length can't say much about this sort of thing. The problem is that the customer/bartender story is all about the internal structure of the minimum description - the (possibly implicit) "categories" which the two inductors use inside of their minimal descriptions in order to compress their raw data. The theory of minimum description length typically treats programs as black boxes, and doesn't attempt to talk about their internal structure. In this post, we'll show one potential way to solve the puzzle - one potential way for two minimum-description-length-based minds to coordinate on a categorization. Main Tool: Natural Latents for Minimum Description Length Fundamental Theorem Here's the main foundational theorem we'll use. (Just the statement for now, more later.) We have a set of n data points (binary strings) {xi}, and a Turing machine TM. Suppose we find some programs/strings Λ,{ϕi},Λ',{ϕ'i} such that: Mediation: (Λ,ϕ1,…,ϕn) is an approximately-shortest string such that (TM(Λ,ϕi) = xi for all i) Redundancy: For all i, (Λ',ϕ'i) is an approximately-shortest string such that TM(Λ',ϕ'i) = xi.[1] Then: the K-complexity of Λ' given Λ,K(Λ'|Λ), is approximately zero - in other words, Λ' is approximately determined by Λ, in a K-complexity sense. (As a preview: later we'll assume that both Λ and Λ' satisfy both conditions, so both K(Λ'|Λ) and K(Λ|Λ') are approximately zero. In that case, Λ and Λ' are "approximately isomorphic" in the sense that either can be computed from the other by a short program. We'll eventually tackle the customer/bartender puzzle from the start of this post by suggesting that Λ and Λ' each encode a summary of things in one category according to one inductor, so the theorem then says that their category summaries are "approximately isomorphic".) The Intuition What does this theorem mean intuitively? Let's start with the first condition: (Λ,ϕ1,…,ϕn) is an approximately-shortest string such that (TM(Λ,ϕi) = xi for all i). Notice that there's a somewhat-trivial way to satisfy that condition: take Λ to be a minimal description of the whole dataset {xi}, take ϕi=i, and then add a little bit of code to Λ to pick out the datapoint at index ϕi[2]. So TM(Λ,ϕi) computes all of {xi} from Λ, then picks out index i. Now, that might not be the only approximately-minimal description (though it does imply that whatever approximately-minimal Λ,ϕ we do use is approximately a minimal description for all of x). ...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: "AI achieves silver-medal standard solving International Mathematical Olympiad problems", published by gjm on July 25, 2024 on LessWrong. Google DeepMind reports on a system for solving mathematical problems that allegedly is able to give complete solutions to four of the six problems on the 2024 IMO, putting it near the top of the silver-medal category. Well, actually, two systems for solving mathematical problems: AlphaProof, which is more general-purpose, and AlphaGeometry, which is specifically for geometry problems. (This is AlphaGeometry 2; they reported earlier this year on a previous version of AlphaGeometry.) AlphaProof works in the "obvious" way: an LLM generates candidate next steps which are checked using a formal proof-checking system, in this case Lean. One not-so-obvious thing, though: "The training loop was also applied during the contest, reinforcing proofs of self-generated variations of the contest problems until a full solution could be found." (That last bit is reminiscent of something from the world of computer go: a couple of years ago someone trained a custom version of KataGo specifically to solve the infamous Igo Hatsuyoron problem 120, starting with ordinary KataGo and feeding it training data containing positions reachable from the problem's starting position. They claim to have laid that problem to rest at last.) AlphaGeometry is similar but uses something specialized for (I think) Euclidean planar geometry problems in place of Lean. The previous version of AlphaGeometry allegedly already performed at gold-medal IMO standard; they don't say anything about whether that version was already able to solve the 2024 IMO problem that was solved using AlphaGeometry 2. AlphaProof was able to solve questions 1, 2, and 6 on this year's IMO (two algebra, one number theory). It produces Lean-formalized proofs. AlphaGeometry 2 was able to solve question 4 (plane geometry). It produces proofs in its own notation. The solutions found by the Alpha... systems are at https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/imo-2024-solutions/index.html. (There are links in the top-of-page navbar to solutions to the individual problems.) (If you're curious about the IMO questions or want to try them yourself before looking at the machine-generated proofs, you can find them -- and those for previous years -- at https://www.imo-official.org/problems.aspx.) One caveat (note: an earlier version of what I wrote failed to notice this and quite wrongly explicitly claimed something different): "First, the problems were manually translated into formal mathematical language for our systems to understand." It feels to me like it shouldn't be so hard to teach an LLM to convert IMO problems into Lean or whatever, but apparently they aren't doing that yet. Another caveat: "Our systems solved one problem within minutes and took up to three days to solve the others." Later on they say that AlphaGeometry 2 solved the geometry question within 19 seconds, so I guess that was also the one that was done "within minutes". Three days is a lot longer than human IMO contestants get given, but this feels to me like the sort of thing that will predictably improve pretty rapidly. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Llama Llama-3-405B?, published by Zvi on July 25, 2024 on LessWrong. It's here. The horse has left the barn. Llama-3.1-405B, and also Llama-3.1-70B and Llama-3.1-8B, have been released, and are now open weights. Early indications are that these are very good models. They were likely the best open weight models of their respective sizes at time of release. Zuckerberg claims that open weights models are now competitive with closed models. Yann LeCun says 'performance is on par with the best closed models.' This is closer to true than in the past, and as corporate hype I will essentially allow it, but it looks like this is not yet fully true. Llama-3.1-405B not as good as GPT-4o or Claude Sonnet. Certainly Llama-3.1-70B is not as good as the similarly sized Claude Sonnet. If you are going to straight up use an API or chat interface, there seems to be little reason to use Llama. That is a preliminary result. It is still early, and there has been relatively little feedback. But what feedback I have seen is consistent on this. Prediction markets are modestly more optimistic. This market still has it 29% to be the #1 model on Arena, which seems unlikely given Meta's own results. Another market has it 74% to beat GPT-4-Turbo-2024-04-09, which currently is in 5th position. That is a big chance for it to land in a narrow window between 1257 and 1287. This market affirms that directly on tiny volume. Such open models like Llama-3.1-405B are of course still useful even if a chatbot user would have better options. There are cost advantages, privacy advantages and freedom of action advantages to not going through OpenAI or Anthropic or Google. In particular, if you want to distill or fine-tune a new model, and especially if you want to fully own the results, Llama-3-405B is here to help you, and Llama-3-70B and 8B are here as potential jumping off points. I expect this to be the main practical effect this time around. If you want to do other things that you can't do with the closed options? Well, technically you can't do most of them under Meta's conditions either, but there is no reason to expect that will stop people, especially those overseas including in China. For some of these uses that's a good thing. Others, not as good. Zuckerberg also used the moment to offer a standard issue open source manifesto, in which he abandons any sense of balance and goes all-in, which he affirmed in a softball interview with Rowan Cheung. On the safety front, while I do not think they did their safety testing in a way that would have caught issues if there had been issues, my assumption is there was nothing to catch. The capabilities are not that dangerous at this time. Thus I do not predict anything especially bad will happen here. I expect the direct impact of Llama-3.1-405B to be positive, with the downsides remaining mundane and relatively minor. The only exception would be the extent to which this enables the development of future models. I worry that this differentially accelerates and enables our rivals and enemies and hurts our national security, and indeed that this will be its largest impact. And I worry more that this kind of action and rhetoric will lead us down the path where if things get dangerous in the future, it will become increasingly hard not to get ourselves into deep trouble, both in terms of models being irrevocably opened up when they shouldn't be and increasing pressure on everyone else to proceed even when things are not safe, up to and including loss of control and other existential risks. If Zuckerberg had affirmed a reasonable policy going forward but thought the line could be drawn farther down the line, I would have said this was all net good. Instead, I am dismayed. I do get into the arguments about open weights at the end of this post, because it felt obligato...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The last era of human mistakes, published by owencb on July 25, 2024 on LessWrong. Suppose we had to take moves in a high-stakes chess game, with thousands of lives at stake. We wouldn't just find a good chess player and ask them to play carefully. We would consult a computer. It would be deeply irresponsible to do otherwise. Computers are better than humans at chess, and more reliable. We'd probably still keep some good chess players in the loop, to try to catch possible computer error. (Similarly we still have pilots for planes, even though the autopilot is often safer.) But by consulting the computer we'd remove the opportunity for humans to make a certain type of high stakes mistake. A lot of the high stakes decisions people make today don't look like chess, or flying a plane. They happen in domains where computers are much worse than humans. But that's a contingent fact about our technology level. If we had sufficiently good AI systems, they could catch and prevent significant human errors in whichever domains we wanted them to. In such a world, I think that they would come to be employed for just about all suitable and important decisions. If some actors didn't take advice from AI systems, I would expect them to lose power over time to actors who did. And if public institutions were making consequential decisions, I expect that it would (eventually) be seen as deeply irresponsible not to consult computers. In this world, humans could still be responsible for taking decisions (with advice). And humans might keep closer to sole responsibility for some decisions. Perhaps deciding what, ultimately, is valued. And many less consequential decisions, but still potentially large at the scale of an individual's life (such as who to marry, where to live, or whether to have children), might be deliberately kept under human control[1]. Such a world might still collapse. It might face external challenges which were just too difficult. But it would not fail because of anything we would parse as foolish errors. In many ways I'm not so interested in that era. It feels out of reach. Not that we won't get there, but that there's no prospect for us to help the people of that era to navigate it better. My attention is drawn, instead, to the period before it. This is a time when AI will (I expect) be advancing rapidly. Important decisions may be made in a hurry. And while automation-of-advice will be on the up, it seems like wildly unprecedented situations will be among the hardest things to automate good advice for. We might think of it as the last era of consequential human mistakes[2]. Can we do anything to help people navigate those? I honestly don't know. It feels very difficult (given the difficulty at our remove in even identifying the challenges properly). But it doesn't feel obviously impossible. What will this era look like? Perhaps AI progress is blisteringly fast and we move from something like the world of today straight to a world where human mistakes don't matter. But I doubt it. On my mainline picture of things, this era - the final one in which human incompetence (and hence human competence) really matters - might look something like this: Cognitive labour approaching the level of human thinking in many domains is widespread, and cheap People are starting to build elaborate ecosystems leveraging its cheapness … … since if one of the basic inputs to the economy is changed, the optimal arrangement of things is probably quite different (cf. the ecosystem of things built on the internet); … but that process hasn't reached maturity. There is widespread access to standard advice, which helps to avoid some foolish errors, though this is only applicable to "standard" situations, and it isn't universal to seek that advice In some domains, AI performance is significantly bet...