The Nonlinear Library: LessWrong

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: ... Wait, our models of semantics should inform fluid mechanics?!?, published by johnswentworth on August 26, 2024 on LessWrong. This post is styled after conversations we've had in the course of our research, put together in a way that hopefully highlights a bunch of relatively recent and (ironically) hard-to-articulate ideas around natural abstractions. John: So we've been working a bit on semantics, and also separately on fluid mechanics. Our main goal for both of them is to figure out more of the higher-level natural abstract data structures. But I'm concerned that the two threads haven't been informing each other as much as they should. David: Okay…what do you mean by "as much as they should"? I mean, there's the foundational natural latent framework, and that's been useful for our thinking on both semantics and fluid mechanics. But beyond that, concretely, in what ways do (should?) semantics and fluid mechanics inform each other? John: We should see the same types of higher-level data structures across both - e.g. the "geometry + trajectory" natural latents we used in the semantics post should, insofar as the post correctly captures the relevant concepts, generalize to recognizable "objects" in a fluid flow, like eddies (modulo adjustments for nonrigid objects). David: Sure, I did think it was intuitive to think along those lines as a model for eddies in fluid flow. But in general, why expect to see the same types of data structures for semantics and fluid flow? Why not expect various phenomena in fluid flow to be more suited to representation in some data structures which aren't the exact same type as those used for the referrents of human words? John: Specifically, I claim that the types of high-level data structures which are natural for fluid flow should be a subset of the types needed for semantics. If there's a type of high-level data structure which is natural for fluid flow, but doesn't match any of the semantic types (noun, verb, adjective, short phrases constructed from those, etc), then that pretty directly disproves at least one version of the natural abstraction hypothesis (and it's a version which I currently think is probably true). David: Woah, hold up, that sounds like a very different form of the natural abstraction hypothesis than our audience has heard before! It almost sounds like you're saying that there are no "non-linguistic concepts". But I know you actually think that much/most of human cognition routes through "non-linguistic concepts". John: Ok, there's a couple different subtleties here. First: there's the distinction between a word or phrase or sentence vs the concept(s) to which it points. Like, the word "dog" evokes this whole concept in your head, this whole "data structure" so to speak, and that data structure is not itself linguistic. It involves visual concepts, probably some unnamed concepts, things which your "inner simulator" can use, etc. Usually when I say that "most human concepts/cognition are not linguistic", that's the main thing I'm pointing to. Second: there's concepts for which we don't yet have names, but could assign names to. One easy way to find examples is to look for words in other languages which don't have any equivalent in our language. The key point about those concepts is that they're still the same "types of concepts" which we normally assign words to, i.e. they're still nouns or adjectives or verbs or…, we just don't happen to have given them names. Now with both of those subtleties highlighted, I'll once again try to state the claim: roughly speaking, all of the concepts used internally by humans fall into one of a few different "types", and we have standard ways of describing each of those types of concept with words (again, think nouns, verbs, etc, but also think of the referents of short phrases y...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: DIY LessWrong Jewelry, published by Fluffnutt on August 26, 2024 on LessWrong. [There are no affiliate links in this post and I haven't been paid by anyone; I just happened to have bought the charms from Shein. You can probably get similar things on Temu, Aliexpress or Amazon] I made a cute LessWrong themed necklace/earring set for cheap, here's how. For the necklace, buy a pair of these earrings from Shein: US: https://us.shein.com/1pair-Fashionable-Rhinestone-Star-Decor-Drop-Earrings-For-Women-For-Daily-Decoration-p-15276285.html CAN: https://ca.shein.com/1pair-Fashionable-Rhinestone-Star-Decor-Drop-Earrings-For-Women-For-Daily-Decoration-p-15276285.html. Then remove the dangly charm and use a wire to attach it to a necklace chain. (I bought a cheap necklace from Shein to use as the chain but it tarnished within days. Amazon seems to sell decent chains/necklaces for ~20$, but I haven't bought any so can't personally recommend them (I used an old chain from my grandmother instead). If you're buying a necklace make sure any pre-existing charms can be removed. You can either get two pairs of the above earrings to make a set, or get these ones if you prefer studs: US: https://us.shein.com/1pair-S999-Sterling-Silver-Cubic-Zirconia-Eight-pointed-Star-Design-Symmetric-Earrings-For-Women-p-23036383.html CAN: https://ca.shein.com/1pair-S999-Sterling-Silver-Cubic-Zirconia-Eight-pointed-Star-Design-Symmetric-Earrings-For-Women-p-23036383.html Here they all are together: Note the stud is quite small. Here is what it looks like on my ear, in comparison with the hoop earrings: Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Darwinian Traps and Existential Risks, published by KristianRonn on August 26, 2024 on LessWrong. This part 1 in a 3-part sequence summarizes my book, The Darwinian Trap. The book aims to popularize the concept of multipolar traps and establish them as a broader cause area. If you find this series intriguing and want to spread the word and learn more: 1. Share this post with others on X or other social media platforms. 2. Pre-order the book here. 3. Sign up for my mailing list here before September 24 for a 20% chance to win a free hardcover copy of the book (it takes 5 seconds). 4. Contact me at kristian@kristianronn.com if you have any input or ideas. Global coordination stands as arguably the most critical challenge facing humanity today, functioning both as a necessary component for solving existential risks and as a significant barrier to effective mitigation. From nuclear proliferation to artificial intelligence development and climate change, our inability to collaborate effectively on a global scale not only exacerbates these threats but also perpetuates the emergence of new systemic vulnerabilities if left unaddressed. In this sequence, I will argue that the root of this coordination problem lies in the very mechanisms that shaped our species: natural selection. This evolutionary process, operating as a trial-and-error optimization algorithm, prioritizes immediate survival and reproduction over long-term, global outcomes. As a result, our innate tendencies often favor short-term gains and localized benefits, even when they conflict with the greater good of our species and planet. The inherent limitations of natural selection in predicting future optimal states have left us ill-equipped to handle global-scale challenges. In a world of finite resources, competition rather than cooperation has often been the more adaptive trait, leading to the emergence of self-interested behaviors that arguably dominate modern societies. This evolutionary legacy manifests in the form of nationalistic tendencies, economic rivalries, dangerous arms races and a general reluctance to sacrifice immediate benefits for long-term collective gains. This three-part series summarizes my book: The Darwinian Trap: The Hidden Evolutionary Forces That Explain Our World (and Threaten Our Future). Part 1 (the part you are reading): This part explores how evolutionary selection pressures contribute to existential risks, particularly through global resource, power, and intelligence arms races. Part 2: This part delves into the evolution of cooperation and discusses how survivorship bias might mislead us into thinking cooperation is easier to achieve than it actually is. It suggests that life may be inherently fragile, potentially containing the seeds of its own destruction, which could provide an explanation for the Fermi Paradox. Part 3: The final part examines strategies to overcome global coordination challenges, both through centralized and decentralized approaches. It introduces the concept of "reputational markets" (a variant of prediction markets) as a tool for supply chain governance, aimed at reshaping the evolutionary trajectory toward cooperation and long-term survival and flourishing. The Universal Algorithm of Natural Selection Evolution isn't just a biological process; it's a universal optimization algorithm that applies to any type of entity - be it chemicals, groups, countries, companies, or even memes - as long as the system in question fulfills the following three key features: 1. Variation: In any population, members differ in characteristics. For example, mice may vary in size, speed, and color. In a digital landscape, social media platforms might vary in features, such as content delivery methods or user engagement strategies. 2. Selection: Different characteristics lead to ...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Secular interpretations of core perennialist claims, published by zhukeepa on August 26, 2024 on LessWrong. After the release of Ben Pace's extended interview with me about my views on religion, I felt inspired to publish more of my thinking about religion in a format that's more detailed, compact, and organized. This post is the second publication in my series of intended posts about religion. Thanks to Ben Pace, Chris Lakin, Richard Ngo, Damon Pourtahmaseb-Sasi, Marcello Herreshoff, Renshin Lauren Lee, Mark Miller, Roger Thisdell, and Imam Ammar Amonette for their feedback on this post, and thanks to Kaj Sotala, Tomáš Gavenčiak, Paul Colognese, and David Spivak for reviewing earlier versions of this post. Thanks especially to Renshin Lauren Lee, Roger Thisdell, and Imam Ammar Amonette for their input on my claims about perennialism, and Mark Miller for vetting my claims about predictive processing. In my previous post, I introduced the idea that there are broad convergences among the mystical traditions of the major world religions, corresponding to a shared underlying essence, called the perennial philosophy, that gave rise to each of these mystical traditions. I think there's nothing fundamentally mysterious, incomprehensible, or supernatural about the claims in the perennial philosophy. My intention in this post is to articulate my interpretations of some central claims of the perennial philosophy, and present them as legible hypotheses about possible ways the world could be. It is not my intention in this post to justify why I believe these claims can be found in the mystical traditions of the major world religions, or why I believe the mystical traditions are centered around claims like these. I also don't expect these hypotheses to seem plausible in and of themselves - these hypotheses only started seeming plausible to me as I went deeper into my own journey of inner work, and started noticing general patterns about my psychology consistent with these claims. I will warn in advance that in many cases, the strongest versions of these claims might not be compatible with the standard scientific worldview, and may require nonstandard metaphysical assumptions to fully make sense of.[1] (No bearded interventionist sky fathers, though!) I intend to explore the metaphysical foundations of the perennialist worldview in a future post; for now, I will simply note where I think nonstandard metaphysical assumptions may be necessary. The Goodness of Reality Sometimes, we feel that reality is bad for being the way it is, and feel a sense of charge around this. To illustrate the phenomenology of this sense of charge, consider the connotation that's present in the typical usages of "blame" that aren't present in the typical usages of "hold responsible"; ditto "punish" vs "disincentivize"; ditto "bad" vs "dispreferred". I don't think there's a word in the English language that unambiguously captures this sense of charge, but I think it's captured pretty well by the technical Buddhist term tanha, which is often translated as "thirst" or "craving". I interpret this sense of charge present in common usages of the words "blame", "punish", and "bad" as corresponding to the phenomenology of "thirst" or "craving"[2] for reality to be different from how it actually is. When our active blind spots get triggered, we scapegoat reality. We point a finger at reality and say "this is bad for being the way it is" with feelings of tanha, when really there's some vulnerability getting triggered that we're trying to avoid acknowledging. This naturally invites the following question: of the times we point at reality and say "this is bad for being the way it is" with feelings of tanha, what portion of these stem from active blind spots, and what portion of these responses should we fully endorse ...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: you should probably eat oatmeal sometimes, published by bhauth on August 25, 2024 on LessWrong. Inspired by all the blog posts I've seen promoting unusual diets, nootropic drugs, unusual sleep cycles, and so on, I've decided to make my own post suggesting some radical lifestyle choice. My suggestion here is, as the title says: you should probably eat oatmeal sometimes. Yes, I know, oats are <2% of global grain production, so this is a pretty crazy-sounding idea, but bear with me. Eating oatmeal sometimes will literally change your life.* *slightly oatmeal vs rice White rice is one of the most popular foods in the world. Compared to white rice: oats have ~2x the protein and much more fiber their amino acid composition is somewhat more balanced their arsenic levels are generally lower oatmeal is slightly more expensive, but still cheap oatmeal is generally faster to cook about whole-wheat flour Whole-grain wheat flour goes rancid much faster than white flour. Wikipedia, quoting a 2004 book, says: The germ is rich in polyunsaturated fats (which have a tendency to oxidize and become rancid on storage) and so germ removal improves the storage qualities of flour. (The "germ" is the part of the seed that actually grows into a new plant. As you'd expect, it's the part with the most protein and vitamins.) Hmm. If that's the problem, why do (unground) wheat berries last for 10+ years, longer than white flour? Does the bran protect stuff from oxygen? I don't think so, it's not metal foil, it has some gas permeability. Maybe there's a correction of Wikipedia from some reliable source, like Wikipedia. Here's a post from a food testing company that says: The lipolytic enzyme lipase reacts with triglycerides to form free fatty acids in a degradation process known as hydrolytic rancidity; lipase enzymes cleave fatty acids from triglycerides. ... Oats also contain a powerful lipoxygenase that adds oxygen to the double bonds of unsaturated fat to form peroxides, as discussed above. The other enzyme present, peroxidase, reduces peroxides producing mono-, di-, and tri-hydroxy acids, which are extremely bitter. These compounds cause the bitter flavor of rancid wheat germ. ... In most biological systems, peroxidase requires much more heat to destroy than lipases, lipoxygenases or any of the other enzymes that may be present. Ah yes, enzymes. When wheat berries are ground to flour, the enzymes start doing things and the flour goes rancid, but if you remove stuff to make white flour, you remove most of the enzymes. As for oatmeal, rolled oats have such enzymes activated, so they must be treated with steam. Oats have the germ in the bottom, and can be cut up more easily than wheat without triggering enzymes too much, but steamed rolled oats still last longer than unsteamed steel-cut oats. It's similar to how cutting onions releases alliinase but if you microwave the onion first it deactivates most of that. Also, I think the rancidity-relevant enzymes in oatmeal might have lower thermal stability than the ones in wheat. But it's still possible to steam-treat whole wheat so that whole wheat flour lasts longer. I think people have relatively recently found that you want to use superheated steam for that. Maybe whole wheat flour treated with superheated steam will be a thing in the future. oatmeal is versatile Oatmeal works with a variety of different flavors, and other ingredients can often simply be added to it before or after cooking. There are several options for the liquid used, including: water milk tea coconut milk There are also many reasonable options for additional flavors, including: fruit paste fruit syrup frozen fruit pulp chocolate brown sugar maple syrup Flavor combinations I've used for oatmeal include: tea + blackcurrant concentrate + brown sugar passionfruit pulp + brown sugar ...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Referendum Mechanics in a Marketplace of Ideas, published by Martin Sustrik on August 25, 2024 on LessWrong. This is a cross post from https://250bpm.substack.com/p/referendum-mechanics-in-a-marketplace. In a referendum, people vote either for or against the proposal and that's it. Right? Wrong! One can definitely make referendums more complex by mixing in unrelated stuff, unnecessary restrictions, and complexities. But that's not the case with Swiss referendums. Quite the contrary, they seem to be designed for simplicity: First: There is no quorum. Even if the turnout is low, the referendum is valid. If three people cast their votes in a national ballot and two of them vote yes, the proposal is accepted. It is then written into the constitution, so even the parliament cannot overrule it. No quorum means no strategic voting or, more precisely, no strategic withholding of votes to sabotage the referendum by making it invalid due to not meeting the quorum, as often happens elsewhere. Second: The referendum process takes many years, often five or more, so it can't be used for short-term, tribal politics, like calling for premature elections. By the time the referendum reaches voters, five or six years after its initiation, other people are already in power, and the original reason for the referendum has long since become irrelevant. Hot-button issues of yesteryear are already blissfully forgotten. Third: We know that defaults matter. If the referendum question is worded differently - when yes and no votes switch their meanings - it could lead to a different voting outcome. But in Switzerland, referendum questions are always worded the same way: It's a proposal to change the constitution. "Yes" always means a vote for change, while "No" always means to keep the status quo. And if the history of past referendums teaches us anything, it is that the default option is always "No." Only a few referendums in their 150-year history have been successful. If people don't understand or don't care about the proposal, they vote to keep the status quo by default. Given all the above, why am I suggesting that the referendums can get complex? Read on and discover the fascinating world of referendum politics! Imagine that the initiators of the referendum demand lowering a tax by 4%. (And yes, in Switzerland, any changes to taxes must be approved in a referendum.) The government doesn't want to change the tax. If they did, they would have already put it on the ballot. So, the government commissions a survey, which reveals that 60% of voters are going to vote for the proposal. What can they do to fight it? Well, of course! They can launch a counterproposal. They can propose that the tax should be lowered by only 2%. Now, voters can choose to either keep the tax at the current level, lower it by 2%, or lower it by 4%. If they can vote for only one of the two latter options, the counterproposal does more than just deliver more choice to the voter. It also splits the voter base. If originally 40% of voters were about to vote for keeping the current tax rate and 60% for lowering it, the anti-tax people would be clear winners. However, introduction of the counterproposal dramatically changes the landscape. Now 40% vote for keeping the tax at its current level, just like before, but 25% now vote for lowering it by 2% and 35% for lowering it by 4%. Pro-taxers suddenly and miraculously win the referendum! Eventually, it became obvious that the system was not working as intended and could be easily manipulated. So since 1987, voters can vote for both the original proposal and the counterproposal. (So called "double yes".) If they do so, they can also indicate which option they prefer in case both proposals pass. The splitting of the voter base, as described above, does not happen. Okay, fair enoug...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Please stop using mediocre AI art in your posts, published by Raemon on August 25, 2024 on LessWrong. Epistemic Status: Old Man Blogs at Cloud Lately there's been a wave of people on LessWrong (and maybe the whole internet) starting off their essays with some Dall-E3 art. I don't object (and think it can be cool if done nicely) to get some good ML art to add some visual interest to your posts. But, the default thing I'm seeing is mediocre and making the web feel tacky. [1] I don't want to just yell at people and make them feel self-conscious about their fun. I have an art background, which means I both have a lot of experience making/evaluating art and also probably have random-ass snooty connoisseurship syndrome. Also I was involved with the fleshing out of the current LessWrong Watercolor Aesthetic, and random clip-art looking images undercut it. (But, to be fair nobody actually voted for or opted into the LessWrong Watercolor Aesthetic, and maybe that's just my problem). I think not all posts need art. But, if you do want art for your post, here's some hopefully slightly constructive advice/requests. 1. Generally, make landscape (widescreen) art. Most image models output square images by default. This actually fits fairly awkwardly into blogposts - it either takes up a huge amount of vertical space, or you shrink it to fit and then it has weird padding. (The motivating instance for this blogpost was this post, which IMO would be greatly improved if the image was designed to take up more horizontal space). Sometimes a post makes good use of a very tall art, to set some kind of mood, but it works better for more contemplative posts. (See On Green for an example). 2. Good AI art still takes a fair number of generations and effort. I'm not sure how many generations people typically do that outputs that mediocre art, but I do want to note, for reference, when I'm making a quick piece of AI art for something (like a facebook event) it still usually involves at least like 5 generations (in Midjourney, where each generation includes 4 options), and often more like 20. And when I'm trying to make actually good art for something I want people to really appreciate (such as the Carving of Reality books), it might be hundreds of generations. 3. Think about the mood you want to convey, not just the posts' intellectual content. Good art sets a tone and helps shift someone into a particular headspace. This doesn't just include the content of the art but the style and color palette. This is particularly important if you're opening the post with art, where it's setting the very first impression (and is also more likely to show up in the Recent Discussion feed, where it'll look more random). That's probably not very helpful advice on it's own. Making good art is, like, a whole-ass skill. But, on the offchance you weren't thinking about that at all, maybe giving it at least some consideration will probably help. Okay, that's it I guess thank you for indulging my rant. 1. ^ Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Perplexity wins my AI race, published by Elizabeth on August 24, 2024 on LessWrong. Perplexity is the first generalized AI chatbot I've found useful enough to integrate into any part of my daily workflow, much less across multiple domains. It speeds me up enough that I'm planning an increase in my freelancing rate. Perplexity has three key advantages: 1. It provides citations, cleanly, in context 2. It has the persona of a sharp human instead of an intolerable customer service agent. 3. It is useful (and sufferable) across a wide variety of domains. The citations are a bigger deal than they sound at first. I could of course google anything ChatGPT tells me and check the results, but that's costly. I have to spin up a new mental thread (and browser tab), sort through Google's increasingly shitty results, remember what claim I'm trying to check… the process eats up a lot of working memory at a time when it is scarce. If I wanted to trawl a sea of content farms I would have gone to Google in the first place. Perplexity's high quality inline citations solve that . Want to check if a claim is justified? Click the number next to it and look for the relevant quote. Not a perfect substitute for a 30 hour literature search, but a lot of power per minute. I don't have to spin up a new mental thread, so the whole thing feels much lower friction. Also thus far I've never caught it in a hallucination on a question of fact, although if real answers are scarce it will sometimes do the thing where it draws increasingly tenuous connections instead of just telling me no. I also found perplexity easier to talk to while trying to learn some math and math history. It felt about as useful as my human tutor- better at some things, worse at others. Test Questions The real test was which AI do I find myself using, and the answer to that is definitively Perplexity. But just for fun or in case I was wrong, I ran a few questions Perplexity through all major engines to see what they said. In all cases I used the top free model available. I have Perplexity Pro thanks to a promo, but I haven't ended up finding Pro to be much of an improvement. I do expect to use the pro-only PDF-analysis tool in the near future. Comments from me are in italics and [brackets] How do fractals relate to chaos theory? I have a project investigating chaos theory as a case study in the development of scientific/mathematical paradigms. This doesn't require I become a mathematician, but I do need to understand how discoveries relate to each other and generally not be an idiot on the topic. Perplexity is my always available, infinitely patient tutor. This should have been a challenging role for an AI to fill, because I have a friend who loves explaining math more than almost anything, and did his PhD in a related area. I even feel comfortable looking stupid in front of him. And yet Perplexity still had a value add, because it was never offline and I didn't feel a need to respect its time the way I would a friend's. Perplexity (link to result including citations) Fractals and chaos theory are closely intertwined concepts in mathematics and physics, with several key connections: Self-Similarity and Complex Patterns Fractals are geometric objects that exhibit self-similarity across different scales, meaning smaller parts resemble the whole. This property is often found in chaotic systems, where complex patterns emerge from simple rules. Visualization of Chaotic Processes Fractals frequently arise as visual representations of chaotic systems: The Feigenbaum diagram, which illustrates population dynamics in chaos theory, displays fractal-like structures. The Mandelbrot set, a famous fractal, is closely related to chaotic processes and can be used to visualize complex number behavior in iterative functions. Characteristics of Co...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What's important in "AI for epistemics"?, published by Lukas Finnveden on August 24, 2024 on LessWrong. Summary This post gives my personal take on "AI for epistemics" and how important it might be to work on. Some background context: AI capabilities are advancing rapidly and I think it's important to think ahead and prepare for the possible development of AI that could automate almost all economically relevant tasks that humans can do.[1] That kind of AI would have a huge impact on key epistemic processes in our society. (I.e.: It would have a huge impact on how new facts get found, how new research gets done, how new forecasts get made, and how all kinds of information spread through society.) I think it's very important for our society to have excellent epistemic processes. (I.e.: For important decisions in our society to be made by people or AI systems who have informed and unbiased beliefs that take into account as much of the available evidence as is practical.) Accordingly, I'm interested in affecting the development and usage of AI technology in ways that lead towards better epistemic processes. So: How can we affect AI to contribute to better epistemic processes? When looking at concrete projects, here, I find it helpful to distinguish between two different categories of work: 1. Working to increase AIs' epistemic capabilities, and in particular, differentially advancing them compared to other AI capabilities. Here, I also include technical work to measure AIs' epistemic capabilities.[2] 2. Efforts to enable the diffusion and appropriate trust of AI-discovered information. This is focused on social dynamics that could cause AI-produced information to be insufficiently or excessively trusted. It's also focused on AIs' role in communicating information (as opposed to just producing it). Examples of interventions, here, include "create an independent organization that evaluates popular AIs' truthfulness", or "work for countries to adopt good (and avoid bad) legislation of AI communication". I'd be very excited about thoughtful and competent efforts in this second category. However, I talk significantly more about efforts in the first category, in this post. This is just an artifact of how this post came to be, historically - it's not because I think work on the second category of projects is less important.[3] For the first category of projects: Technical projects to differentially advance epistemic capabilities seem somewhat more "shovel-ready". Here, I'm especially excited about projects that differentially boost AI epistemic capabilities in a manner that's some combination of durable and/or especially good at demonstrating those capabilities to key actors. Durable means that projects should (i) take the bitter lesson into account by working on problems that won't be solved-by-default when more compute is available, and (ii) work on problems that industry isn't already incentivized to put huge efforts into (such as "making AIs into generally better agents"). (More on these criteria here.) Two example projects that I think fulfill these criteria (I discuss a lot more projects here): Experiments on what sort of arguments and decompositions make it easier for humans to reach the truth in hard-to-verify areas. (Strongly related to scalable oversight.) Using AI to generate large quantities of forecasting data, such as by automatically generating and resolving questions. Separately, I think there's value in demonstrating the potential of AI epistemic advice to key actors - especially frontier AI companies and governments. When transformative AI (TAI)[4] is first developed, it seems likely that these actors will (i) have a big advantage in their ability to accelerate AI-for-epistemics via their access to frontier models and algorithms, and (ii) that I especially car...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: "Can AI Scaling Continue Through 2030?", Epoch AI (yes), published by gwern on August 24, 2024 on LessWrong. We investigate the scalability of AI training runs. We identify electric power, chip manufacturing, data and latency as constraints. We conclude that 2e29 FLOP training runs will likely be feasible by 2030. Introduction In recent years, the capabilities of AI models have significantly improved. Our research suggests that this growth in computational resources accounts for a significant portion of AI performance improvements. 1 The consistent and predictable improvements from scaling have led AI labs to aggressively expand the scale of training, with training compute expanding at a rate of approximately 4x per year. To put this 4x annual growth in AI training compute into perspective, it outpaces even some of the fastest technological expansions in recent history. It surpasses the peak growth rates of mobile phone adoption (2x/year, 1980-1987), solar energy capacity installation (1.5x/year, 2001-2010), and h uman genome sequencing (3.3x/year, 2008-2015). Here, we examine whether it is technically feasible for the current rapid pace of AI training scaling - approximately 4x per year - to continue through 2030. We investigate four key factors that might constrain scaling: power availability, chip manufacturing capacity, data scarcity, and the "latency wall", a fundamental speed limit imposed by unavoidable delays in AI training computations. Our analysis incorporates the expansion of production capabilities, investment, and technological advancements. This includes, among other factors, examining planned growth in advanced chip packaging facilities, construction of additional power plants, and the geographic spread of data centers to leverage multiple power networks. To account for these changes, we incorporate projections from various public sources: semiconductor foundries' planned expansions, electricity providers' capacity growth forecasts, other relevant industry data, and our own research. We find that training runs of 2e29 FLOP will likely be feasible by the end of this decade. In other words, by 2030 it will be very likely possible to train models that exceed GPT-4 in scale to the same degree that GPT-4 exceeds GPT-2 in scale. 2 If pursued, we might see by the end of the decade advances in AI as drastic as the difference between the rudimentary text generation of GPT-2 in 2019 and the sophisticated problem-solving abilities of GPT-4 in 2023. Whether AI developers will actually pursue this level of scaling depends on their willingness to invest hundreds of billions of dollars in AI expansion over the coming years. While we briefly discuss the economics of AI investment later, a thorough analysis of investment decisions is beyond the scope of this report. For each bottleneck we offer a conservative estimate of the relevant supply and the largest training run they would allow. 3 Throughout our analysis, we assume that training runs could last between two to nine months, reflecting the trend towards longer durations. We also assume that when distributing AI data center power for distributed training and chips companies will only be able to muster about 10% to 40% of the existing supply. 4 Power constraints. Plans for data center campuses of 1 to 5 GW by 2030 have already been discussed, which would support training runs ranging from 1e28 to 3e29 FLOP (for reference, GPT-4 was likely around 2e25 FLOP). Geographically distributed training could tap into multiple regions' energy infrastructure to scale further. Given current projections of US data center expansion, a US distributed network could likely accommodate 2 to 45 GW, which assuming sufficient inter-data center bandwidth would support training runs from 2e28 to 2e30 FLOP. Beyond this, an actor willing to...