The Nonlinear Library: LessWrong

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI #71: Farewell to Chevron, published by Zvi on July 5, 2024 on LessWrong. Chevron deference is no more. How will this impact AI regulation? The obvious answer is it is now much harder for us to 'muddle through via existing laws and regulations until we learn more,' because the court narrowed our affordances to do that. And similarly, if and when Congress does pass bills regulating AI, they are going to need to 'lock in' more decisions and grant more explicit authority, to avoid court challenges. The argument against state regulations is similarly weaker now. Similar logic also applies outside of AI. I am overall happy about overturning Chevron and I believe it was the right decision, but 'Congress decides to step up and do its job now' is not in the cards. We should be very careful what we have wished for, and perhaps a bit burdened by what has been. The AI world continues to otherwise be quiet. I am sure you will find other news. Table of Contents 1. Introduction. 2. Table of Contents. 3. Language Models Offer Mundane Utility. How will word get out? 4. Language Models Don't Offer Mundane Utility. Ask not what you cannot do. 5. Man in the Arena. Why is Claude Sonnet 3.5 not at the top of the Arena ratings? 6. Fun With Image Generation. A map of your options. 7. Deepfaketown and Botpocalypse Soon. How often do you need to catch them? 8. They Took Our Jobs. The torture of office culture is now available for LLMs. 9. The Art of the Jailbreak. Rather than getting harder, it might be getting easier. 10. Get Involved. NYC space, Vienna happy hour, work with Bengio, evals, 80k hours. 11. Introducing. Mixture of experts becomes mixture of model sizes. 12. In Other AI News. Pixel screenshots as the true opt-in Microsoft Recall. 13. Quiet Speculations. People are hard to impress. 14. The Quest for Sane Regulation. SB 1047 bad faith attacks continue. 15. Chevron Overturned. A nation of laws. Whatever shall we do? 16. The Week in Audio. Carl Shulman on 80k hours and several others. 17. Oh Anthropic. You also get a nondisparagement agreement. 18. Open Weights Are Unsafe and Nothing Can Fix This. Says Lawrence Lessig. 19. Rhetorical Innovation. You are here. 20. Aligning a Smarter Than Human Intelligence is Difficult. Fix your own mistakes? 21. People Are Worried About AI Killing Everyone. The path of increased risks. 22. Other People Are Not As Worried About AI Killing Everyone. Feel no AGI. 23. The Lighter Side. Don't. I said don't. Language Models Offer Mundane Utility Guys. Guys. Ouail Kitouni: if you don't know what claude is im afraid you're not going to get what this ad even is :/ Ben Smith: Claude finds this very confusing. I get it, because I already get it. But who is the customer here? I would have spent a few extra words to ensure people knew this was an AI and LLM thing? Anthropic's marketing problem is that no one knows about Claude or Anthropic. They do not even know Claude is a large language model. Many do not even appreciate what a large language model is in general. I realize this is SFO. Claude anticipates only 5%-10% of people will understand what it means, and while some will be intrigued and look it up, most won't. So you are getting very vague brand awareness and targeting the congnesenti who run the tech companies, I suppose? Claude calls it a 'bold move that reflects confidence.' Language Models Don't Offer Mundane Utility David Althus reports that Claude does not work for him because of its refusals around discussions of violence. Once again, where are all our cool AI games? Summarize everything your users did yesterday? Steve Krouse: As a product owner it'd be nice to have an llm summary of everything my users did yesterday. Calling out cool success stories or troublesome error states I should reach out to debug. Has anyone tried such a thing? I am th...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Doomsday Argument and the False Dilemma of Anthropic Reasoning, published by Ape in the coat on July 5, 2024 on LessWrong. Doomsday Inference Can we use probability theory to estimate how many people there will be throughout the whole human history? Sure. We can build a probability model, that takes into account birth rates, possible existential hazards, ways to mitigate them and multiple other factors. Such models tend not to be very precise, so we would have pretty large confidence intervals but, we would still have some estimate. Hmm... this sounds like a lot of work for not much of a result. Can't we just use the incredible psychic powers of anthropics to circumvent all that, and get a very confident estimate just from the fact that we exist? Consider this: Suppose that there are two indistinguishable possibilities: a short human history, in which there are only 100 billion people and a long human history, in whichthere are 100 trillion people. You happen to be born among the 6th 10-billion group of people. What should be your credence that the history is short? As short and long history are a priori indistinguishable and mutually exclusive: P(Short)=P(Long)=1/2 Assuming that you are a random person among all the people destined to be born: P(6|Short)=1/10 P(6|Long)=1/10000 According to the Law of Total Probability: P(6)=P(6|Short)P(Short)+P(6|Long)P(Long)=0.05005 Therefore by Bayes' Theorem: P(Short|6)=P(6|Short)P(Short)/P(6)>0.999 We should be extremely confident that humanity will have a short history, just by the fact that we exist right now. This strong update in favor of short history solely due to the knowledge of your birth rank is known as the Doomsday Inference. I remember encountering it for the first time. I immediately felt that it can't be right. Back in the day I didn't have the right lexicon to explain why cognition engines can't produce knowledge this way. I wasn't familiar with the concept of noticing my own confusion. But I've already accustomed myself with several sophisms, and even practiced constructing some myself. So I noticed the familiar feeling of "trickery" that signaled that one of the assumptions is wrong. I think it took me a couple of minutes to find it. I recommend for everyone to try to do it themselves right now. It's not a difficult problem to begin with, and should be especially easy if you've read and understood my sequence on Sleeping Beauty problem. . . . . . . . . . Did you do it? . . . . . . . . . Well, regardless, there will be more time for it. First, let's discuss the fact that both major anthropic theories SSA and SIA accept the doomsday inference, because they are crazy and wrong and we live in an extremely embarrassing timeline. Biting the Doomsday Bullet Consider this simple and totally non-anthropic probability theory problem: Suppose there are two undistinguishable bags with numbered pieces of paper. The first bag has 10 pieces of paper and the second has 10000. You were given a random piece of paper from one of the bags and it happens to have number 6. What should be your credence that you've just picked a piece of paper from the first bag? The solution is totally analogous to the Doomsday Inference above: P(First)=P(Second)=1/2 P(6|First)=1/10 P(6|Second)=1/10000 P(6)=P(6|First)P(First)+P(6|Second)P(Second)=0.05005 P(First|6)=P(6|First)P(First)/P(6)=0.05/0.05005>0.999 But here there is no controversy. Nothing appears to be out of order. This is the experiment you can conduct and see for yourself that indeed, the absolute majority of cases where you get the piece of paper with number 6 happen when the paper was picked from the first bag. And so if we accept this logic here, we should also accept the Doomsday Inference, shouldn't we? Unless you want to defy Bayes' theorem itself! Maybe the ability to predict the ...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Consider the humble rock (or: why the dumb thing kills you), published by pleiotroth on July 5, 2024 on LessWrong. When people think about street-fights and what they should do when they find themselves in the unfortunate position of being in one, they tend to stumble across a pretty concerning thought relatively early on: "What if my attacker has a knife?" . Then they will put loads of cognitive effort into strategies for how to deal with attackers wielding blades. On first glance this makes sense. Knives aren't that uncommon and they are very scary, so it feels pretty dignified to have prepared for such scenarios (I apologize if this anecdote is horribly unrelatable to Statesians). The issue is that -all in all- knife related injuries from brawls or random attacks aren't that common in most settings. Weapons of opportunity (a rock, a brick, a bottle, some piece of metal, anything you can pick up in the moment) are much more common. They are less scary, but everyone has access to them and I've met few people without experience who come up with plans for defending against those before they start thinking about knives. It's not the really scary thing that kills you. It's the minimum viable thing. When deliberating poisons, people tend to think of the flashy, potent ones. Cyanide, Strychnine, Tetrodotoxin. Anything sufficiently scary with LDs in the low milligrams. The ones that are difficult to defend against and known first and foremost for their toxicity. On first pass this seems reasonable, but the fact that they are scary and hard to defend against means that it is very rare to encounter them. It is staggeringly more likely that you will suffer poisoning from Acetaminophen or the likes. OTC medications, cleaning products, batteries, pesticides, supplements. Poisons which are weak enough to be common. It's not the really scary thing that kills you. It's the minimum viable thing. My impression is that people in AI safety circles follow a similar pattern of directing most of their attention at the very competent, very scary parts of risk-space, rather than the large parts. Unless I am missing something, it feels pretty clear that the majority of doom-worlds are ones in which we die stupidly. Not by the deft hands of some superintelligent optimizer tiling the universe with its will, but the clumsy ones of a process that is powerful enough to kill a significant chunk of humanity but not smart enough to do anything impressive after that point. Not a schemer but an unstable idiot placed a little too close to a very spooky button by other unstable idiots. Killing enough of humanity that the rest will die soon after isn't that hard. We are very very fragile. Of course the sorts of scenarios which kill everyone immediately are less likely in worlds where there isn't competent, directed effort, but the post-apocalypse is a dangerous place and the odds that the people equipped to rebuild civilisation will be among the survivors, find themselves around the means to do so, make a few more lucky rolls on location and keep that spark going down a number of generations are low. Nowhere near zero but low. In bits of branch-space in which it is technically possible to bounce back given some factors, lots of timelines get shredded. You don't need a lot of general intelligence to design a bio-weapon or cause the leak of one. With militaries increasingly happy to hand weapons to black-boxes, you don't need to be very clever to start a nuclear incident. The meme which makes humanity destroy itself too might be relatively simple. In most worlds, before you get competent maximizers with the kind of goal content integrity, embedded agency and all the rest to kill humanity deliberately, keep the lights on afterwards and have a plan for what to do next, you get a truly baffling number of fla...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Static Analysis As A Lifestyle, published by adamShimi on July 4, 2024 on LessWrong. I've been watching French Top Chef (the best Top Chef, fight me) with my wife again, and I'm always impressed by how often the mentoring chefs, all with multiple michelin stars and years of experience, can just guess that a dish will work or that it will be missing something. By far, whenever a chef points to an error (not a risk, an error), it's then immediately validated experimentally: either the candidate corrected it and the jury comments positively on that aspect of the dish, or they refused to and failed because of that aspect of the dish. Obviously, this incredible skill comes from years of cooking experience. But at its core, this is one of the fundamental idea of epistemology that experts and masters rediscover again and again in their field: static analysis. The core intuition of static analysis is that when you write a computer program, you can check some things without even running it, just by looking at it and analyzing it. What most programmers know best are type systems, which capture what can be done with different values in the program, and forbid incompatible operations (like adding a number and a string of characters together, or more advanced things like using memory that might already be deallocated). But static analysis is far larger than that: it include verifying programs with proof assistants, model checking where you simulate many different possible situations without even running tests, abstract interpretation where you approximate the program so you can check key properties on them… At its core, static analysis focuses on what can be checked rationally, intellectually, logically, without needing to dirty your hands in the real world. Which is precisely what the mentoring chefs are doing! They're leveraging their experience and knowledge to simulate the dish, and figure out if it runs into some known problems: lack of a given texture, preponderance of a taste, lack of complexity (for the advanced gastronomy recipes that Top Chef candidates need to invent)… Another key intuition from static analysis which translates well to the Top Chef example is that it's much easier to check for specific failure modes than to verify correctness. It's easier to check that I'm not adding a number and a string than it is to check that I'm adding the right two number, say the price of the wedding venue and the price of the DJ. It's this aspect of static analysis, looking for the mistakes that you know (from experience or scholarship, which is at its best the distilled experience of others), which is such a key epistemological technique. I opened with the Top Chef example, but almost any field of knowledge, engineering, art, is full of similar cases: In Physics, there is notably dimensional analysis, which checks that two sides of an equation have the same unit, and order of magnitude estimates, which check that a computation is not ridiculously off. In Chemistry, there is the balancing of chemical equations, in terms of atoms and electrons. In Drug Testing, there are specific receptors that you know your compound should absolutely not bind with, or it will completely mess up the patient. In most traditional field of engineering, you have simulations and back of the envelope checks that let's you avoid the most egregious failures. In Animation, the original Disney animators came up with the half-filled flour sack test to check that they hadn't squashed and stretched their characters beyond recognition But there's something even deeper about these checks: they are often incomplete. In technical terms, a static analysis technique is complete if it accepts every correct program (and sound if it rejects all incorrect programs, but that's not the main point here). Of course, there...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: When Are Results from Computational Complexity Not Too Coarse?, published by Dalcy on July 4, 2024 on LessWrong. Tl;dr, While an algorithm's computational complexity may be exponential in general (worst-case), it is often possible to stratify its input via some dimension k that makes it polynomial for a fixed k, and only exponential in k. Conceptually, this quantity captures the core aspect of a problem's structure that makes specific instances of it 'harder' than others, often with intuitive interpretations. Example: Bayesian Inference and Treewidth One can easily prove exact inference (decision problem of: "is P(X)>0?") is NP-hard by encoding SAT-3 as a Bayes Net. Showing that it's NP is easy too. Therefore, inference is NP-complete, implying that algorithms are worst-case exponential. But this can't be the typical case! Let's examine the example of a Bayes Net whose structure is a chain ABCD, and say you want to compute the marginals P(D). The Naive Algorithm for marginalization would be to literally multiply all the conditional probability distribution (CPD) tables for each of the Bayes Net's nodes, and sum over all the variables other than X. If we assume each variable has at most v values, then the computational complexity is exponential in the number of variables n. P(D)=ABCP(A,B,C,D), which is O(vn). But because of the factorization P(A,B,C,D)=P(A)P(B|A)P(C|B)P(D|C) due to the chain structure, we can shift the order of the sum around like this: P(D)=CP(D|C)BP(C|B)AP(A)P(B|A) and now the sum can be done in O(nv2). Why? Notice AP(A)P(B|A) is P(B), and to compute P(B=b) we need to multiply v times and sum v1 times, overall O(v). This needs to be done for every b, so O(v2). Now we have cached P(B), and we move on to BP(C|B)P(B), where the same analysis applies. This is basically dynamic programming. So, at least for chains, inference can be done in linear time in input size. The earlier NP-completeness result, remember, is a worst-case analysis that applies to all possible Bayes Nets, ignoring the possible structure in each instance that may make some easier to solve than the other. Let's attempt a more fine complexity analysis by taking into account the structure of the Bayes Net provided, based on the chain example. Intuitively, the relevant structure of the Bayes Net that determines the difficulty of marginalization is the 'degree of interaction' among the variables, since the complexity is exponential in the "maximum number of factors ever seen within a sum," which was 2 in the case of a chain. How do we generalize this quantity to graphs other than chains? Since we could've shuffled the order of the sums and products differently (which would still yield O(nv2) for chains, but for general graphs the exponent may change significantly), for a given graph we want to find the sum-shuffling-order that minimizes the number of factors ever seen within a sum, and call that number k, an invariant of the graph that captures the difficulty of inference - O(mvk)[1] This is a graphical quantity of your graph called treewidth[2][3]. So, to sum up: We've parameterized the possible input Bayes Nets using some quantity k. Where k stratifies the inference problem in terms of their inherent difficulty, i.e. computational complexity is exponential in k, but linear under fixed or bounded k. We see that k is actually a graphical quantity known as treewidth, which intuitively corresponds to the notion of 'degree of interaction' among variables. General Lesson While I was studying basic computational complexity theory, I found myself skeptical of the value of various complexity classes, especially due to the classes being too coarse and not particularly exploiting the structures specific to the problem instance: The motif of proving NP-hardness by finding a clever way to encode 3-SA...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Introduction to French AI Policy, published by Lucie Philippon on July 4, 2024 on LessWrong. This post was written as part of the AI Governance Fundamentals course by BlueDot. I thank Charles Beasley and the students from my cohort for their feedback and encouragements. Disclaimer: The French policy landscape is in rapid flux, after president Macron called for a snap election on 1st and 7th July. The situation is still unfolding, and the state of French AI policy may be significantly altered. At various AI governance events, I noticed that most people had a very unclear vision of what was happening in AI policy in France, why the French government seemed dismissive of potential AI risks, and what that would that mean for the next AI Safety Summit in France. The post below is my attempt at giving a quick intro to the key stakeholders of AI policy in France, their positions and how they influence international AI policy efforts. My knowledge comes from hanging around AI safety circles in France for a year and a half, and working since January with the French Government on AI Governance. Therefore, I'm confident in the facts, but less in the interpretations, as I'm no policy expert myself. Generative Artificial Intelligence Committee The first major development in AI policy in France was the creation of a committee advising the government on Generative AI questions. This committee was created in September 2023 by former Prime Minister Elisabeth Borne.[1] The goals of the committee were: Strengthening AI training programs to develop more AI talent in France Investing in AI to promoting French innovation on the international stage Defining appropriate regulation for different sectors to protect against abuses. This committee was composed of notable academics and companies in the French AI field. This is a list of their notable member: Co-chairs: Philippe Aghion, an influential French economist specializing in innovation. He thinks AI will give a major productivity boost and that the EU should invest in major research projects on AI and disruptive technologies. Anne Bouverot, chair of the board of directors of ENS, the most prestigious scientific college in France. She was later nominated as leading organizer of the next AI Safety Summit. She is mainly concerned about the risks of bias and discrimination from AI systems, as well as risks of concentration of power. Notable members: Joëlle Barral, scientific director at Google Nozha Boujemaa, co-chair of the OECD AI expert group and Digital Trust Officer at Decathlon Yann LeCun, VP and Chief AI Scientist at Meta, generative AI expert He is a notable skeptic of catastrophic risks from AI Arthur Mensch, founder of Mistral He is a notable skeptic of catastrophic risks from AI Cédric O, consultant, former Secretary of State for Digital Affairs He invested in Mistral and worked to loosen the regulations on general systems in the EU AI Act. Martin Tisné, board member of Partnership on AI He will lead the "AI for good" track of the next Summit. See the full list of members in the announcement: Comité de l'intelligence artificielle générative. "AI: Our Ambition for France" In March 2024, the committee published a report highlighting 25 recommendations to the French government regarding AI. An official English version is available. The report makes recommendations on how to make France competitive and a leader in AI, by investing in training, R&D and compute. This report is not anticipating future development, and treats the current capabilities of AI as a fixed point we need to work with. They don't think about future capabilities of AI models, and are overly dismissive of AI risks. Some highlights from the report: It dismisses most risks from AI, including catastrophic risks, saying that concerns are overblown. They compare fear of...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: 3C's: A Recipe For Mathing Concepts, published by johnswentworth on July 3, 2024 on LessWrong. Opening Example: Teleology When people say "the heart's purpose is to pump blood" or "a pencil's function is to write", what does that mean physically? What are "purpose" or "function", not merely in intuitive terms, but in terms of math and physics? That's the core question of what philosophers call teleology - the study of "telos", i.e. purpose or function or goal. This post is about a particular way of approaching conceptual/philosophical questions, especially for finding "True Names" - i.e. mathematical operationalizations of concepts which are sufficiently robust to hold up under optimization pressure. We're going to apply the method to teleology as an example. We'll outline the general approach in abstract later; for now, try to pay attention to the sequence of questions we ask in the context of teleology. Cognition We start from the subjective view: set aside (temporarily) the question of what "purpose" or "function" mean physically. Instead, first ask what it means for me to view a heart as "having the purpose of pumping blood", or ascribe the "function of writing" to a pencil. What does it mean to model things as having purpose or function? Proposed answer: when I ascribe purpose or function to something, I model it as having been optimized (in the sense usually used on LessWrong) to do something. That's basically the standard answer among philosophers, modulo expressing the idea in terms of the LessWrong notion of optimization. (From there, philosophers typically ask about "original teleology" - i.e. a hammer has been optimized by a human, and the human has itself been optimized by evolution, but where does that chain ground out? What optimization process was not itself produced by another optimization process? And then the obvious answer is "evolution", and philosophers debate whether all teleology grounds out in evolution-like phenomena. But we're going to go in a different direction, and ask entirely different questions.) Convergence Next: I notice that there's an awful lot of convergence in what things different people model as having been optimized, and what different people model things as having been optimized for. Notably, this convergence occurs even when people don't actually know about the optimization process - for instance, humans correctly guessed millenia ago that living organisms had been heavily optimized somehow, even though those humans were totally wrong about what process optimized all those organisms; they thought it was some human-like-but-more-capable designer, and only later figured out evolution. Why the convergence? Our everyday experience implies that there is some property of e.g. a heron such that many different people can look at the heron, convergently realize that the heron has been optimized for something, and even converge to some degree on which things the heron (or the parts of the heron) have been optimized for - for instance, that the heron's heart has been optimized to pump blood. (Not necessarily perfect convergence, not necessarily everyone, but any convergence beyond random chance is a surprise to be explained if we're starting from a subjective account.) Crucially, it's a property of the heron, and maybe of the heron's immediate surroundings, not of the heron's whole ancestral environment - because people can convergently figure out that the heron has been optimized just by observing the heron in its usual habitat. So now we arrive at the second big question: what are the patterns out in the world which different people convergently recognize as hallmarks of having-been-optimized? What is it about herons, for instance, which makes it clear that they've been optimized, even before we know all the details of the optimizati...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: List of Collective Intelligence Projects, published by Chipmonk on July 3, 2024 on LessWrong. During the last Foresight Intelligent Cooperation Workshop I got very curious about what collective intelligence tools currently exist. A list: Pol.is: "Input Crowd, Output Meaning" Inspired Twitter/X community notes People: Colin Megill, et al. Collective Intelligence Project vibe: democratic AI, "How AI and Democracy Can Fix Each Other" People: Divya Siddharth, Saffron Huang, et al. AI Objectives Institute Talk to the City: "an open-source LLM interface for improving collective deliberation and decision-making by analyzing detailed, qualitative data. It aggregates responses and arranges similar arguments into clusters." AI Objectives Institute works closely with the Taiwanese government. Other projects in development. People: Colleen McKenzie, Değer Turan, et al. Meaning Alignment Institute vibe: democratic AI, kinda. I think they think that if you can help individuals make wiser decisions, at scale, then this converges to be equivalent with solving outer alignment. Remesh Similar to pol.is AFAIK? I haven't played with it. People: Andrew Konya, et al. Loomio: "a flexible decision-making tool that helps you create a more engaged and collaborative culture, build trust and coordinate action" Deliberative Technology for Alignment paper They also discuss other tools for this use like Discord, Snapshot, Dembrane People: Andrew Konya, Deger Turan, Aviv Ovadya, Lina Qui, Daanish Masood, Flynn Devine, Lisa Schirch, Isabella Roberts, and Deliberative Alignment Forum Someone in the know told me to only read sections 4 and 5 of this paper Plurality Institute People: David Bloomin, Rose Bloomin, et al. Also working on some de-escalator bots for essentially Reddit comment wars Lots of crypto projects Quadratic voting Gitcoin Metagov: "a laboratory for digital governance" Soulbound tokens Various voting and aggregation systems, liquid democracy Decidem Decide Madrid Consider.it Stanford Online Deliberation Platform Lightcone Chord (in development) Brief description People: Jacob Lagerros (LessWrong) All of the prediction markets Manifold, Kalshi, Metaculus, PredictIt, etc. Midjourney has a Collective Intelligence Team now according to Ivan Vendrov's website. I couldn't find any other information online. What about small group collective intelligence tools? Most of the examples above are for large group collective intelligence (which I'm defining as ~300 people or much larger). But what about small groups? Are there tools that will help me coordinate with 30 friends? Or just one friend? I'm mostly unaware of any recent innovations for small group collective intelligence tools. Do you know of any? Nexae (in development) "Nexae Systems builds sociotechnical infrastructure to enable the creation of new types of businesses and organizations." double crux bot I'm surprised I haven't heard of many other LLM-facilitated communication tools Medium group (~30-300 people) projects: Jason Benn's unconference tools, eg Idea Ranker. Other lists @exgenesis short tweet thread. Couple things I haven't listed here. Plurality Institute's (WIP) map of related orgs, etc. Know of any I should add? Opportunities RFP: Interoperable Deliberative Tools | interop, $200k. Oops this closed before I published this post. Metagov is running https://metagov.org/projects/ai-palace which seems similar Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How ARENA course material gets made, published by CallumMcDougall on July 3, 2024 on LessWrong. TL;DR In this post, I describe my methodology for building new material for ARENA. I'll mostly be referring to the exercises on IOI, Superposition and Function Vectors as case studies. I expect this to be useful for people who are interested in designing material for ARENA or ARENA-like courses, as well as people who are interested in pedagogy or ML paper replications. The process has 3 steps: 1. Start with something concrete 2. First pass: replicate, and understand 3. Second pass: exercise-ify Summary I'm mostly basing this on the following 3 sets of exercises: Indirect Object Identification - these exercises focus on the IOI paper (from Conmy et al). The goal is to have people understand what exploratory analysis of transformers looks like, and introduce the key ideas of the circuits agenda. Superposition & SAEs - these exercises focus on understanding superposition and the agenda of dictionary learning (specifically sparse autoencoders). Most of the exercises explore Anthropic's Toy Models of Superposition paper, except for the last 2 sections which explore sparse autoencoders (firstly by applying them to the toy model setup, secondly by exploring a sparse autoencoder trained on a language model). Function Vectors - these exercises focus on the Function Vectors paper by David Bau et al, although they also make connections with related work such as Alex Turner's GPT2-XL steering vector work. These exercises were interesting because they also had the secondary goal of being an introduction to the nnsight library, in much the same way that the intro to mech interp exercises were also an introduction to TransformerLens. The steps I go through are listed below. I'm indexing from zero because I'm a software engineer so of course I am. The steps assume you already have an idea of what exercises you want to create; in Appendix (1) you can read some thoughts on what makes for a good exercise set. 1. Start with something concrete When creating material, you don't want to be starting from scratch. It's useful to have source code available to browse - bonus points if that takes the form of a Colab or something which is self-contained and has easily visible output. IOI - this was Neel's "Exploratory Analysis Demo" exercises. The rest of the exercises came from replicating the paper directly. Superposition - this was Anthroic's Colab notebook (although the final version went quite far beyond this). The very last section (SAEs on transformers) was based on Neel Nanda's demo Colab). Function Vectors - I started with the NDIF demo notebook, to show how some basic nnsight syntax worked. As for replicating the actual function vectors paper, unlike the other 2 examples I was mostly just working from the paper directly. It helped that I was collaborating with some of this paper's authors, so I was able to ask them some questions to clarify aspects of the paper. 2. First-pass: replicate, and understand The first thing I'd done in each of these cases was go through the material I started with, and make sure I understood what was going on. Paper replication is a deep enough topic for its own series of blog posts (many already exist), although I'll emphasise that I'm not usually talking about full paper replication here, because ideally you'll be starting from something a it further along, be that a Colab, a different tutorial, or something else. And even when you are just working directly from a paper, you shouldn't make the replication any harder for yourself than you need to. If there's code you can take from somewhere else, then do. My replication usually takes the form of working through a notebook in VSCode. I'll either start from scratch, or from a downloaded Colab if I'm using one as a ...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Economics Roundup #2, published by Zvi on July 3, 2024 on LessWrong. Previously: Economics Roundup #1 Let's take advantage of the normality while we have it. In all senses. Insane Tax Proposals There is Trump's proposal to replace income taxes with tariffs, but he is not alone. So here is your periodic reminder, since this is not actually new at core: Biden's proposed budgets include completely insane tax regimes that would cripple our economic dynamism and growth if enacted. As in for high net worth individuals, taking unrealized capital gains at 25% and realized capital gains, such as those you are forced to take to pay your unrealized capital gains tax, at 44.6% plus state taxes. Austen Allred explains how this plausibly destroys the entire startup ecosystem. Which I know is confusing because in other contexts he also talks about how other laws (such as SB 1047) that would in no way apply to startups would also destroy the startup ecosystem. But in this case he is right. Austen Allred: It's difficult to describe how insane a 25% tax on unrealized capital gains is. Not a one-time 25% hit. It's compounding, annually taking 25% of every dollar of potential increase before it can grow. Not an exaggeration to say it could single-handedly crush the economy. An example to show how insane this is: You're a founder and you start a company. You own… let's say 30% of it. Everything is booming, you raise a round that values the company at at $500 million. You now personally owe $37.5 million in taxes. This year. In cash. Now there are investors who want to invest in the company, but you can't just raise $37.5 million in cash overnight. So what happens? Well, you simply decide not to have a company worth a few hundred million dollars. Oh well, that's only a handful of companies right? Well, as an investor, the only way the entire ecosystem works is if a few companies become worth hundreds of millions. Without that, venture capital no longer works. Investment is gone. Y Combinator no longer works. No more funding, mass layoffs, companies shutting down crushes the revenue of those that are still around. Economic armageddon. We've seen how these spirals work, and it's really bad for everyone. Just because bad policy only targets rich people doesn't mean it can't kill the economy or make it good policy. I do think they are attempting to deal with this via another idea he thought was crazy, the 'nine annual payments' for the first year's tax and 'five annual payments' for the subsequent tax. So the theory would be that the first year you 'only' owe 3.5%. Then the second year you owe another 3.5% of the old gain and 5% of the next year's gain. That is less horrendous, but still super horrendous, especially if the taxes do not go away if the asset values subsequently decline, risking putting you into infinite debt. This is only the beginning. They are even worse than Warren's proposed wealth taxes, because the acute effects and forcing function here are so bad. At the time this was far worse than the various stupid and destructive economic policies Trump was proposing, although he has recently stepped it up to the point where that is unclear. The good news is that these policies are for now complete political non-starters. Never will a single Republican vote for this, and many Democrats know better. I would like to think the same thing in reverse, as well. Also, this is probably unconstitutional in the actually-thrown-out-by-SCOTUS sense, not only in the violates-the-literal-constitution sense. But yes, it is rather terrifying what would happen if they had the kind of majorities that could enact things like this. On either side. Why didn't the super high taxes in the 1950s kill growth? Taxes for most people were not actually that high, the super-high marginal rates like 91% kicked in...