The Nonlinear Library

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI #71: Farewell to Chevron, published by Zvi on July 5, 2024 on LessWrong. Chevron deference is no more. How will this impact AI regulation? The obvious answer is it is now much harder for us to 'muddle through via existing laws and regulations until we learn more,' because the court narrowed our affordances to do that. And similarly, if and when Congress does pass bills regulating AI, they are going to need to 'lock in' more decisions and grant more explicit authority, to avoid court challenges. The argument against state regulations is similarly weaker now. Similar logic also applies outside of AI. I am overall happy about overturning Chevron and I believe it was the right decision, but 'Congress decides to step up and do its job now' is not in the cards. We should be very careful what we have wished for, and perhaps a bit burdened by what has been. The AI world continues to otherwise be quiet. I am sure you will find other news. Table of Contents 1. Introduction. 2. Table of Contents. 3. Language Models Offer Mundane Utility. How will word get out? 4. Language Models Don't Offer Mundane Utility. Ask not what you cannot do. 5. Man in the Arena. Why is Claude Sonnet 3.5 not at the top of the Arena ratings? 6. Fun With Image Generation. A map of your options. 7. Deepfaketown and Botpocalypse Soon. How often do you need to catch them? 8. They Took Our Jobs. The torture of office culture is now available for LLMs. 9. The Art of the Jailbreak. Rather than getting harder, it might be getting easier. 10. Get Involved. NYC space, Vienna happy hour, work with Bengio, evals, 80k hours. 11. Introducing. Mixture of experts becomes mixture of model sizes. 12. In Other AI News. Pixel screenshots as the true opt-in Microsoft Recall. 13. Quiet Speculations. People are hard to impress. 14. The Quest for Sane Regulation. SB 1047 bad faith attacks continue. 15. Chevron Overturned. A nation of laws. Whatever shall we do? 16. The Week in Audio. Carl Shulman on 80k hours and several others. 17. Oh Anthropic. You also get a nondisparagement agreement. 18. Open Weights Are Unsafe and Nothing Can Fix This. Says Lawrence Lessig. 19. Rhetorical Innovation. You are here. 20. Aligning a Smarter Than Human Intelligence is Difficult. Fix your own mistakes? 21. People Are Worried About AI Killing Everyone. The path of increased risks. 22. Other People Are Not As Worried About AI Killing Everyone. Feel no AGI. 23. The Lighter Side. Don't. I said don't. Language Models Offer Mundane Utility Guys. Guys. Ouail Kitouni: if you don't know what claude is im afraid you're not going to get what this ad even is :/ Ben Smith: Claude finds this very confusing. I get it, because I already get it. But who is the customer here? I would have spent a few extra words to ensure people knew this was an AI and LLM thing? Anthropic's marketing problem is that no one knows about Claude or Anthropic. They do not even know Claude is a large language model. Many do not even appreciate what a large language model is in general. I realize this is SFO. Claude anticipates only 5%-10% of people will understand what it means, and while some will be intrigued and look it up, most won't. So you are getting very vague brand awareness and targeting the congnesenti who run the tech companies, I suppose? Claude calls it a 'bold move that reflects confidence.' Language Models Don't Offer Mundane Utility David Althus reports that Claude does not work for him because of its refusals around discussions of violence. Once again, where are all our cool AI games? Summarize everything your users did yesterday? Steve Krouse: As a product owner it'd be nice to have an llm summary of everything my users did yesterday. Calling out cool success stories or troublesome error states I should reach out to debug. Has anyone tried such a thing? I am th...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Interim research report] Activation plateaus & sensitive directions in GPT2, published by Stefan Heimersheim on July 5, 2024 on The AI Alignment Forum. This part-report / part-proposal describes ongoing research, but I'd like to share early results for feedback. I am especially interested in any comment finding mistakes or trivial explanations for these results. I will work on this proposal with a LASR Labs team over the next 3 months. If you are working (or want to work) on something similar I would love to chat! Experiments and write-up by Stefan, with substantial inspiration and advice from Jake (who doesn't necessarily endorse every sloppy statement I write). Work produced at Apollo Research. TL,DR: Toy models of how neural networks compute new features in superposition seem to imply that neural networks that utilize superposition require some form of error correction to avoid interference spiraling out of control. This means small variations along a feature direction shouldn't affect model outputs, which I can test: 1. Activation plateaus: Real activations should be resistant to small perturbations. There should be a "plateau" in the output as a function of perturbation size. 2. Sensitive directions: Perturbations towards the direction of a feature should change the model output earlier (at a lower perturbation size) than perturbations into a random direction. I find that both of these predictions hold; the latter when I operationalize "feature" as the difference between two real model activations. As next steps we are planning to Test both predictions for SAE features: We have some evidence for the latter by Gurnee (2024) and Lindsey (2024). Are there different types of SAE features, atomic and composite features? Can we get a handle on the total number of features? If sensitivity-features line up with SAE features, can we find or improve SAE feature directions by finding local optima in sensitivity (similar to how Mack & Turner (2024) find steering vectors)? My motivation for this project is to get data on computation in superposition, and to get dataset-independent evidence for (SAE-)features. Core results & discussion I run two different experiments that test the error correction hypothesis: 1. Activation Plateaus: A real activation is the center of a plateau, in the sense that perturbing the activation affects the model output less than expected. Concretely: applying random-direction perturbations to an activation generated from a random openwebtext input ("real activation") has less effect than applying the same perturbations to a random activation (generated from a Normal distribution). This effect on the model can be measured in KL divergence of logits (shown below) but also L2 difference or cosine similarity of late-layer activations. 2. Sensitive directions: Perturbing a (real) activation into a direction towards another real activation ("poor man's feature directions") affects the model-outputs more than perturbing the same activation into a random direction. In the plot below focus on the size of the "plateau" in the left-hand side 1. Naive random direction vs mean & covariance-adjusted random: Naive isotropic random directions are much less sensitive. Thus we use mean & covariance-adjusted random activations everywhere else in this report. 2. The sensitive direction results are related to Gurnee (2024, SAE-replacement-error direction vs naive random direction) and Lindsey (2024, Anthropic April Updates, SAE-feature direction vs naive random direction). The theoretical explanation for activation plateaus & sensitive direction may be error correction (also referred to as noise suppression): NNs in superposition should expect small amounts of noise in feature activations due to interference. (The exact properties depend on how computation happens in s...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Doomsday Argument and the False Dilemma of Anthropic Reasoning, published by Ape in the coat on July 5, 2024 on LessWrong. Doomsday Inference Can we use probability theory to estimate how many people there will be throughout the whole human history? Sure. We can build a probability model, that takes into account birth rates, possible existential hazards, ways to mitigate them and multiple other factors. Such models tend not to be very precise, so we would have pretty large confidence intervals but, we would still have some estimate. Hmm... this sounds like a lot of work for not much of a result. Can't we just use the incredible psychic powers of anthropics to circumvent all that, and get a very confident estimate just from the fact that we exist? Consider this: Suppose that there are two indistinguishable possibilities: a short human history, in which there are only 100 billion people and a long human history, in whichthere are 100 trillion people. You happen to be born among the 6th 10-billion group of people. What should be your credence that the history is short? As short and long history are a priori indistinguishable and mutually exclusive: P(Short)=P(Long)=1/2 Assuming that you are a random person among all the people destined to be born: P(6|Short)=1/10 P(6|Long)=1/10000 According to the Law of Total Probability: P(6)=P(6|Short)P(Short)+P(6|Long)P(Long)=0.05005 Therefore by Bayes' Theorem: P(Short|6)=P(6|Short)P(Short)/P(6)>0.999 We should be extremely confident that humanity will have a short history, just by the fact that we exist right now. This strong update in favor of short history solely due to the knowledge of your birth rank is known as the Doomsday Inference. I remember encountering it for the first time. I immediately felt that it can't be right. Back in the day I didn't have the right lexicon to explain why cognition engines can't produce knowledge this way. I wasn't familiar with the concept of noticing my own confusion. But I've already accustomed myself with several sophisms, and even practiced constructing some myself. So I noticed the familiar feeling of "trickery" that signaled that one of the assumptions is wrong. I think it took me a couple of minutes to find it. I recommend for everyone to try to do it themselves right now. It's not a difficult problem to begin with, and should be especially easy if you've read and understood my sequence on Sleeping Beauty problem. . . . . . . . . . Did you do it? . . . . . . . . . Well, regardless, there will be more time for it. First, let's discuss the fact that both major anthropic theories SSA and SIA accept the doomsday inference, because they are crazy and wrong and we live in an extremely embarrassing timeline. Biting the Doomsday Bullet Consider this simple and totally non-anthropic probability theory problem: Suppose there are two undistinguishable bags with numbered pieces of paper. The first bag has 10 pieces of paper and the second has 10000. You were given a random piece of paper from one of the bags and it happens to have number 6. What should be your credence that you've just picked a piece of paper from the first bag? The solution is totally analogous to the Doomsday Inference above: P(First)=P(Second)=1/2 P(6|First)=1/10 P(6|Second)=1/10000 P(6)=P(6|First)P(First)+P(6|Second)P(Second)=0.05005 P(First|6)=P(6|First)P(First)/P(6)=0.05/0.05005>0.999 But here there is no controversy. Nothing appears to be out of order. This is the experiment you can conduct and see for yourself that indeed, the absolute majority of cases where you get the piece of paper with number 6 happen when the paper was picked from the first bag. And so if we accept this logic here, we should also accept the Doomsday Inference, shouldn't we? Unless you want to defy Bayes' theorem itself! Maybe the ability to predict the ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: ML4Good Summer Bootcamps - Applications Open, published by Nia on July 5, 2024 on The Effective Altruism Forum. Apply to ML4Good's Summer Bootcamps! ML4Good bootcamps are 10-day bootcamps focusing on upskilling in technical AI safety, exploring governance, and delving into conceptual topics. ML4Good is a non-profit project and free of charge for the participants. We're seeking motivated individuals with some coding experience who want to make a difference in the field of AI Safety. Activities: Peer-coding sessions following a technical curriculum with mentors Presentations by experts in the field Review and discussion of AI Safety literature Personal career advice and mentorship Dates: Applications are now open for our UK and Germany bootcamps. UK Bootcamp August 31st to September 10th, 2024 Application deadline: July 7th, 2024 Germany Bootcamp September 23rd to October 3rd, 2024 Application deadline: July 14th, 2024 More camps are being planned - sign up on our website to stay informed about upcoming bootcamps and application deadlines. More details on our website: https://www.ml4good.org/programmes/upcoming-bootcamps Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Rethink Priorities' Digital Consciousness Project Announcement, published by Bob Fischer on July 5, 2024 on The Effective Altruism Forum. One of the core questions regarding the moral status of AI concerns their consciousness. Is there anything it's like to be them? Contemporary AI systems are widely regarded as clearly not conscious, but there seems to be growing concern among experts that we may see conscious AI systems in the not-too-distant future. Understanding our duties to the AI systems we create will involve assessing the nature of their minds, and thus their moral status. There are many important questions about AI minds that bear on their moral status, but whether they are consciousness has a clear and widely recognized role. In addition, it may be important in securing or denying AIs the public's moral consideration. Existing consciousness research revolves first and foremost around human beings. The physical bases (or neural correlates) of consciousness in humans remain uncertain. Leading proposals are both vague and highly controversial. Extending theories of consciousness to AIs will require careful thought about how to generalize beyond the human case. Alternatively, we might look to identify behavioral indicators of consciousness. Behavior has a much more salient role in swaying our attitudes than abstract considerations of architecture. But modern AIs are carefully trained to behave like us, and so it is not easy to tell whether their behaviors indicate anything beyond mimicry. Therefore, we see a variety of kinds of uncertainty at play: there is methodological uncertainty, uncertainty regarding the underpinnings of human consciousness, uncertainty regarding the significance of behavioral evidence, uncertainty about how AIs work, etc. Coming up with any concrete estimate of the probability of consciousness in AI systems will require mapping, measuring, and aggregating these uncertainties. Rethink Priorities has overcome similar challenges before. Our Moral Weight Project wrangled patchy evidence about behavioral traits and cognitive capacities across the animal kingdom through a Monte Carlo framework that output probabilistic estimates of welfare ranges for different species. We learned a lot from this work and we are eager to apply those lessons to a new challenge. We are now turning to the question of how best to assess the probability of AI consciousness. Over the coming months, we plan to carry out a project encompassing the following tasks: 1. Evaluating different modeling approaches to AI consciousness estimation. What different paradigms are worth exploring? What are the pros and cons of each? 2. Identifying some plausible proxies for consciousness to feed into these models. What are the challenges in pinning down values for these proxies? Where might future technical work be most fruitful? 3. Producing a prototype model that translates uncertainty about different sources of evidence into probability ranges for contemporary and hypothetical future AI models. Given our uncertainties, what should we conclude about the overall probability of consciousness? Having such a model is valuable in a few different ways. First, we can produce an overall estimate of the probability that a given system is conscious - an estimate that's informed by, rather than undermined by, our uncertainty about the correct theory of consciousness. Second, because the inputs to the process can be updated with new information as, say, new capabilities come online, we can readily update our overall estimate of the probability of consciousness. Third, because we can repeat this process based on the capabilities that were present at earlier dates, we can also model the historical rate of change in the probability of digital consciousness. In principle, we can use that to make...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Consider the humble rock (or: why the dumb thing kills you), published by pleiotroth on July 5, 2024 on LessWrong. When people think about street-fights and what they should do when they find themselves in the unfortunate position of being in one, they tend to stumble across a pretty concerning thought relatively early on: "What if my attacker has a knife?" . Then they will put loads of cognitive effort into strategies for how to deal with attackers wielding blades. On first glance this makes sense. Knives aren't that uncommon and they are very scary, so it feels pretty dignified to have prepared for such scenarios (I apologize if this anecdote is horribly unrelatable to Statesians). The issue is that -all in all- knife related injuries from brawls or random attacks aren't that common in most settings. Weapons of opportunity (a rock, a brick, a bottle, some piece of metal, anything you can pick up in the moment) are much more common. They are less scary, but everyone has access to them and I've met few people without experience who come up with plans for defending against those before they start thinking about knives. It's not the really scary thing that kills you. It's the minimum viable thing. When deliberating poisons, people tend to think of the flashy, potent ones. Cyanide, Strychnine, Tetrodotoxin. Anything sufficiently scary with LDs in the low milligrams. The ones that are difficult to defend against and known first and foremost for their toxicity. On first pass this seems reasonable, but the fact that they are scary and hard to defend against means that it is very rare to encounter them. It is staggeringly more likely that you will suffer poisoning from Acetaminophen or the likes. OTC medications, cleaning products, batteries, pesticides, supplements. Poisons which are weak enough to be common. It's not the really scary thing that kills you. It's the minimum viable thing. My impression is that people in AI safety circles follow a similar pattern of directing most of their attention at the very competent, very scary parts of risk-space, rather than the large parts. Unless I am missing something, it feels pretty clear that the majority of doom-worlds are ones in which we die stupidly. Not by the deft hands of some superintelligent optimizer tiling the universe with its will, but the clumsy ones of a process that is powerful enough to kill a significant chunk of humanity but not smart enough to do anything impressive after that point. Not a schemer but an unstable idiot placed a little too close to a very spooky button by other unstable idiots. Killing enough of humanity that the rest will die soon after isn't that hard. We are very very fragile. Of course the sorts of scenarios which kill everyone immediately are less likely in worlds where there isn't competent, directed effort, but the post-apocalypse is a dangerous place and the odds that the people equipped to rebuild civilisation will be among the survivors, find themselves around the means to do so, make a few more lucky rolls on location and keep that spark going down a number of generations are low. Nowhere near zero but low. In bits of branch-space in which it is technically possible to bounce back given some factors, lots of timelines get shredded. You don't need a lot of general intelligence to design a bio-weapon or cause the leak of one. With militaries increasingly happy to hand weapons to black-boxes, you don't need to be very clever to start a nuclear incident. The meme which makes humanity destroy itself too might be relatively simple. In most worlds, before you get competent maximizers with the kind of goal content integrity, embedded agency and all the rest to kill humanity deliberately, keep the lights on afterwards and have a plan for what to do next, you get a truly baffling number of fla...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Finding the Wisdom to Build Safe AI, published by Gordon Seidoh Worley on July 4, 2024 on The AI Alignment Forum. We may soon build superintelligent AI. Such AI poses an existential threat to humanity, and all life on Earth, if it is not aligned with our flourishing. Aligning superintelligent AI is likely to be difficult because smarts and values are mostly orthogonal and because Goodhart effects are robust, so we can neither rely on AI to naturally decide to be safe on its own nor can we expect to train it to stay safe. We stand a better chance of creating safe, aligned, superintelligent AI if we create AI that is "wise", in the sense that it knows how to do the right things to achieve desired outcomes and doesn't fall into intellectual or optimization traps. Unfortunately, I'm not sure how to create wise AI, because I'm not exactly sure what it is to be wise myself. My current, high-level plan for creating wise AI is to first get wiser myself, then help people working on AI safety to get wiser, and finally hope that wise AI safety researchers can create wise, aligned AI that is safe. For close to a decade now I've been working on getting wiser, and in that time I've figured out a bit of what it is to be wise. I'm starting to work on helping others get wiser by writing a book that explains some useful epistemological insights I picked up between pursuing wisdom and trying to solve a subproblem in AI alignment, and have vague plans for another book that will be more directly focused on the wisdom I've found. I thus far have limited ideas about how to create wise AI, but I'll share my current thinking anyway in the hope that it inspires thoughts for others. Why would wise AI safety researchers matter? My theory is that it would be hard for someone to know what's needed to build a wise AI without first being wise themself, or at least having a wiser person to check ideas against. Wisdom clearly isn't sufficient for knowing how to build wise and aligned AI, but it does seem necessary under my assumptions, in the same way that it would be hard to develop a good decision theory for AI if one could not reason for oneself how to maximize expected value in games. How did I get wiser? Mostly by practicing Zen Buddhism, but also by studying philosophy, psychology, mathematics, and game theory to help me think about how to build aligned AI. I started practicing Zen in 2017. I picked Zen with much reluctance after trying many other things that didn't work for me, or worked for a while and then had nothing else to offer me. Things I tried included Less Wrong style rationality training, therapy, secular meditation, and various positive psychology practices. I even tried other forms of Buddhism, but Zen was the only tradition I felt at home with. Consequently, my understanding of wisdom is biased by Zen, but I don't think Zen has a monopoly on wisdom, and other traditions might produce different but equally useful theories of wisdom than what I will discuss below. What does it mean to be wise? I roughly define wisdom as doing the right thing at the right time for the right reasons. This definition puts the word "right" through a strenuous workout, so let's break it down. The "right thing" is doing that which causes outcomes that we like upon reflection. The "right time" is doing the right thing when it will have the desired impact. And the "right reasons" is having an accurate model of the world that correctly predicts the right thing and time. How can the right reasons be known? The straightforward method is to have true beliefs and use correct logic. Alas, we're constantly uncertain about what's true and, in real-world scenarios, the logic becomes uncomputable, so instead we often rely on heuristics that lead to good outcomes and avoid optimization traps. That we rely on heuristi...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: MHFC Spring '24 Grants, published by wtroy on July 4, 2024 on The Effective Altruism Forum. Summary: AIM's Mental Health Funding Circle ran its latest round of grants this spring, distributing $367,000 across six grantees. About the MHFC: Global mental health continues to be highly neglected despite contributing more to cumulative suffering than many more highly-prioritized physical health conditions. A handful of effectiveness-minded funders come together twice a year for an open grants round, looking to find and fund the most impactful mental health projects. If you or someone you know may be interested in joining us, we would warmly welcome new members! If you give (or are interested in giving) at least $50,000 annually to effective global mental health projects, please reach out or apply here to join. The Grants: As all circle members make their own funding decisions, the justification given for each grant may not represent the thinking behind each funder's actions. That being said, I'm writing some justifications because it's fun to do so, and otherwise this is just a dry and boring list. It's worth noting that due to limited resources, we were unfortunately unable to fund some highly promising applicants. Nonetheless, we are thrilled to support the following grantees: $83,000 to Restore Hope Liberia (RHL): RHL provides interpersonal group therapy (IPT-G) to depressed individuals in (you guessed it) Liberia. IPT-G, made famous by the likes of Strong Minds and Friendship Bench, is the gold standard therapy for cost-effective mental health treatment (as documented extensively by the Happier Lives Institute). RHL is one of the few mental health organizations operating in Liberia, a country and greater region with well-documented rates of depression and trauma. In addition, RHL is developing a tailored LLM that will help train their facilitators, eliminating one of their biggest bottlenecks to scaling. $125,000 to Overcome*: This organization provides free digital and phone-based therapy to sufferers of a variety of mental disorders in LMICs. By exploiting a unique demand for client-facing practice hours by highly-trained graduate students in high-income countries, Overcome can provide extremely cheap mental health treatment for sufferers with truly no other options. In addition to being potentially highly cost-effective, Overcome has shown great potential in its first year as an organization. $80,000 to the Swiss Tropical and Public Health Institute*: This grant supports the work of Irene Falgas-Bague on suicidality in Zambia. Given the scarcity of suicide data in Sub-Saharan Africa, and the proven success of initiatives like the Centre for Pesticide Suicide Prevention, this research could inform highly cost-effective interventions to reduce suicide rates. Preliminary data suggests that a significant percentage of suicides in Zambia may involve pesticides or other poisonous materials. $50,000 to VIMBO: This digital startup provides mental health support for individuals with depression in South Africa. Digital mental health interventions offer enormous scaling potential, and VIMBO's promising financial model could potentially become self-sustaining at scale. $50,000 to FineMind*: This grant supports FineMind's work on stepped care in northern Uganda. Their increasingly cost-effective intervention shows significant scaling potential. The stepped care model allows for the assessment of a large population and the provision of appropriate care based on symptom severity. $69,000 to Phlourish*: This funding supports Phlourish's work on guided self-help for adolescents in the Philippines. Guided self-help is one of the most promising interventions, and Filipino adolescents have disturbingly high rates of depression and self-harm. Phlourish is a promising young organization...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Static Analysis As A Lifestyle, published by adamShimi on July 4, 2024 on LessWrong. I've been watching French Top Chef (the best Top Chef, fight me) with my wife again, and I'm always impressed by how often the mentoring chefs, all with multiple michelin stars and years of experience, can just guess that a dish will work or that it will be missing something. By far, whenever a chef points to an error (not a risk, an error), it's then immediately validated experimentally: either the candidate corrected it and the jury comments positively on that aspect of the dish, or they refused to and failed because of that aspect of the dish. Obviously, this incredible skill comes from years of cooking experience. But at its core, this is one of the fundamental idea of epistemology that experts and masters rediscover again and again in their field: static analysis. The core intuition of static analysis is that when you write a computer program, you can check some things without even running it, just by looking at it and analyzing it. What most programmers know best are type systems, which capture what can be done with different values in the program, and forbid incompatible operations (like adding a number and a string of characters together, or more advanced things like using memory that might already be deallocated). But static analysis is far larger than that: it include verifying programs with proof assistants, model checking where you simulate many different possible situations without even running tests, abstract interpretation where you approximate the program so you can check key properties on them… At its core, static analysis focuses on what can be checked rationally, intellectually, logically, without needing to dirty your hands in the real world. Which is precisely what the mentoring chefs are doing! They're leveraging their experience and knowledge to simulate the dish, and figure out if it runs into some known problems: lack of a given texture, preponderance of a taste, lack of complexity (for the advanced gastronomy recipes that Top Chef candidates need to invent)… Another key intuition from static analysis which translates well to the Top Chef example is that it's much easier to check for specific failure modes than to verify correctness. It's easier to check that I'm not adding a number and a string than it is to check that I'm adding the right two number, say the price of the wedding venue and the price of the DJ. It's this aspect of static analysis, looking for the mistakes that you know (from experience or scholarship, which is at its best the distilled experience of others), which is such a key epistemological technique. I opened with the Top Chef example, but almost any field of knowledge, engineering, art, is full of similar cases: In Physics, there is notably dimensional analysis, which checks that two sides of an equation have the same unit, and order of magnitude estimates, which check that a computation is not ridiculously off. In Chemistry, there is the balancing of chemical equations, in terms of atoms and electrons. In Drug Testing, there are specific receptors that you know your compound should absolutely not bind with, or it will completely mess up the patient. In most traditional field of engineering, you have simulations and back of the envelope checks that let's you avoid the most egregious failures. In Animation, the original Disney animators came up with the half-filled flour sack test to check that they hadn't squashed and stretched their characters beyond recognition But there's something even deeper about these checks: they are often incomplete. In technical terms, a static analysis technique is complete if it accepts every correct program (and sound if it rejects all incorrect programs, but that's not the main point here). Of course, there...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: When Are Results from Computational Complexity Not Too Coarse?, published by Dalcy on July 4, 2024 on LessWrong. Tl;dr, While an algorithm's computational complexity may be exponential in general (worst-case), it is often possible to stratify its input via some dimension k that makes it polynomial for a fixed k, and only exponential in k. Conceptually, this quantity captures the core aspect of a problem's structure that makes specific instances of it 'harder' than others, often with intuitive interpretations. Example: Bayesian Inference and Treewidth One can easily prove exact inference (decision problem of: "is P(X)>0?") is NP-hard by encoding SAT-3 as a Bayes Net. Showing that it's NP is easy too. Therefore, inference is NP-complete, implying that algorithms are worst-case exponential. But this can't be the typical case! Let's examine the example of a Bayes Net whose structure is a chain ABCD, and say you want to compute the marginals P(D). The Naive Algorithm for marginalization would be to literally multiply all the conditional probability distribution (CPD) tables for each of the Bayes Net's nodes, and sum over all the variables other than X. If we assume each variable has at most v values, then the computational complexity is exponential in the number of variables n. P(D)=ABCP(A,B,C,D), which is O(vn). But because of the factorization P(A,B,C,D)=P(A)P(B|A)P(C|B)P(D|C) due to the chain structure, we can shift the order of the sum around like this: P(D)=CP(D|C)BP(C|B)AP(A)P(B|A) and now the sum can be done in O(nv2). Why? Notice AP(A)P(B|A) is P(B), and to compute P(B=b) we need to multiply v times and sum v1 times, overall O(v). This needs to be done for every b, so O(v2). Now we have cached P(B), and we move on to BP(C|B)P(B), where the same analysis applies. This is basically dynamic programming. So, at least for chains, inference can be done in linear time in input size. The earlier NP-completeness result, remember, is a worst-case analysis that applies to all possible Bayes Nets, ignoring the possible structure in each instance that may make some easier to solve than the other. Let's attempt a more fine complexity analysis by taking into account the structure of the Bayes Net provided, based on the chain example. Intuitively, the relevant structure of the Bayes Net that determines the difficulty of marginalization is the 'degree of interaction' among the variables, since the complexity is exponential in the "maximum number of factors ever seen within a sum," which was 2 in the case of a chain. How do we generalize this quantity to graphs other than chains? Since we could've shuffled the order of the sums and products differently (which would still yield O(nv2) for chains, but for general graphs the exponent may change significantly), for a given graph we want to find the sum-shuffling-order that minimizes the number of factors ever seen within a sum, and call that number k, an invariant of the graph that captures the difficulty of inference - O(mvk)[1] This is a graphical quantity of your graph called treewidth[2][3]. So, to sum up: We've parameterized the possible input Bayes Nets using some quantity k. Where k stratifies the inference problem in terms of their inherent difficulty, i.e. computational complexity is exponential in k, but linear under fixed or bounded k. We see that k is actually a graphical quantity known as treewidth, which intuitively corresponds to the notion of 'degree of interaction' among variables. General Lesson While I was studying basic computational complexity theory, I found myself skeptical of the value of various complexity classes, especially due to the classes being too coarse and not particularly exploiting the structures specific to the problem instance: The motif of proving NP-hardness by finding a clever way to encode 3-SA...