The Nonlinear Library: LessWrong

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How do open AI models affect incentive to race?, published by jessicata on May 7, 2024 on LessWrong. I see it said sometimes that open models contribute to AI race dynamics. My guess is that they don't, and if anything, reduce AI race dynamics. I will consider a simplified model that only takes into account the cost of training a model, not the cost to deploy it (which tends to be small relative to revenue anyway). Let f(x) map a training expense x to a "value per day per customer" of the trained model, under the assumption that the training makes efficient use of the cost. That is, a customer values using an AI model trained with x compute at $f(x) per day. I assume there are n identical customers here; of course, there are complexities where some customers value AI more than others, incentivizing price discrimination, but I'm abstracting this consideration out. (In general, variation in how much customers value a product will tend to increase consumer surplus while reducing revenue, as it makes it harder to charge customers just under the maximum amount they're willing to pay.) I'm also assuming there is only one company that trains closed models for profit. This assumption is flawed because there is competition between different companies that train closed models. However, perfect competition assumptions would tend to reduce the incentive to train models. Suppose two companies have closed models of equivalent expense x. They each want to charge slightly less than the minimum of f(x) and the competitor's price, per customer per day. If each competitor undercuts the other slightly, the cost will approach 0. See the Traveler's Dilemma for a comparison. The reasons why this doesn't happen have to do with considerations like differences in models' performance on different tasks, e.g. some models are better for programming than others. If models are sufficiently specialized (allowing this sort of niche-monopolization), each specialized type of model can be modeled independently as a monopoly. So I'll analyze the case of a closed model monopoly, noting that translation to the real world is more complex. Suppose the best open model has compute x and a company trains a closed model with compute y > x. Each customer will now spend up to f(y) - f(x) per day for the model; I'll assume the company charges f(y) - f(x) and the customers purchase this, noting that they could charge just below this amount to create a positive incentive for customers. So the company's revenue over m days is nm(f(y) - f(x)). Clearly, this is decreasing in x. So the better the open model is, the less expected revenue there is from training a closed model. But this is simply comparing doing nothing to training a model of a fixed cost y. So consider instead comparing expected revenue between two different model costs, y and z, both greater than x. The revenue from y is nm(f(y) - f(x)), and from z it is nm(f(z) - f(x)). The difference between the z revenue and the y revenue is nm(f(z) - f(y)). This is unaffected by x. This can model a case where the company has already trained a model of cost y and is considering upgrading to z. In this case, the open model doesn't affect the expected additional revenue from the upgrade. Things get more complex when we assume there will be a future improvement to the open model. Suppose that, for k days, the open model has training cost x, and for the remaining m-k days, it has training cost x' > x. Now suppose that the closed AI company has already trained a model of cost y, where x < y < x'. They are considering upgrading to a model of cost z, where z > x'. Suppose they do not upgrade. Then they get nk(f(y) - f(x)) revenue from the first k days and nothing thereafter. Suppose they do upgrade, immediately. Then they get nk(f(z) - f(x)) revenue from the first k days, an...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Rapid capability gain around supergenius level seems probable even without intelligence needing to improve intelligence, published by Towards Keeperhood on May 6, 2024 on LessWrong. TLDR: 1. Around Einstein-level, relatively small changes in intelligence can lead to large changes in what one is capable to accomplish. 1. E.g. Einstein was a bit better than the other best physi at seeing deep connections and reasoning, but was able to accomplish much more in terms of impressive scientific output. 2. There are architectures where small changes can have significant effects on intelligence. 1. E.g. small changes in human-brain-hyperparameters: Einstein's brain didn't need to be trained on 3x the compute than normal physics professors for him to become much better at forming deep understanding, even without intelligence improving intelligence. Einstein and the heavytail of human intelligence 1905 is often described as the "annus mirabilis" of Albert Einstein. He founded quantum physics by postulating the existence of (light) quanta, explained Brownian motion, introduced the special relativity theory and derived E=mc from it. All of this. In one year. While having a full-time job in the Swiss patent office. With the exception of John von Neumann, we'd say those discoveries alone seem more than any other scientist of the 20th century achieved in their lifetime (though it's debatable). Though perhaps even more impressive is that Einstein was able to derive general relativity. Einstein was often so far ahead of his time that even years after he published his theories the majority of physicists rejected them because they couldn't understand them, sometimes even though there was experimental evidence favoring Einstein's theories. After solving the greatest open physics problems at the time in 1905, he continued working in the patent office until 1908, since the universities were too slow on the uptake to hire him earlier. Example for how far ahead of his time Einstein was: Deriving the theory of light quanta The following section is based on parts of the 8th chapter of "Surfaces and Essences" by Douglas Hofstadter. For an analysis of some of Einstein's discoveries, which show how far ahead of his time he was, I can recommend reading it. At the time, one of the biggest problems in physics was the "Blackbody spectrum", which describes the spectrum of electromagnetic wavelengths emitted by a Blackbody. The problem with it was that the emitted spectrum was not explainable by known physics. Einstein achieved a breakthrough by considering light not just as a wave, but also as light quanta. Although this idea sufficiently explained the Blackbody spectrum, physicists (at least almost) unanimously rejected it. The fight between the "light is corpuscles" and "light is a wave" faction had been decided a century ago, with a clear victory for the "wave" faction. Being aware of these possible doubts, Einstein proposed three experiments to prove his idea, one of which was the photoelectric effect. In the following years, Robert Millikan carried out various experiments on the photoelectric effect, which all confirmed Einstein's predictions. Still, Millikan insisted that the light-quanta theory had no theoretical basis and even falsely claimed that Einstein himself did not believe in his idea anymore. From Surfaces and Essences (p.611): To add insult to injury, although the 1921 Nobel Prize in Physics was awarded to Albert Einstein, it was not for his theory of light quanta but "for his discovery of the law of the photoelectric effect". Weirdly, in the citation there was no mention of the ideas behind that law, since no one on the Nobel Committee (or in all of physics) believed in them! [1][...] And thus Albert Einstein's revolutionary ideas on the nature of light, that most fundamental and all-...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Explaining a Math Magic Trick, published by Robert AIZI on May 5, 2024 on LessWrong. Introduction A recent popular tweet did a "math magic trick", and I want to explain why it works and use that as an excuse to talk about cool math (functional analysis). The tweet in question: This is a cute magic trick, and like any good trick they nonchalantly gloss over the most important step. Did you spot it? Did you notice your confusion? Here's the key question: Why did they switch from a differential equation to an integral equation? If you can use (1x)1=1+x+x2+... when x=, why not use it when x=d/dx? Well, lets try it, writing D for the derivative: f'=f(1D)f=0f=(1+D+D2+...)0f=0+0+0+...f=0 So now you may be disappointed, but relieved: yes, this version fails, but at least it fails-safe, giving you the trivial solution, right? But no, actually (1D)1=1+D+D2+... can fail catastrophically, which we can see if we try a nonhomogeneous equation like f'=f+ex (which you may recall has solution xex): f'=f+ex(1D)f=exf=(1+D+D2+...)exf=ex+ex+ex+...f=? However, the integral version still works. To formalize the original approach: we define the function I (for integral) to take in a function f(x) and produce the function If defined by If(x)=x0f(t)dt. This rigorizes the original trick, elegantly incorporates the initial conditions of the differential equation, and fully generalizes to solving nonhomogeneous versions like f'=f+ex (left as an exercise to the reader, of course). So why does (1D)1=1+D+D2+... fail, but (1I)1=1+I+I2+... works robustly? The answer is functional analysis! Functional Analysis Savvy readers may already be screaming that the trick (1x)1=1+x+x2+... for numbers only holds true for |x|<1, and this is indeed the key to explaining what happens with D and I! But how can we define the "absolute value" of "the derivative function" or "the integral function"? What we're looking for is a norm, a function that generalizes absolute values. A norm is a function x||x|| satisfying these properties: 1. ||x||0 for all x (positivity), and ||x||=0 if and only if x=0 (positive-definite) 2. ||x+y||||x||+||y|| for all x and y (triangle inequality) 3. ||cx||=|c|||x|| for all x and real numbers c, where |c| denotes the usual absolute value (absolute homogeneity) Here's an important example of a norm: fix some compact subset of R, say X=[10,10], and for a continuous function f:XR define ||f||=maxxX|f(x)|, which would commonly be called the L-norm of f. (We may use a maximum here due to the Extreme Value Theorem. In general you would use a supremum instead.) Again I shall leave it to the reader to check that this is a norm. This example takes us halfway to our goal: we can now talk about the "absolute value" of a continuous function that takes in a real number and spits out a real number, but D and I take in functions and spit out functions (what we usually call an operator, so what we need is an operator norm). Put another way, the L-norm is "the largest output of the function", and this will serve as the inspiration for our operator norm. Doing the minimal changes possible, we might try to define ||I||=maxf continuous||If||. There are two problems with this: 1. First, since I is linear, you can make ||If|| arbitrarily large by scaling f by 10x, or 100x, etc. We can fix this by restricting the set of valid f for these purposes, just like how for the L example restricted the inputs of f to the compact set X=[10,10]. Unsurprisingly nice choice of set to restrict to is the "unit ball" of functions, the set of functions with ||f||1. 2. Second, we must bid tearful farewell to the innocent childhood of maxima, and enter the liberating adulthood of suprema. This is necessary since f ranges over the infinite-dimensional vector space of continuous functions, so the Heine-Borel theorem no longer guarant...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Some Experiments I'd Like Someone To Try With An Amnestic, published by johnswentworth on May 5, 2024 on LessWrong. A couple years ago, I had a great conversation at a research retreat about the cool things we could do if only we had safe, reliable amnestic drugs - i.e. drugs which would allow us to act more-or-less normally for some time, but not remember it at all later on. And then nothing came of that conversation, because as far as any of us knew such drugs were science fiction. … so yesterday when I read Eric Neyman's fun post My hour of memoryless lucidity, I was pretty surprised to learn that what sounded like a pretty ideal amnestic drug was used in routine surgery. A little googling suggested that the drug was probably a benzodiazepine (think valium). Which means it's not only a great amnestic, it's also apparently one of the most heavily prescribed drug classes historically, and used recreationally - which puts very strong lower bounds on the drug's safety in practice, and means it's probably readily available. With that in mind, here are some experiments I'd love for someone to try (and report back on) using benzodiazepines. Tests IIUC, benzodiazepines (at the right doses) specifically block long-term memory formation: someone on the drug can keep things in working memory just fine, and can recall everything they already knew just fine, but basically won't remember new information past a few minutes. One very broad class of tests which such drugs open up is: put someone in a situation, see what they do for a minute or two, wait 5 minutes for them to forget, then repeat. Assuming their behavior is highly reproducible, that gives an ideal platform for testing interventions. I'm particularly interested in seeing this approach applied to IQ tests. The individual items on a typical IQ test fit comfortably in the few-minutes-long window allowed by the amnestic. So, basic test: give a few questions from a standard IQ test, repeat the questions five minutes later, and hopefully the person's responses are highly reproducible. Ideally, this would eliminate essentially all the usual test-retest variance seen on IQ tests, as well as the "learning the test" issues. Assuming that baseline works (i.e. results are very highly reproducible with little variance), the effects of interventions should be much easier to measure than they typically are in psych studies. Start with the basics: track room temperature and lighting, blood glucose and oxygenation, ventilation, background noise. As those change, measure the effects on performance on IQ test items. Run the test a few times on different days and in different places, and try to nail down the exact sources of all the variance seen day-to-day and place-to-place. Tracking down the causes of all that "everyday variance" is where most of the value would be. Once performance on different days is very precisely predictable, move to bigger interventions. Have the participant exercise in the middle of testing, or get a second participant and have them work together under the drug's effects, or tell the participant to "think step-by-step", or whatever other ideas you have. With the baseline sources of variance all nailed down, all this stuff should be much more precisely measurable than in the sort of studies typically done by research psychologists. Implementation Notes This is presumably the sort of thing which is tough to get past an institutional review board these days, but easy to do yourself over the weekend with a friend or two. So it's exactly the sort of scientific project perfectly suited to LessWrong. Unless you've used benzodiazepines before and know what dose you need, you should probably google around for dosing guidance. Note that this use-case is different from the standard recreational use-case; you might want d...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Does reducing the amount of RL for a given capability level make AI safer?, published by Chris Leong on May 5, 2024 on LessWrong. Some people have suggested that a lot of the danger of training a powerful AI comes from reinforcement learning. Given an objective, RL will reinforce any method of achieving the objective that the model tries and finds to be successful including things like deceiving us or increasing its power. If this were the case, then if we want to build a model with capability level X, it might make sense to try to train that model either without RL or with as little RL as possible. For example, we could attempt to achieve the objective using imitation learning instead. However, if, for example, the alternate was imitation learning, it would be possible to push back and argue that this is still a black-box that uses gradient descent so we would have no way of knowing that the internals were safe. Would this be likely to lead to a safer model or is the risk mostly independent of RL? Notes: Obviously, someone could probably then apply RL to any such model in order to produce a more powerful model. And having a safe model of capacity level X doesn't save you from someone else building an unsafe model of capacity X unless you've got a plan of how to use the model to change the strategic situation. But I think it's worth considering this question all the same, just in case some of the governance interventions end up bearing fruit and we do end up with the option to accept less powerful systems. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: introduction to cancer vaccines, published by bhauth on May 5, 2024 on LessWrong. cancer neoantigens For cells to become cancerous, they must have mutations that cause uncontrolled replication and mutations that prevent that uncontrolled replication from causing apoptosis. Because cancer requires several mutations, it often begins with damage to mutation-preventing mechanisms. As such, cancers often have many mutations not required for their growth, which often cause changes to structure of some surface proteins. The modified surface proteins of cancer cells are called "neoantigens". An approach to cancer treatment that's currently being researched is to identify some specific neoantigens of a patient's cancer, and create a personalized vaccine to cause their immune system to recognize them. Such vaccines would use either mRNA or synthetic long peptides. The steps required are as follows: 1. The cancer must develop neoantigens that are sufficiently distinct from human surface proteins and consistent across the cancer. 2. Cancer cells must be isolated and have their surface proteins characterized. 3. A surface protein must be found that the immune system can recognize well without (much) cross-reactivity to normal human proteins. 4. A vaccine that contains that neoantigen or its RNA sequence must be produced. Most drugs are mass-produced, but with cancer vaccines that target neoantigens, all those steps must be done for every patient, which is expensive. protein characterization The current methods for (2) are DNA sequencing and mass spectrometry. sequencing DNA sequencing is now good enough to sequence the full genome of cancer cells. That sequence can be compared to the DNA of normal cells, and some algorithms can be used to find differences that correspond to mutant proteins. However, guessing how DNA will be transcribed, how proteins will be modified, and which proteins will be displayed on the surface is difficult. Practical nanopore sequencing has been a long time coming, but it's recently become a good option for sequencing cancer cell DNA. MHC mass spec Proteins are often bound to a MHC for presentation on the surface, and those complexes can be isolated by mass spectrometry. You then know that the attached proteins can be on the cell surface. However... It's currently hard to guess which of those MHC-bound proteins could have a good immune response. This requires more cells than sequencing. This doesn't find all the mutant surface proteins. Peptide sequencing is necessary, and it's not easy. comments on AlphaFold I've seen a lot of comments on AlphaFold by people who don't really understand how it works or what it can do, so I thought I'd explain that. AlphaFold (and similar systems) input the amino acid sequence of a protein to a neural network, using a typical Transformer design. That NN predicts relative positions of atoms, which is possible because: Some sequences form common types of local structures, and relative positions within those structures can be predicted. Some distant pairs of sequences tend to bind to each other. AlphaFold training included evolutionary history, and multiple mutations that happen at the same time tend to be near each other. The positions predicted by the neural network are not used directly; they're an initial guess for a protein force field model. What neural networks provide is a better initialization than previous approaches. The above points indicate some limitations that AlphaFold-type approaches have, such as: They're not as good for prions or otherwise "unnatural" proteins. They don't predict protein functions from structure, or vice-versa. They're not as good when evolutionary history isn't available. While this approach is more limited than some people seem to think, it's still effective enough that, if a surface prot...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI #61: Meta Trouble, published by Zvi on May 5, 2024 on LessWrong. Note by habryka: This post failed to import automatically from RSS for some reason, so it's a week late. Sorry for the hassle. The week's big news was supposed to be Meta's release of two versions of Llama-3. Everyone was impressed. These were definitely strong models. Investors felt differently. After earnings yesterday showed strong revenues but that Meta was investing heavily in AI, they took Meta stock down 15%. DeepMind and Anthropic also shipped, but in their cases it was multiple papers on AI alignment and threat mitigation. They get their own sections. We also did identify someone who wants to do what people claim the worried want to do, who is indeed reasonably identified as a 'doomer.' Because the universe has a sense of humor, that person's name is Tucker Carlson. Also we have a robot dog with a flamethrower. Table of Contents Previous post: On Llama-3 and Dwarkesh Patel's Podcast with Zuckerberg. 1. Introduction. 2. Table of Contents. 3. Language Models Offer Mundane Utility. Take the XML. Leave the hypnosis. 4. Language Models Don't Offer Mundane Utility. I have to praise you. It's my job. 5. Llama We Doing This Again. Investors are having none of it. 6. Fun With Image Generation. Everything is fun if you are William Shatner. 7. Deepfaketown and Botpocalypse Soon. How to protect your image model? 8. They Took Our Jobs. Well, they took some particular jobs. 9. Get Involved. OMB, DeepMind and CivAI are hiring. 10. Introducing. A robot dog with a flamethrower. You in? 11. In Other AI News. Mission first. Lots of other things after. 12. Quiet Speculations. Will it work? And if so, when? 13. Rhetorical Innovation. Sadly predictable. 14. Wouldn't You Prefer a Nice Game of Chess. Game theory in action. 15. The Battle of the Board. Reproducing an exchange on it for posterity. 16. New Anthropic Papers. Sleeper agents, detected and undetected. 17. New DeepMind Papers. Problems with agents, problems with manipulation. 18. Aligning a Smarter Than Human Intelligence is Difficult. Listen to the prompt. 19. People Are Worried About AI Killing Everyone. Tucker Carlson. I know. 20. Other People Are Not As Worried About AI Killing Everyone. Roon. 21. The Lighter Side. Click here. Language Models Offer Mundane Utility I too love XML for this and realize I keep forgetting to use it. Even among humans, every time I see or use it I think 'this is great, this is exceptionally clear.' Hamel Husain: At first when I saw xml for Claude I was like "WTF Why XML". Now I LOVE xml so much, can't prompt without it. Never going back. Example from the docs: User: Hey Claude. Here is an email: {{EMAIL}}. Make this email more {{ADJECTIVE}}. Write the new version in XML tags. Assistant: Also notice the "prefill" for the answer (a nice thing to use w/xml) Imbure's CEO suggests that agents are not 'empowering' to individuals or 'democratizing' unless the individuals can code their own agent. The problem is of course that almost everyone wants to do zero setup work let alone writing of code. People do not even want to toggle a handful of settings and you want them creating their own agents? And of course, when we say 'set up your own agent' what we actually mean is 'type into a chat box what you want and someone else's agent creates your agent.' Not only is this not empowering to individuals, it seems like a good way to start disempowering humanity in general. Claude can hypnotize a willing user. [EDIT: It has been pointed out to me that I misinterpreted this, and Janus was not actually hypnotized. I apologize for the error. I do still strongly believe that Claude could do it to a willing user, but we no longer have the example.] The variable names it chose are… somethi...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Introducing AI-Powered Audiobooks of Rational Fiction Classics, published by Askwho on May 4, 2024 on LessWrong. (ElevenLabs reading of this post:) I'm excited to share a project I've been working on that I think many in the Lesswrong community will appreciate - converting some rational fiction into high-quality audiobooks using cutting-edge AI voice technology from ElevenLabs, under the name "Askwho Casts AI". The keystone of this project is an audiobook version of Planecrash (AKA Project Lawful), the epic glowfic authored by Eliezer Yudkowsky and Lintamande. Given the scope and scale of this work, with its large cast of characters, I'm using ElevenLabs to give each character their own distinct voice. It's a labor of love to convert this audiobook version of this story, and I hope if anyone has bounced off it before, this might be a more accessible version. Alongside Planecrash, I'm also working on audiobook versions of two other rational fiction favorites: Luminosity by Alicorn (to be followed by its sequel Radiance) Animorphs: The Reckoning by Duncan Sabien I'm also putting out a feed where I convert any articles I find interesting, a lot of which are in the Rat Sphere. My goal with this project is to make some of my personal favorite rational stories more accessible by allowing people to enjoy them in audiobook format. I know how powerful these stories can be, and I want to help bring them to a wider audience and to make them easier for existing fans to re-experience. I wanted to share this here on Lesswrong to connect with others who might find value in these audiobooks. If you're a fan of any of these stories, I'd love to get your thoughts and feedback! And if you know other aspiring rationalists who might enjoy them, please help spread the word. What other classic works of rational fiction would you love to see converted into AI audiobooks? Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Now THIS is forecasting: understanding Epoch's Direct Approach, published by Elliot Mckernon on May 4, 2024 on LessWrong. Happy May the 4th from Convergence Analysis! Cross-posted on the EA Forum. As part of Convergence Analysis's scenario research, we've been looking into how AI organisations, experts, and forecasters make predictions about the future of AI. In February 2023, the AI research institute Epoch published a report in which its authors use neural scaling laws to make quantitative predictions about when AI will reach human-level performance and become transformative. The report has a corresponding blog post, an interactive model, and a Python notebook. We found this approach really interesting, but also hard to understand intuitively. While trying to follow how the authors derive a forecast from their assumptions, we wrote a breakdown that may be useful to others thinking about AI timelines and forecasting. In what follows, we set out our interpretation of Epoch's 'Direct Approach' to forecasting the arrival of transformative AI (TAI). We're eager to see how closely our understanding of this matches others'. We've also fiddled with Epoch's interactive model and include some findings on its sensitivity to plausible changes in parameters. The Epoch team recently attempted to replicate DeepMind's influential Chinchilla scaling law, an important quantitative input to Epoch's forecasting model, but found inconsistencies in DeepMind's presented data. We'll summarise these findings and explore how an improved model might affect Epoch's forecasting results. This is where the fun begins (the assumptions) The goal of Epoch's Direct Approach is to quantitatively predict the progress of AI capabilities. The approach is 'direct' in the sense that it uses observed scaling laws and empirical measurements to directly predict performance improvements as computing power increases. This stands in contrast to indirect techniques, which instead seek to estimate a proxy for performance. A notable example is Ajeya Cotra's Biological Anchors model, which approximates AI performance improvements by appealing to analogies between AIs and human brains. Both of these approaches are discussed and compared, along with expert surveys and other forecasting models, in Zershaaneh Qureshi's recent post, Timelines to Transformative AI: an investigation. In their blog post, Epoch summarises the Direct Approach as follows: The Direct Approach is our name for the idea of forecasting AI timelines by directly extrapolating and interpreting the loss of machine learning models as described by scaling laws. Let's start with scaling laws. Generally, these are just numerical relationships between two quantities, but in machine learning they specifically refer to the various relationships between a model's size, the amount of data it was trained with, its cost of training, and its performance. These relationships seem to fit simple mathematical trends, and so we can use them to make predictions: if we make the model twice as big - give it twice as much 'compute' - how much will its performance improve? Does the answer change if we use less training data? And so on. If we combine these relationships with projections of how much compute AI developers will have access to at certain times in the future, we can build a model which predicts when AI will cross certain performance thresholds. Epoch, like Convergence, is interested in when we'll see the emergence of transformative AI (TAI): AI powerful enough to revolutionise our society at a scale comparable to the agricultural and industrial revolutions. To understand why Convergence is especially interested in that milestone, see our recent post 'Transformative AI and Scenario Planning for AI X-risk'. Specifically, Epoch uses an empirically measured scaling ...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: My hour of memoryless lucidity, published by Eric Neyman on May 4, 2024 on LessWrong. Yesterday, I had a coronectomy: the top halves of my bottom wisdom teeth were surgically removed. It was my first time being sedated, and I didn't know what to expect. While I was unconscious during the surgery, the hour after surgery turned out to be a fascinating experience, because I was completely lucid but had almost zero short-term memory. My girlfriend, who had kindly agreed to accompany me to the surgery, was with me during that hour. And so - apparently against the advice of the nurses - I spent that whole hour talking to her and asking her questions. The biggest reason I find my experience fascinating is that it has mostly answered a question that I've had about myself for quite a long time: how deterministic am I? In computer science, we say that an algorithm is deterministic if it's not random: if it always behaves the same way when it's in the same state. In this case, my "state" was my environment (lying drugged on a bed with my IV in and my girlfriend sitting next to me) plus the contents of my memory. Normally, I don't ask the same question over and over again because the contents of my memory change when I ask the question the first time: after I get an answer, the answer is in my memory, so I don't need to ask the question again. But for that hour, the information I processed came in one ear and out the other in a matter of minutes. And so it was a natural test of whether my memory is the only thing keeping me from saying the same things on loop forever, or whether I'm more random/spontaneous than that.[1] And as it turns out, I'm pretty deterministic! According to my girlfriend, I spent a lot of that hour cycling between the same few questions on loop: "How did the surgery go?" (it went well), "Did they just do a coronectomy or did they take out my whole teeth?" (just a coronectomy), "Is my IV still in?" (yes), "how long was the surgery?" (an hour and a half), "what time is it?", and "how long have you been here?". (The length of that cycle is also interesting, because it gives an estimate of how long I was able to retain memories for - apparently about two minutes.) (Toward the end of that hour, I remember asking, "I know I've already asked this twice, but did they just do a coronectomy?" (The answer: "actually you've asked that much more than twice, and yes, it was just a coronectomy.)) Those weren't my only questions, though. About five minutes into that hour, I apparently asked my girlfriend for two 2-digit numbers to multiply, to check how cognitively impaired I was. She gave me 27*69, and said that I had no trouble doing the multiplication in the obvious way (27*7*10 - 27), except that I kept having to ask her to remind me what the numbers were. Interestingly, I asked her for two 2-digit numbers again toward the end of that hour, having no memory that I had already done this. She told me that she had already given me two numbers, and asked whether I wanted the same numbers again. I said yes (so I could compare my performance). The second time, I was able to do the multiplication pretty quickly without needing to ask for the numbers to be repeated. Also, about 20 minutes into the hour, I asked my girlfriend to give me the letters to that day's New York Times Spelling Bee, which is a puzzle where you're given seven letters and try to form words using the letters. (The letters were W, A, M, O, R, T, and Y.) I found the pangram - the word that uses every letter at least once[2] - in about 30 seconds, which is about average for me, except that yesterday I was holding the letters in my head instead of looking at them on a screen. I also got most of the way to the "genius" rank - a little better than I normally do - and my girlfriend got us the rest of the way ther...