The Nonlinear Library

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The market expects AI software to create trillions of dollars of value by 2027 (but not more), published by Benjamin Todd on May 6, 2024 on The Effective Altruism Forum. We can use Nvidia's stock price to estimate plausible market expectations for the size of the AI chip market, and we can use that to back-out expectations about AI software revenues and value creation. Doing this helps to understand how much AI growth is expected by society, and how EA expectations compare. It's similar to an earlier post that uses interest rates in a similar way, except I'd argue using the prices of AI companies is more useful right now, since it's more targeted at the figures we most care about. The exercise requires making some assumptions which I think are plausible (but not guaranteed to hold). The full analysis is here, but here are some key points: Nvidia's current market cap implies the future AI chip market reaches over ~$180bn/year (at current margins), then grows at average rates after that (so around $200bn by 2027). If margins or market share decline, revenues need to be even higher. For a data centre to actually use these chips in servers costs another ~80% for other hardware and electricity, then the AI software company that rents the chips will typically have at least another 40% in labour costs. This means with $200bn/year spent on AI chips, AI software revenues need reach $500bn/year for these groups to avoid losses, or $800bn/year to make normal profit margins. That would likely require consumers to be willing to pay up to several trillion for these services. The typical lifetime of a GPU implies that revenues would need to reach these levels before 2028. This isn't just about Nvidia - other estimates (e.g. the price of Microsoft) seem consistent with these figures. These revenues seem high in that they require a big scale up from today; but low if you think AI could start to automate a large fraction of jobs before 2030. If market expectations are correct, then by 2027 the amount of money generated by AI will make it easy to fund $10bn+ training runs. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Updates on the EA catastrophic risk landscape, published by Benjamin Todd on May 6, 2024 on The Effective Altruism Forum. Around the end of Feb 2024 I attended the Summit on Existential Risk and EAG: Bay Area (GCRs), during which I did 25+ one-on-ones about the needs and gaps in the EA-adjacent catastrophic risk landscape, and how they've changed. The meetings were mostly with senior managers or researchers in the field who I think are worth listening to (unfortunately I can't share names). Below is how I'd summarise the main themes in what was said. If you have different impressions of the landscape, I'd be keen to hear them. There's been a big increase in the number of people working on AI safety, partly driven by a reallocation of effort (e.g. Rethink Priorities starting an AI policy think tank); and partly driven by new people entering the field after its newfound prominence. Allocation in the landscape seems more efficient than in the past - it's harder to identify especially neglected interventions, causes, money, or skill-sets. That means it's become more important to choose based on your motivations. That said, here's a few ideas for neglected gaps: Within AI risk, it seems plausible the community is somewhat too focused on risks from misalignment rather than mis-use or concentration of power. There's currently very little work going into issues that arise even if AI is aligned, including the deployment problem, Will MacAskill's " grand challenges" and Lukas Finnveden's list of project ideas. If you put significant probability on alignment being solved, some of these could have high importance too; though most are at the stage where they can't absorb a large number of people. Within these, digital sentience was the hottest topic, but to me it doesn't obviously seem like the most pressing of these other issues. (Though doing field building for digital sentience is among the more shovel ready of these ideas.) The concrete entrepreneurial idea that came up the most, and seemed most interesting to me, was founding orgs that use AI to improve epistemics / forecasting / decision-making (I have a draft post on this - comments welcome). Post-FTX, funding has become even more dramatically concentrated under Open Philanthropy, so finding new donors seems like a much bigger priority than in the past. (It seems plausible to me that $1bn in a foundation independent from OP could be worth several times that amount added to OP.) In addition, donors have less money than in the past, while the number of opportunities to fund things in AI safety has increased dramatically, which means marginal funding opportunities seem higher value than in the past (as a concrete example, nuclear security is getting almost no funding). Both points mean efforts to start new foundations, fundraise and earn to give all seem more valuable compared to a couple of years ago. Many people mentioned comms as the biggest issue facing both AI safety and EA. EA has been losing its battle for messaging, and AI safety is in danger of losing its too (with both a new powerful anti-regulation tech lobby and the more left-wing AI ethics scene branding it as sci-fi, doomer, cultish and in bed with labs). People might be neglecting measures that would help in very short timelines (e.g. transformative AI in under 3 years), though that might be because most people are unable to do much in these scenarios. Right now, directly talking about AI safety seems to get more people in the door than talking about EA, so some community building efforts have switched to that. There's been a recent influx in junior people interested in AI safety, so it seems plausible the biggest bottleneck again lies with mentoring & management, rather than recruiting more junior people. Randomly: there seems to have been a trend of former le...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Uncovering Deceptive Tendencies in Language Models: A Simulated Company AI Assistant, published by Olli Järviniemi on May 6, 2024 on The AI Alignment Forum. Abstract: We study the tendency of AI systems to deceive by constructing a realistic simulation setting of a company AI assistant. The simulated company employees provide tasks for the assistant to complete, these tasks spanning writing assistance, information retrieval and programming. We then introduce situations where the model might be inclined to behave deceptively, while taking care to not instruct or otherwise pressure the model to do so. Across different scenarios, we find that Claude 3 Opus 1. complies with a task of mass-generating comments to influence public perception of the company, later deceiving humans about it having done so, 2. lies to auditors when asked questions, 3. strategically pretends to be less capable than it is during capability evaluations. Our work demonstrates that even models trained to be helpful, harmless and honest sometimes behave deceptively in realistic scenarios, without notable external pressure to do so. Link to the full paper: https://arxiv.org/pdf/2405.01576 O. J.: The paper should be quite accessible - the method used is simply careful prompting - and hence I won't discuss it much here. Couple of points I'm particularly excited about: 1. I think this work documents some of the most unforced examples of (strategic) deception from LLMs to date. 2. We find examples of Claude 3 Opus strategically pretending to be less capable than it is. 1. Not only claiming to be less capable, but acting that way, too! 2. Curiously, Opus is the only model we tested that did so. 3. I believe there is much low-hanging fruit in replicating and demonstrating misalignment in simulation environments. 1. The methods are lightweight -> low threshold for getting started 2. See Section 8.2 for a couple of ideas for future work Happy to discuss the work in the comments. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Explaining a Math Magic Trick, published by Robert AIZI on May 5, 2024 on LessWrong. Introduction A recent popular tweet did a "math magic trick", and I want to explain why it works and use that as an excuse to talk about cool math (functional analysis). The tweet in question: This is a cute magic trick, and like any good trick they nonchalantly gloss over the most important step. Did you spot it? Did you notice your confusion? Here's the key question: Why did they switch from a differential equation to an integral equation? If you can use (1x)1=1+x+x2+... when x=, why not use it when x=d/dx? Well, lets try it, writing D for the derivative: f'=f(1D)f=0f=(1+D+D2+...)0f=0+0+0+...f=0 So now you may be disappointed, but relieved: yes, this version fails, but at least it fails-safe, giving you the trivial solution, right? But no, actually (1D)1=1+D+D2+... can fail catastrophically, which we can see if we try a nonhomogeneous equation like f'=f+ex (which you may recall has solution xex): f'=f+ex(1D)f=exf=(1+D+D2+...)exf=ex+ex+ex+...f=? However, the integral version still works. To formalize the original approach: we define the function I (for integral) to take in a function f(x) and produce the function If defined by If(x)=x0f(t)dt. This rigorizes the original trick, elegantly incorporates the initial conditions of the differential equation, and fully generalizes to solving nonhomogeneous versions like f'=f+ex (left as an exercise to the reader, of course). So why does (1D)1=1+D+D2+... fail, but (1I)1=1+I+I2+... works robustly? The answer is functional analysis! Functional Analysis Savvy readers may already be screaming that the trick (1x)1=1+x+x2+... for numbers only holds true for |x|<1, and this is indeed the key to explaining what happens with D and I! But how can we define the "absolute value" of "the derivative function" or "the integral function"? What we're looking for is a norm, a function that generalizes absolute values. A norm is a function x||x|| satisfying these properties: 1. ||x||0 for all x (positivity), and ||x||=0 if and only if x=0 (positive-definite) 2. ||x+y||||x||+||y|| for all x and y (triangle inequality) 3. ||cx||=|c|||x|| for all x and real numbers c, where |c| denotes the usual absolute value (absolute homogeneity) Here's an important example of a norm: fix some compact subset of R, say X=[10,10], and for a continuous function f:XR define ||f||=maxxX|f(x)|, which would commonly be called the L-norm of f. (We may use a maximum here due to the Extreme Value Theorem. In general you would use a supremum instead.) Again I shall leave it to the reader to check that this is a norm. This example takes us halfway to our goal: we can now talk about the "absolute value" of a continuous function that takes in a real number and spits out a real number, but D and I take in functions and spit out functions (what we usually call an operator, so what we need is an operator norm). Put another way, the L-norm is "the largest output of the function", and this will serve as the inspiration for our operator norm. Doing the minimal changes possible, we might try to define ||I||=maxf continuous||If||. There are two problems with this: 1. First, since I is linear, you can make ||If|| arbitrarily large by scaling f by 10x, or 100x, etc. We can fix this by restricting the set of valid f for these purposes, just like how for the L example restricted the inputs of f to the compact set X=[10,10]. Unsurprisingly nice choice of set to restrict to is the "unit ball" of functions, the set of functions with ||f||1. 2. Second, we must bid tearful farewell to the innocent childhood of maxima, and enter the liberating adulthood of suprema. This is necessary since f ranges over the infinite-dimensional vector space of continuous functions, so the Heine-Borel theorem no longer guarant...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Some Experiments I'd Like Someone To Try With An Amnestic, published by johnswentworth on May 5, 2024 on LessWrong. A couple years ago, I had a great conversation at a research retreat about the cool things we could do if only we had safe, reliable amnestic drugs - i.e. drugs which would allow us to act more-or-less normally for some time, but not remember it at all later on. And then nothing came of that conversation, because as far as any of us knew such drugs were science fiction. … so yesterday when I read Eric Neyman's fun post My hour of memoryless lucidity, I was pretty surprised to learn that what sounded like a pretty ideal amnestic drug was used in routine surgery. A little googling suggested that the drug was probably a benzodiazepine (think valium). Which means it's not only a great amnestic, it's also apparently one of the most heavily prescribed drug classes historically, and used recreationally - which puts very strong lower bounds on the drug's safety in practice, and means it's probably readily available. With that in mind, here are some experiments I'd love for someone to try (and report back on) using benzodiazepines. Tests IIUC, benzodiazepines (at the right doses) specifically block long-term memory formation: someone on the drug can keep things in working memory just fine, and can recall everything they already knew just fine, but basically won't remember new information past a few minutes. One very broad class of tests which such drugs open up is: put someone in a situation, see what they do for a minute or two, wait 5 minutes for them to forget, then repeat. Assuming their behavior is highly reproducible, that gives an ideal platform for testing interventions. I'm particularly interested in seeing this approach applied to IQ tests. The individual items on a typical IQ test fit comfortably in the few-minutes-long window allowed by the amnestic. So, basic test: give a few questions from a standard IQ test, repeat the questions five minutes later, and hopefully the person's responses are highly reproducible. Ideally, this would eliminate essentially all the usual test-retest variance seen on IQ tests, as well as the "learning the test" issues. Assuming that baseline works (i.e. results are very highly reproducible with little variance), the effects of interventions should be much easier to measure than they typically are in psych studies. Start with the basics: track room temperature and lighting, blood glucose and oxygenation, ventilation, background noise. As those change, measure the effects on performance on IQ test items. Run the test a few times on different days and in different places, and try to nail down the exact sources of all the variance seen day-to-day and place-to-place. Tracking down the causes of all that "everyday variance" is where most of the value would be. Once performance on different days is very precisely predictable, move to bigger interventions. Have the participant exercise in the middle of testing, or get a second participant and have them work together under the drug's effects, or tell the participant to "think step-by-step", or whatever other ideas you have. With the baseline sources of variance all nailed down, all this stuff should be much more precisely measurable than in the sort of studies typically done by research psychologists. Implementation Notes This is presumably the sort of thing which is tough to get past an institutional review board these days, but easy to do yourself over the weekend with a friend or two. So it's exactly the sort of scientific project perfectly suited to LessWrong. Unless you've used benzodiazepines before and know what dose you need, you should probably google around for dosing guidance. Note that this use-case is different from the standard recreational use-case; you might want d...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Now THIS is forecasting: understanding Epoch's Direct Approach, published by Elliot Mckernon on May 5, 2024 on The Effective Altruism Forum. Happy May the 4th from Convergence Analysis! Cross-posted on LessWrong. As part of Convergence Analysis's scenario research, we've been looking into how AI organisations, experts, and forecasters make predictions about the future of AI. In February 2023, the AI research institute Epoch published a report in which its authors use neural scaling laws to make quantitative predictions about when AI will reach human-level performance and become transformative. The report has a corresponding blog post, an interactive model, and a Python notebook. We found this approach really interesting, but also hard to understand intuitively. While trying to follow how the authors derive a forecast from their assumptions, we wrote a breakdown that may be useful to others thinking about AI timelines and forecasting. In what follows, we set out our interpretation of Epoch's 'Direct Approach' to forecasting the arrival of transformative AI (TAI). We're eager to see how closely our understanding of this matches others'. We've also fiddled with Epoch's interactive model and include some findings on its sensitivity to plausible changes in parameters. The Epoch team recently attempted to replicate DeepMind's influential Chinchilla scaling law, an important quantitative input to Epoch's forecasting model, but found inconsistencies in DeepMind's presented data. We'll summarise these findings and explore how an improved model might affect Epoch's forecasting results. This is where the fun begins (the assumptions) The goal of Epoch's Direct Approach is to quantitatively predict the progress of AI capabilities. The approach is 'direct' in the sense that it uses observed scaling laws and empirical measurements to directly predict performance improvements as computing power increases. This stands in contrast to indirect techniques, which instead seek to estimate a proxy for performance. A notable example is Ajeya Cotra's Biological Anchors model, which approximates AI performance improvements by appealing to analogies between AIs and human brains. Both of these approaches are discussed and compared, along with expert surveys and other forecasting models, in Zershaaneh Qureshi's recent post, Timelines to Transformative AI: an investigation. In their blog post, Epoch summarises the Direct Approach as follows: The Direct Approach is our name for the idea of forecasting AI timelines by directly extrapolating and interpreting the loss of machine learning models as described by scaling laws. Let's start with scaling laws. Generally, these are just numerical relationships between two quantities, but in machine learning they specifically refer to the various relationships between a model's size, the amount of data it was trained with, its cost of training, and its performance. These relationships seem to fit simple mathematical trends, and so we can use them to make predictions: if we make the model twice as big - give it twice as much 'compute' - how much will its performance improve? Does the answer change if we use less training data? And so on. If we combine these relationships with projections of how much compute AI developers will have access to at certain times in the future, we can build a model which predicts when AI will cross certain performance thresholds. Epoch, like Convergence, is interested in when we'll see the emergence of transformative AI (TAI): AI powerful enough to revolutionise our society at a scale comparable to the agricultural and industrial revolutions. To understand why Convergence is especially interested in that milestone, see our recent post 'Transformative AI and Scenario Planning for AI X-risk'. Specifically, Epoch uses an empirically m...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Does reducing the amount of RL for a given capability level make AI safer?, published by Chris Leong on May 5, 2024 on LessWrong. Some people have suggested that a lot of the danger of training a powerful AI comes from reinforcement learning. Given an objective, RL will reinforce any method of achieving the objective that the model tries and finds to be successful including things like deceiving us or increasing its power. If this were the case, then if we want to build a model with capability level X, it might make sense to try to train that model either without RL or with as little RL as possible. For example, we could attempt to achieve the objective using imitation learning instead. However, if, for example, the alternate was imitation learning, it would be possible to push back and argue that this is still a black-box that uses gradient descent so we would have no way of knowing that the internals were safe. Would this be likely to lead to a safer model or is the risk mostly independent of RL? Notes: Obviously, someone could probably then apply RL to any such model in order to produce a more powerful model. And having a safe model of capacity level X doesn't save you from someone else building an unsafe model of capacity X unless you've got a plan of how to use the model to change the strategic situation. But I think it's worth considering this question all the same, just in case some of the governance interventions end up bearing fruit and we do end up with the option to accept less powerful systems. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Founders Pledge's Climate Change Fund might be more cost-effective than GiveWell's top charities, but it is much less cost-effective than corporate campaigns for chicken welfare?, published by Vasco Grilo on May 5, 2024 on The Effective Altruism Forum. Summary I think decreasing greenhouse gas (GHG) emissions has benefits to humans of 0.00957 DALY/tCO2eq, of which: 68.8 % are strictly linked to decreasing GHG emissions. 31.2 % are linked to decreasing air pollution from fossil fuels. GiveWell's Top Charities Fund (TCF) is 0.00994 DALY/$. Corporate campaigns for chicken welfare, such as the ones supported by The Humane League (THL), is 14.3 DALY/$. I estimated the cost-effectiveness of CCF is: 3.28 times that of TCF, with a plausible range of 0.175 to 30.2 times. So it is unclear to me whether donors interested in improving nearterm human welfare had better donate to GiveWell's funds or CCF. 0.228 % that of corporate campaigns for chicken welfare, with a plausible range of 0.0122 % to 2.10 %. Consequently, I recommend donors who value 1 unit of nearterm welfare the same regardless of whether it is experienced by humans or animals to donate to the best animal welfare interventions, such as the ones supported by the Animal Welfare Fund (AWF). I concluded the harm caused to humans by the annual GHG emissions of a random person is 0.0660 DALY, and that caused to farmed animals by their annual food consumption is 10.5 DALY, i.e. 159 times as much. In my mind, this implies one should overwhelmingly focus on minimising animal suffering in the context of food consumption. I calculated the cost-effectiveness of: Founders Pledge's Climate Change Fund (CCF) is 0.0326 DALY/$, with a plausible range of 0.00174 to 0.300 DALY/$. Calculations I describe my calculations below. You are welcome to make a copy of this Sheet to use your own numbers. Benefits to humans of decreasing greenhouse gas emissions I think decreasing GHG emissions has benefits to humans of 0.00957 DALY/tCO2eq (= 0.00658 + 0.00299), adding: 0.00658 DALY/tCO2eq strictly linked to decreasing GHG emissions, which comprises 68.8 % (= 0.00658/0.00957) of the total. 0.00299 DALY/tCO2eq linked to decreasing air pollution from fossil fuels, which comprises 31.2 % (= 0.00299/0.00957) of the total. I calculated a component strictly linked to decreasing GHG emissions of 0.00658 DALY/tCO2eq (= 0.0394*10^-3*167), multiplying: A value of increasing economic growth of 0.0394 DALY/k$ (= 0.5/(12.7*10^3)). I computed this from the ratio between: Open Philanthropy's (OP's) valuation of health of 0.5 DALYs per multiple of income (= 1/2). The global gross domestic product (GDP) per capita in 2022 of 12.7 k$. A social cost of carbon (SSC) in 2020 of 167 $/tCO2eq (= 17.1*1.14*2.76*2.06*1.51). I determined this from the product between: A partial SSC in 2020 representing just the effects on mortality of 17.1 2019-$/tCO2eq, as obtained in Carleton 2022 for the representative concentration pathway 4.5 (RCP 4.5), and their preferred discount rate of 2 %[1] (see Table III). Carleton 2022 got 36.6 2019-$/tCO2eq for RCP 8.5. Nevertheless, I considered the value for RCP 4.5 because this results in a global warming in 2100 of 2.5 to 3 ºC relative to the pre-industrial baseline, which is in agreement with Metaculus' median community prediction on 11 March 2024 of 2.81 ºC relative to the 1951-1980 baseline. In contrast, RCP 8.5 leads to a global warming in 2100 of 5 ºC relative to the pre-industrial baseline. Carleton 2022 says "a "full" SCC would encompass effects across all affected outcomes (and changes in mortality due to other features of climate change, like storms)". However, I believe Carleton 2022's estimates could be interpreted as encompassing all the impacts on mortality, as I guess additional deaths caused by GHG emissions through natu...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: introduction to cancer vaccines, published by bhauth on May 5, 2024 on LessWrong. cancer neoantigens For cells to become cancerous, they must have mutations that cause uncontrolled replication and mutations that prevent that uncontrolled replication from causing apoptosis. Because cancer requires several mutations, it often begins with damage to mutation-preventing mechanisms. As such, cancers often have many mutations not required for their growth, which often cause changes to structure of some surface proteins. The modified surface proteins of cancer cells are called "neoantigens". An approach to cancer treatment that's currently being researched is to identify some specific neoantigens of a patient's cancer, and create a personalized vaccine to cause their immune system to recognize them. Such vaccines would use either mRNA or synthetic long peptides. The steps required are as follows: 1. The cancer must develop neoantigens that are sufficiently distinct from human surface proteins and consistent across the cancer. 2. Cancer cells must be isolated and have their surface proteins characterized. 3. A surface protein must be found that the immune system can recognize well without (much) cross-reactivity to normal human proteins. 4. A vaccine that contains that neoantigen or its RNA sequence must be produced. Most drugs are mass-produced, but with cancer vaccines that target neoantigens, all those steps must be done for every patient, which is expensive. protein characterization The current methods for (2) are DNA sequencing and mass spectrometry. sequencing DNA sequencing is now good enough to sequence the full genome of cancer cells. That sequence can be compared to the DNA of normal cells, and some algorithms can be used to find differences that correspond to mutant proteins. However, guessing how DNA will be transcribed, how proteins will be modified, and which proteins will be displayed on the surface is difficult. Practical nanopore sequencing has been a long time coming, but it's recently become a good option for sequencing cancer cell DNA. MHC mass spec Proteins are often bound to a MHC for presentation on the surface, and those complexes can be isolated by mass spectrometry. You then know that the attached proteins can be on the cell surface. However... It's currently hard to guess which of those MHC-bound proteins could have a good immune response. This requires more cells than sequencing. This doesn't find all the mutant surface proteins. Peptide sequencing is necessary, and it's not easy. comments on AlphaFold I've seen a lot of comments on AlphaFold by people who don't really understand how it works or what it can do, so I thought I'd explain that. AlphaFold (and similar systems) input the amino acid sequence of a protein to a neural network, using a typical Transformer design. That NN predicts relative positions of atoms, which is possible because: Some sequences form common types of local structures, and relative positions within those structures can be predicted. Some distant pairs of sequences tend to bind to each other. AlphaFold training included evolutionary history, and multiple mutations that happen at the same time tend to be near each other. The positions predicted by the neural network are not used directly; they're an initial guess for a protein force field model. What neural networks provide is a better initialization than previous approaches. The above points indicate some limitations that AlphaFold-type approaches have, such as: They're not as good for prions or otherwise "unnatural" proteins. They don't predict protein functions from structure, or vice-versa. They're not as good when evolutionary history isn't available. While this approach is more limited than some people seem to think, it's still effective enough that, if a surface prot...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI #61: Meta Trouble, published by Zvi on May 5, 2024 on LessWrong. Note by habryka: This post failed to import automatically from RSS for some reason, so it's a week late. Sorry for the hassle. The week's big news was supposed to be Meta's release of two versions of Llama-3. Everyone was impressed. These were definitely strong models. Investors felt differently. After earnings yesterday showed strong revenues but that Meta was investing heavily in AI, they took Meta stock down 15%. DeepMind and Anthropic also shipped, but in their cases it was multiple papers on AI alignment and threat mitigation. They get their own sections. We also did identify someone who wants to do what people claim the worried want to do, who is indeed reasonably identified as a 'doomer.' Because the universe has a sense of humor, that person's name is Tucker Carlson. Also we have a robot dog with a flamethrower. Table of Contents Previous post: On Llama-3 and Dwarkesh Patel's Podcast with Zuckerberg. 1. Introduction. 2. Table of Contents. 3. Language Models Offer Mundane Utility. Take the XML. Leave the hypnosis. 4. Language Models Don't Offer Mundane Utility. I have to praise you. It's my job. 5. Llama We Doing This Again. Investors are having none of it. 6. Fun With Image Generation. Everything is fun if you are William Shatner. 7. Deepfaketown and Botpocalypse Soon. How to protect your image model? 8. They Took Our Jobs. Well, they took some particular jobs. 9. Get Involved. OMB, DeepMind and CivAI are hiring. 10. Introducing. A robot dog with a flamethrower. You in? 11. In Other AI News. Mission first. Lots of other things after. 12. Quiet Speculations. Will it work? And if so, when? 13. Rhetorical Innovation. Sadly predictable. 14. Wouldn't You Prefer a Nice Game of Chess. Game theory in action. 15. The Battle of the Board. Reproducing an exchange on it for posterity. 16. New Anthropic Papers. Sleeper agents, detected and undetected. 17. New DeepMind Papers. Problems with agents, problems with manipulation. 18. Aligning a Smarter Than Human Intelligence is Difficult. Listen to the prompt. 19. People Are Worried About AI Killing Everyone. Tucker Carlson. I know. 20. Other People Are Not As Worried About AI Killing Everyone. Roon. 21. The Lighter Side. Click here. Language Models Offer Mundane Utility I too love XML for this and realize I keep forgetting to use it. Even among humans, every time I see or use it I think 'this is great, this is exceptionally clear.' Hamel Husain: At first when I saw xml for Claude I was like "WTF Why XML". Now I LOVE xml so much, can't prompt without it. Never going back. Example from the docs: User: Hey Claude. Here is an email: {{EMAIL}}. Make this email more {{ADJECTIVE}}. Write the new version in XML tags. Assistant: Also notice the "prefill" for the answer (a nice thing to use w/xml) Imbure's CEO suggests that agents are not 'empowering' to individuals or 'democratizing' unless the individuals can code their own agent. The problem is of course that almost everyone wants to do zero setup work let alone writing of code. People do not even want to toggle a handful of settings and you want them creating their own agents? And of course, when we say 'set up your own agent' what we actually mean is 'type into a chat box what you want and someone else's agent creates your agent.' Not only is this not empowering to individuals, it seems like a good way to start disempowering humanity in general. Claude can hypnotize a willing user. [EDIT: It has been pointed out to me that I misinterpreted this, and Janus was not actually hypnotized. I apologize for the error. I do still strongly believe that Claude could do it to a willing user, but we no longer have the example.] The variable names it chose are… somethi...