The Nonlinear Library: LessWrong

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Unit economics of LLM APIs, published by dschwarz on August 28, 2024 on LessWrong. Disclaimer 1: Our calculations are rough in places; information is sparse, guesstimates abound. Disclaimer 2: This post draws from public info on FutureSearch as well as a paywalled report. If you want the paywalled numbers, email dan@futuresearch.ai with your LW account name and we'll send you the report for free. Here's our view of the unit economics of OpenAI's API. Note: this considers GPT-4-class models only, not audio or image APIs, and only direct API traffic, not usage in ChatGPT products. As of June 2024, OpenAI's API was very likely profitable, with surprisingly high margins. Our median estimate for gross margin (not including model training costs or employee salaries) was 75%. Once all traffic switches over to the new August GPT-4o model and pricing, OpenAI plausibly still will have a healthy profit margin. Our median estimate for the profit margin is 55%. The Information implied that OpenAI rents ~60k A100-equivalents from Microsoft for non-ChatGPT inference. If this is true, OpenAI is massively overprovisioned for the API, even when we account for the need to rent many extra GPUs to account for traffic spikes and future growth (arguably creating something of a mystery). We provide an explicit, simplified first-principles calculation of inference costs for the original GPT-4, and find significantly lower throughput & higher costs than Benjamin Todd's result (which drew from Semianalysis). Summary chart: What does this imply? With any numbers, we see two major scenarios: Scenario one: competition intensifies. With llama, Gemini, and Claude all comparable and cheap, OpenAI will be forced to again drop their prices in half. (With their margins FutureSearch calculates, they can do this without running at a loss.) LLM APIs become like cloud computing: huge revenue, but not very profitable. Scenario two: one LLM pulls away in quality. GPT-5 and Claude-3.5-opus might come out soon at huge quality improvements. If only one LLM is good enough for important workflows (like agents), it may be able to sustain a high price and huge margins. Profits will flow to this one winner. Our numbers update us, in either scenario, towards: An increased likelihood of more significant price drops for GPT-4-class models. A (weak) update that frontier labs are facing less pressure today to race to more capable models. If you thought that GPT-4o (and Claude, Gemini, and hosted versions of llama-405b) were already running at cost in the API, or even at a loss, you would predict that the providers are strongly motivated to release new models to find profit. If our numbers are approximately correct, these businesses may instead feel there is plenty of margin left, and profit to be had, even if GPT-5 and Claude-3.5-opus etc. do not come out for many months. More info at https://futuresearch.ai/openai-api-profit. Feedback welcome and appreciated - we'll update our estimates accordingly. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: In defense of technological unemployment as the main AI concern, published by tailcalled on August 28, 2024 on LessWrong. It seems to me that when normal people are concerned about AI destroying their life, they are mostly worried about technological unemployment, whereas rationalists think that it is a bigger risk that the AI might murder us all, and that automation gives humans more wealth and free time and is therefore good. I'm not entirely unsympathetic to the rationalist position here. If we had a plan for how to use AI to create a utopia where humanity could thrive, I'd be all for it. We have problems (like death) that we are quite far from solving, and which it seems like a superintelligence could in principle quickly solve. But this requires value alignment: we need to be quite careful what we mean by concepts like "humanity", "thrive", etc., so the AI can explicitly maintain good conditions. What kinds of humans do we want, and what kinds of thriving should they have? This needs to be explicitly planned by any agent which solves this task. Our current society doesn't say "humans should thrive", it says "professional humans should thrive"; certain alternative types of humans like thieves are explicitly suppressed, and other types of humans like beggars are not exactly encouraged. This is of course not an accident: professionals produce value, which is what allows society to exist in the first place. But with technological unemployment, we decouple professional humans from value production, undermining the current society's priority of human welfare. This loss is what causes existential risk. If humanity was indefinitely competitive in most tasks, the AIs would want to trade with us or enslave us instead of murdering us or letting us starve to death. Even if we manage to figure out how to value-align AIs, this loss leads to major questions about what to value-align the AIs to, since e.g. if we value human capabilities, the fact that those capabilities become uncompetitive likely means that they will diminish to the point of being vestigial. It's unclear how to solve this problem. Eliezer's original suggestion was to keep humans more capable than AIs by increasing the capabilities of humans. Yet even increasing the capabilities of humanity is difficult, let alone keeping up with technological development. Robin Hanson suggests that humanity should just sit back and live off our wealth as we got replaced. I guess that's the path we're currently on, but it is really dubious to me whether we'll be able to keep that wealth, and whether the society that replaces us will have any moral worth. Either way, these questions are nearly impossible to separate from the question of, what kinds of production will be performed in the future? Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Am I confused about the "malign universal prior" argument?, published by nostalgebraist on August 28, 2024 on LessWrong. In a 2016 blog post, Paul Christiano argued that the universal prior (hereafter "UP") may be "malign." His argument has received a lot of follow-up discussion, e.g. in Mark Xu's The Solomonoff Prior is Malign Charlie Steiner's The Solomonoff prior is malign. It's not a big deal. among other posts. This argument never made sense to me. The reason it doesn't make sense to me is pretty simple, but I haven't seen it mentioned explicitly in any of the ensuing discussion. This leaves me feeling like either I am misunderstanding the argument in a pretty fundamental way, or that there is a problem with the argument that has gotten little attention from the argument's critics (in which case I don't understand why). I would like to know which of these is the case, and correct my misunderstanding if it exists, hence this post. (Note: In 2018 I wrote a comment on the original post where I tried to state one of my objections to my argument, though I don't feel I expressed myself especially well there.) UP-using "universes" and simulatable "universes" The argument for malignity involves reasoning beings, instantiated in Turing machines (TMs), which try to influence the content of the UP in order to affect other beings who are making decisions using the UP. Famously, the UP is uncomputable. This means the TMs (and reasoning beings inside the TMs) will not be able to use[1] the UP themselves, or simulate anyone else using the UP. At least not if we take "using the UP" in a strict and literal sense. Thus, I am unsure how to interpret claims (which are common in presentations of the argument) about TMs "searching for universes where the UP is used" or the like. For example, from Mark Xu's "The Solomonoff Prior is Malign": In particular, this suggests a good strategy for consequentialists: find a universe that is using a version of the Solomonoff prior that has a very short description of the particular universe the consequentialists find themselves in. Or, from Christiano's original post: So the first step is getting our foot in the door - having control over the parts of the universal prior that are being used to make important decisions. This means looking across the universes we care about, and searching for spots within those universe where someone is using the universal prior to make important decisions. In particular, we want to find places where someone is using a version of the universal prior that puts a lot of mass on the particular universe that we are living in, because those are the places where we have the most leverage. Then the strategy is to implement a distribution over all of those spots, weighted by something like their importance to us (times the fraction of mass they give to the particular universe we are in and the particular channel we are using). That is, we pick one of those spots at random and then read off our subjective distribution over the sequence of bits that will be observed at that spot (which is likely to involve running actual simulations). What exactly are these "universes" that are being searched over? We have two options: 1. They are not computable universes. They permit hypercomputation that can leverage the "actual" UP, in its full uncomputable glory, without approximation. 2. They are computible universes. Thus the UP cannot be used in them. But maybe there is some computible thing that resembles or approximates the UP, and gets used in these universes. Option 1 seems hard to square with the talk about TMs "searching for" universes or "simulating" universes. A TM can't do such things to the universes of option 1. Hence, the argument is presumably about option 2. That is, although we are trying to reason about the content of...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: SB 1047: Final Takes and Also AB 3211, published by Zvi on August 28, 2024 on LessWrong. This is the endgame. Very soon the session will end, and various bills either will or won't head to Newsom's desk. Some will then get signed and become law. Time is rapidly running out to have your voice impact that decision. Since my last weekly, we got a variety of people coming in to stand for or against the final version of SB 1047. There could still be more, but probably all the major players have spoken at this point. So here, today, I'm going to round up all that rhetoric, all those positions, in one place. After this, I plan to be much more stingy about talking about the whole thing, and only cover important new arguments or major news. I'm not going to get into the weeds arguing about the merits of SB 1047 - I stand by my analysis in the Guide to SB 1047, and the reasons I believe it is a good bill, sir. I do however look at the revised AB 3211. I was planning on letting that one go, but it turns out it has a key backer, and thus seems far more worthy of our attention. The Media I saw two major media positions taken, one pro and one anti. Neither worried itself about the details of the bill contents. The Los Angeles Times Editorial Board endorses SB 1047, since the Federal Government is not going to step up, and using an outside view and big picture analysis. I doubt they thought much about the bill's implementation details. The Economist is opposed, in a quite bad editorial calling belief in the possibility of a catastrophic harm 'quasi-religious' without argument, and uses that to dismiss the bill, instead calling for regulations that address mundane harms. That's actually it. OpenAI Opposes SB 1047 The first half of the story is that OpenAI came out publicly against SB 1047. They took four pages to state its only criticism in what could have and should have been a Tweet: That it is a state bill and they would prefer this be handled at the Federal level. To which, I say, okay, I agree that would have been first best and that is one of the best real criticisms. I strongly believe we should pass the bill anyway because I am a realist about Congress, do not expect them to act in similar fashion any time soon even if Harris wins and certainly if Trump wins, and if they pass a similar bill that supersedes this one I will be happily wrong. Except the letter is four pages long, so they can echo various industry talking points, and echo their echoes. In it, they say: Look at all the things we are doing to promote safety, and the bills before Congress, OpenAI says, as if to imply the situation is being handled. Once again, we see the argument 'this might prevent CBRN risks, but it is a state bill, so doing so would not only not be first bet, it would be bad, actually.' They say the bill would 'threaten competitiveness' but provide no evidence or argument for this. They echo, once again without offering any mechanism, reason or evidence, Rep. Lofgren's unsubstantiated claims that this risks companies leaving California. The same with 'stifle innovation.' In four pages, there is no mention of any specific provision that OpenAI thinks would have negative consequences. There is no suggestion of what the bill should have done differently, other than to leave the matter to the Feds. A duck, running after a person, asking for a mechanism. My challenge to OpenAI would be to ask: If SB 1047 was a Federal law, that left all responsibilities in the bill to the USA AISI and NIST and the Department of Justice, funding a national rather than state Compute fund, and was otherwise identical, would OpenAI then support? Would they say their position is Support if Federal? Or, would they admit that the only concrete objection is not their True Objection? I would also confront them with AB 3211, b...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Information: OpenAI shows 'Strawberry' to feds, races to launch it, published by Martín Soto on August 28, 2024 on LessWrong. Two new The Information articles with insider information on OpenAI's next models and moves. They are paywalled, but here are the new bits of information: Strawberry is more expensive and slow at inference time, but can solve complex problems on the first try without hallucinations. It seems to be an application or extension of process supervision Its main purpose is to produce synthetic data for Orion, their next big LLM But now they are also pushing to get a distillation of Strawberry into ChatGPT as soon as this fall They showed it to feds Some excerpts about these: Plus this summer, his team demonstrated the technology [Strawberry] to American national security officials, said a person with direct knowledge of those meetings, which haven't previously been reported. One of the most important applications of Strawberry is to generate high-quality training data for Orion, OpenAI's next flagship large language model that's in development. The codename hasn't previously been reported. Using Strawberry could help Orion reduce the number of hallucinations, or errors, it produces, researchers tell me. That's because AI models learn from their training data, so the more correct examples of complex reasoning they see, the better. But there's also a push within OpenAI to simplify and shrink Strawberry through a process called distillation, so it can be used in a chat-based product before Orion is released. This shouldn't come as a surprise, given the intensifying competition among the top AI developers. We're not sure what a Strawberry-based product might look like, but we can make an educated guess. One obvious idea would be incorporating Strawberry's improved reasoning capabilities into ChatGPT. However, though these answers would likely be more accurate, they also might be slower. Researchers have aimed to launch the new AI, code-named Strawberry (previously called Q*, pronounced Q Star), as part of a chatbot - possibly within ChatGPT - as soon as this fall, said two people who have been involved in the effort. Strawberry can solve math problems it hasn't seen before - something today's chatbots cannot reliably do - and also has been trained to solve problems involving programming. But it's not limited to answering technical questions. When given additional time to "think," the Strawberry model can also answer customers' questions about more subjective topics, such as product marketing strategies. To demonstrate Strawberry's prowess with language-related tasks, OpenAI employees have shown their co-workers how Strawberry can, for example, solve New York Times Connections, a complex word puzzle. But OpenAI's prospects rest in part on the eventual launch of a new flagship LLM it is currently developing, code-named Orion. It isn't clear whether a chatbot version of Strawberry that can boost the performance of GPT-4 and ChatGPT will be good enough to launch this year. The chatbot version is a smaller, simplified version of the original Strawberry model, known as a distillation. However, OpenAI is also using the bigger version of Strawberry to generate data for training Orion, said a person with knowledge of the situation. That kind of AI-generated data is known as "synthetic." It means that Strawberry could help OpenAI overcome limitations on obtaining enough high-quality data to train new models from real-world data such as text or images pulled from the internet. In addition, Strawberry could aid upcoming OpenAI agents, this person said. Using Strawberry to generate higher-quality training data could help OpenAI reduce the number of errors its models generate, otherwise known as hallucinations, said Alex Graveley, CEO of agent startup Minion AI a...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What Depression Is Like, published by Sable on August 27, 2024 on LessWrong. I was thinking to myself about the difficulties I have explaining depression to people, when I thought of a relatively good metaphor for it. Bear with me. Sudoku For anyone unaware, Sudoku is a puzzle where one tries to fill in a 9x9 grid of numbers according to certain rules: Each row, column, and 3x3 square must have the numbers 1-9 in them, without repeating any numbers. Black numbers are given, red numbers start as blank squares and must be solved by the puzzler. It's a common form of brain teaser, much like a crossword puzzle or logic puzzle. Some Sudoku puzzles are difficult and some are easy; for our purposes we'll think about ones that are relatively easy. Brain App Imagine, for a moment, that someone hacked your brain, and installed an app in it (don't worry about the how). What this app does is force you to - whenever you want to do something - solve a mild Sudoku puzzle first. Not a hard one, it's not difficult, just annoying. Want to get out of bed? Solve a Sudoku puzzle. Want to start work in the morning? Solve a Sudoku puzzle. Want to get dressed, workout, eat, talk to someone, etc.? First you've got to solve the puzzle. At first it's irritating, but you adapt. You figure out shortcuts for solving Sudoku puzzles. It's brainpower you're not expending on anything useful, but you get by. This is the base case, the core of the metaphor. Now we expand it. There are two dimensions along which this nefarious app gets more annoying as time goes on: 1. It decreases the granularity of the actions to which it applies. In other words, where before you had to solve a Sudoku puzzle to go to work, now you've got to solve a puzzle to get dressed, a puzzle to get in the car, a puzzle to drive, and a puzzle to actually get started working. Before all of those counted as a single action - 'go to work' - now they're counted separately, as discrete steps, and each requires a puzzle. 2. It increases the number of puzzles you have to solve to do anything. At first it's just one Sudoku puzzle; eventually, it's two, then three, and so on. Having to solve a single Sudoku puzzle whenever you want to do anything is annoying; having to solve five is downright irritating. So what happens to you - what does your life look like - with this app running in your head? Dimension 1 As the depression gets worse, the granularity of the actions requiring Sudoku solves gets smaller. What does this look like? At first you go through your normal morning routine, except that upon waking up, you need to solve the Sudoku puzzle to get started. Then you have to do a Sudoku puzzle to get out of bed, another to make coffee, another to get dressed, another to shower, and so on. Then you have to do a Sudoku puzzle to open your eyes, another to sit up, another to swing your legs around and another to actually stand up. Finally, each individual muscle contraction comes with its own Sudoku puzzle. Want to sit up? That single action is composed of many pieces: your arms shift to support your weight, your stomach contracts to pull you up, your leg muscles tighten to keep your lower body in place. All of those now require their own puzzles. Each puzzle, on its own, isn't particularly difficult. But they do take some nonzero amount of effort, and when you add that required effort to every single thing you do, suddenly you find yourself doing a lot less. 'Getting out of bed' is now a complicated, multi-step operation that takes way more work than it used to. Solving all these puzzles takes time, too, so you're slower than you used to be at everything. Activities or jobs that you used to breeze through in seconds can stretch into minutes. Parts of your routine that never left you tired now leave you feeling like your brain has been lift...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why Large Bureaucratic Organizations?, published by johnswentworth on August 27, 2024 on LessWrong. Large bureaucratic organizations have long seemed… strange to me, on an intuitive level. Why do they exist? Like, in a world where the median person is John Wentworth ("Wentworld"), I'm pretty sure there just aren't large organizations of the sort our world has. Nobody would ever build such an organization, because they're so obviously wildly inefficient and misaligned. And even if somebody tried, everyone would demand prohibitively high prices to work either for the large organization or with it, since it's just so deeply unpleasant to interface with. Nobody would buy anything sold by such an organization, or vote for such an organization to continue to exist, because the organization as an entity is so obviously both incompetent and untrustworthy. So how on Earth (as opposed to Wentworld) are large organizations stable? The economists have some theorizing on the topic (google "theory of the firm"), but none of it makes me feel much less confused about the sort of large organizations I actually see in our world. The large organizations we see are clearly not even remotely economically efficient; for instance, they're notoriously full of "bullshit jobs" which do not add to the bottom line, and it's not like it's particularly difficult to identify the bullshit jobs either. How is that a stable economic equilibrium?!? In this post I'll present a model which attempts to answer that ball of confusion. The summary is: "Status", in the sense of a one-dimensional dominance hierarchy, is A Thing. We'll call it dominance-status to make it clear that we're not talking about some other kind of status. The way dominance-status normally works in higher animals, newcomers to a group generally enter near the bottom of the hierarchy (even if they were previously high-status in some other group). Within a group, dominance-status is mostly reasonably stable. So, one of the main ways group members can move "up" in dominance-status (i.e. get more members "below" them) without a risky fight, is simply to add more members to the group. Managers at large organizations are mostly motivated by dominance-status. So, the main thing for which managers get de-facto social/cognitive positive reinforcement is increasing their dominance-status and/or avoiding decreases in their dominance-status. Then, the natural prediction is that those managers (at all levels) will tend to add as many people as possible to the hierarchy under them, and minimize firing people, since that's what maximizes their dominance-status. … buuuut the drive to expand the hierarchy is limited by the organization's budget. So in practice, organizations will tend to expand until all the profit is eaten up (in the case of for-profit organizations) or until all the allocated budget is eaten up. And then the hungry managers will fight for more budget. Much of what looks like organizational "inefficiency" and "misalignment" from an standard economic efficiency perspective looks like well-aligned dominance-maximization. … so e.g. large companies or government agencies are basically runaway human monuments of dominance and submission, limited mainly by their budget. There's a lot of steps here, and I'm not super-confident in this model. But when I step into the model, large organizations no longer look strange and confusing; the model seems to generate a remarkably good description of most real large organizations, both private and public. Now let's walk through the model in more detail, starting with relevant background studies. Background: Dominance-Status Empirical Ontology Justification: Dominance-Status Is A Thing "Status" typically connotes a mental model in which we could assign everyone a number/rank, and then some kind of beha...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Soft Nationalization: How the US Government Will Control AI Labs, published by Deric Cheng on August 27, 2024 on LessWrong. We have yet to see anyone describe a critical element of effective AI safety planning: a realistic model of the upcoming role the US government will play in controlling frontier AI. The rapid development of AI will lead to increasing national security concerns, which will in turn pressure the US to progressively take action to control frontier AI development. This process has already begun,[1] and it will only escalate as frontier capabilities advance. However, we argue that existing descriptions of nationalization[2] along the lines of a new Manhattan Project[3] are unrealistic and reductive. The state of the frontier AI industry - with more than $1 trillion[4] in private funding, tens of thousands of participants, and pervasive economic impacts - is unlike nuclear research or any previously nationalized industry. The traditional interpretation of nationalization, which entails bringing private assets under the ownership of a state government,[5] is not the only option available. Government consolidation of frontier AI development is legally, politically, and practically unlikely. We expect that AI nationalization won't look like a consolidated government-led "Project", but rather like an evolving application of US government control over frontier AI labs. The US government can select from many different policy levers to gain influence over these labs, and will progressively pull these levers as geopolitical circumstances, particularly around national security, seem to demand it. Government control of AI labs will likely escalate as concerns over national security grow. The boundary between "regulation" and "nationalization" will become hazy. In particular, we believe the US government can and will satisfy its national security concerns in nearly all scenarios by combining sets of these policy levers, and would only turn to total nationalization as a last resort. We're calling the process of progressively increasing government control over frontier AI labs via iterative policy levers soft nationalization. It's important to clarify that we are not advocating for a national security approach to AI governance, nor yet supporting any individual policy actions. Instead, we are describing a model of US behavior that we believe is likely to be accurate to improve the effectiveness of AI safety agendas. Part 1: What is Soft Nationalization? Our Model of US Control Over AI Labs We'd like to define a couple terms used in this article: Total nationalization: The traditional meaning of "nationalization", where a government transforms private industry or organizations into a public asset, taking over full ownership and control. Soft nationalization: In contrast to total nationalization, soft nationalization encompasses a wide-ranging set of policy levers governments can use to increase control over the direction, impact, and applications of a private industry or organization. These levers may allow governments to achieve their high-level goals without taking full ownership of said entity. We argue that soft nationalization is a useful model to characterize the upcoming involvement of the US government in frontier AI labs, based on our following observations: 1. Private US AI labs are currently the leading organizations pushing the frontier of AI development, and will be among the first to develop AI with transformative capabilities. 2. Advanced AI will have significant impacts on national security and the balance of global power. 3. A key priority for the US government is to ensure global military and technological superiority - in particular, relative to geopolitical rivals such as China. 4. Hence, the US government will begin to exert greater control and ...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What is it to solve the alignment problem?, published by Joe Carlsmith on August 27, 2024 on LessWrong. People often talk about "solving the alignment problem." But what is it to do such a thing? I wanted to clarify my thinking about this topic, so I wrote up some notes. In brief, I'll say that you've solved the alignment problem if you've: 1. avoided a bad form of AI takeover, 2. built the dangerous kind of superintelligent AI agents, 3. gained access to the main benefits of superintelligence, and 4. become able to elicit some significant portion of those benefits from some of the superintelligent AI agents at stake in (2).[1] The post also discusses what it would take to do this. In particular: I discuss various options for avoiding bad takeover, notably: Avoiding what I call "vulnerability to alignment" conditions; Ensuring that AIs don't try to take over; Preventing such attempts from succeeding; Trying to ensure that AI takeover is somehow OK. (The alignment discourse has been surprisingly interested in this one; but I think it should be viewed as an extreme last resort.) I discuss different things people can mean by the term "corrigibility"; I suggest that the best definition is something like "does not resist shut-down/values-modification"; and I suggest that we can basically just think about incentives for/against corrigibility in the same way we think about incentives for/against other types of problematic power-seeking, like actively seeking to gain resources. I also don't think you need corrigibility to avoid takeover; and I think avoiding takeover should be our focus. I discuss the additional role of eliciting desired forms of task-performance, even once you've succeeded at avoiding takeover, and I modify the incentives framework I offered in a previous post to reflect the need for the AI to view desired task-performance as the best non-takeover option. I examine the role of different types of "verification" in avoiding takeover and eliciting desired task-performance. In particular: I distinguish between what I call "output-focused" verification and "process-focused" verification, where the former, roughly, focuses on the output whose desirability you want to verify, whereas the latter focuses on the process that produced that output. I suggest that we can view large portions of the alignment problem as the challenge of handling shifts in the amount we can rely on output-focused verification (or at least, our current mechanisms for output-focused verification). I discuss the notion of "epistemic bootstrapping" - i.e., building up from what we can verify, whether by process-focused or output-focused means, in order to extend our epistemic reach much further - as an approach to this challenge.[2] I discuss the relationship between output-focused verification and the "no sandbagging on checkable tasks" hypothesis about capability elicitation. I discuss some example options for process-focused verification. Finally, I express skepticism that solving the alignment problem requires imbuing a superintelligent AI with intrinsic concern for our "extrapolated volition" or our "values-on-reflection." In particular, I think just getting an "honest question-answerer" (plus the ability to gate AI behavior on the answers to various questions) is probably enough, since we can ask it the sorts of questions we wanted extrapolated volition to answer. (And it's not clear that avoiding flagrantly-bad behavior, at least, required answering those questions anyway.) Thanks to Carl Shulman, Lukas Finnveden, and Ryan Greenblatt for discussion. 1. Avoiding vs. handling vs. solving the problem What is it to solve the alignment problem? I think the standard at stake can be quite hazy. And when initially reading Bostrom and Yudkowsky, I think the image that built up most prominently i...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: My Apartment Art Commission Process, published by jenn on August 27, 2024 on LessWrong. When I know that I'm going to be moving out from an apartment soon, I commission a digital artist to draw it for me. Then I print it out and I have a cool art piece. If you love your current place but you don't think you'll spend the rest of your life there, you should consider doing the same. Digital artists are much cheaper than I think they should be. I've paid artists between $200-$500 CAD[1] for my commissions, generally spread across one or two additional housemates. (You should expect to pay more - I limit my own commissions to the common areas since my bedrooms tend to be very plain, and solely used for sleep and other private activities. Also inflation exists.) You can also consider hiring artists from developing countries if you want your dollar to go further, but I don't have any advice on how to seek those folks out specifically. You'll be looking at around 10 hours of effort on your end, frontloaded but spread out across 2-4 months. I detail my process below. But first, here are the pieces that I've commissioned so far: Aren't they sick as hell??? I love them so much. Okay, let's get you started on yours. I'll stick a sample email script at the bottom too. Commissioning An Art Of Your Living Space, Step By Step 1. come up with a budget talk to your roomies if you have them, and come up with a price you're willing to pay altogether. i think my apartment commissions are probably 15-30? hours of work, multiply that by how much you're willing to pay a skilled artisan for an hour of work. (i should note that in 3/3 cases for me, the minimum budget ended up being like 30-100% more than what the artist was willing to accept. digital artists often decline to charge reasonable rates for their labour.) 2. find 2-3 viable artists endorsed strategies involve browsing r/wimmelbilder, the twitter/tumblr hashtag #isometricart, and google imagesing "isometric apartment layout" and clicking around. for maximal exposure to artists that are open to work, search dribbble.com for "isometric", but note that the pickings there are fairly slim. in many isometric tags i find a lot of rendered stuff but i prefer to go for more trad art forms as i expect renderings to be more time consuming (expensive), harder to edit, and worse for the amount of detail i want[2]. also, you don't need to commission specifically an isometric piece! you can go wild at this step finding any artist who illustrates interiors in a way you like. while browsing, it could be a good idea to save sample images that you like; you can then pass them on to the artist of your choice as reference for what kind of art appeals to you. find artists whose work make you feel actively excited, when you think about having your own apartment done in their style. check out the portfolios of artists you like. you're looking for portfolios with a pretty solid number of pieces, ideally at least like ~5 years of stuff, and maybe a consistent style if it's a style you like. new artists could be high variance, and for all you know you might be messaging a talented 15 year old who will drop you like a hot potato when they need to start studying for an exam in earnest (my little brother has turned down commission inquiries for this reason when he was in high school). i don't think AI art is good enough to do this kind of work yet, so I'd stick with traditional digital (lol) artists for now. 3. email the viable artists email the artists whose portfolios passed the vibe check, letting them know what you want to commission them for and your budget, and asking for a quote if they are open to working with you. having 2-3 artists on hand here is good because it's kind of 50/50 if any particular artist online is accepting commissions. don't take it p...