The Nonlinear Library

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Conditional Trees: Generating Informative Forecasting Questions (FRI) -- AI Risk Case Study, published by Forecasting Research Institute on August 13, 2024 on The Effective Altruism Forum. Authors of linked report: Tegan McCaslin, Josh Rosenberg, Ezra Karger, Avital Morris, Molly Hickman, Otto Kuusela, Sam Glover, Zach Jacobs, Phil Tetlock[1] Today, the Forecasting Research Institute (FRI), released "Conditional Trees: A Method for Generating Informative Questions about Complex Topics," which discusses the results of a case study in using conditional trees to generate informative questions about AI risk. In this post, we provide a brief overview of the methods, findings, and directions for further research. For much more analysis and discussion, see the full report: https://forecastingresearch.org/s/AIConditionalTrees.pdf Abstract We test a new process for generating high-value forecasting questions: asking experts to produce "conditional trees," simplified Bayesian networks of quantifiably informative forecasting questions. We test this technique in the context of the current debate about risks from AI. We conduct structured interviews with 21 AI domain experts and 3 highly skilled generalist forecasters ("superforecasters") to generate 75 forecasting questions that would cause participants to significantly update their views about AI risk. We elicit the "Value of Information" (VOI) each question provides for a far-future outcome - whether AI will cause human extinction by 2100 - by collecting conditional forecasts from superforecasters (n=8).[2] In a comparison with the highest-engagement AI questions on two forecasting platforms, the average conditional trees-generated question resolving in 2030 was nine times more informative than the comparison AI-related platform questions (p = .025). This report provides initial evidence that structured interviews of experts focused on generating informative cruxes can produce higher-VOI questions than status quo methods. Executive Summary From May 2022 to October 2023, the Forecasting Research Institute (FRI) (a)[3] experimented with a new method of question generation ("conditional trees"). While the questions elicited in this case study focus on potential risks from advanced AI, the processes we present can be used to generate valuable questions across fields where forecasting can help decision-makers navigate complex, long-term uncertainties. Methods Researchers interviewed 24 participants, including 21 AI and existential risk experts and three highly skilled generalist forecasters ("superforecasters"). We first asked participants to provide their personal forecast of the probability of AI-related extinction by 2100 (the "ultimate question" for this exercise).[4] We then asked participants to identify plausible[5] indicator events that would significantly shift their estimates of the probability of the ultimate question. Following the interviews, we converted these indicators into 75 objectively resolvable forecasting questions. We asked superforecasters (n=8) to provide forecasts on each of these 75 questions (the "AICT" questions), and forecasts on how their beliefs about AI risk would update if each of these questions resolved positively or negatively. We quantitatively ranked the resulting indicators by Value of Information (VOI), a measure of how much each indicator caused superforecasters to update their beliefs about long-run AI risk. To evaluate the informativeness of the conditional trees method relative to widely discussed indicators, we assess a subset of these questions using a standardized version of VOI, comparing them to popular AI questions on existing forecasting platforms (the "status quo" questions). The status quo questions were selected from two popular forecasting platforms by identifying the highest-...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Debate: Get a college degree?, published by Ben Pace on August 13, 2024 on LessWrong. Epistemic Status: Soldier mindset. These are not our actual positions, these are positions we were randomly assigned by a coin toss, and for which we searched for the strongest arguments we could find, over the course of ~1hr 45mins. That said, this debate is a little messy between our performed positions and our personal ones. Sides: Ben is arguing against getting a college degree, and Saul is arguing for . (This is a decision Saul is currently making for himself!) Reading Order: Ben and Saul drafted each round of statements simultaneously. This means that each of Ben's statements you read were written without Ben having read Saul's statements that are immediately proceeding. (This does not apply to the back-and-forth interview.) Saul's Opening Statement first - i do think there's a qualitative difference between the position "getting an undergrad degree is good" vs "getting the typical undergrad experience is good." i think the second is in some ways more defensible than the first, but in most ways less so. For "getting the typical undergrad experience is good" This sort of thing is a strong Chesterton fence. People have been having the typical experience of an undergrad for a while (even while that typical experience changes). General upkeeping of norms/institutions is good. I think that - for a some ppl - their counterfactual is substantially worse. Even if this means college is functionally daycare, I'd rather they be in adult-day-care than otherwise being a drain on society (e.g. crime). It presents the option for automatic solutions to a lot of problems: Socializing high density of possible friends, romantic partners, etc you have to go to classes, talk to ppl, etc Exercise usually a free gym that's at-least functional you gotta walk to class, dining hall, etc Tons of ability to try slightly "weird" stuff you've never tried before - clubs, sports, events, greek life, sexual interactions, classes, etc I think a lot of these things get a lot more difficult when you haven't had the opportunity to experiment w them. A lot of ppl haven't experimented w much of anything before - college gives them an easy opportunity to do that w minimal friction before doing so becomes gated behind a ridiculous amount of friction. E.g. getting into a new hobby as an adult is a bit odd, in most social settings - but in college, it's literally as simple as joining that club. Again - while all of these sorts of things are possible outside of college, they become more difficult, outside of the usual norms, etc. For "getting an undergrad degree is good": This is a strong Chesterton fence. People have been getting undergrad degrees - or similar - for a wihle. It's an extremely legible symbol for a lot of society: Most ppl who get undergrad degrees aren't getting the sort of undergrad degree that ben or i sees - i think most are from huge state schools, followed by the gigantic tail of no-name schools. For those ppl, and for the jobs they typically seek, my guess is that for demonstrating the necessary things, like "i can listen to & follow directions, navigate general beaurocracies, learn things when needed, talk to people when needed, and am unlikely to be a extremely mentally ill, etc" - an undergrad degree is a pretty good signal. my guess is that a big part of the problem is that, despite this legible signal being good, ppl have indexed on it way too hard (& away from other signals of legibility, like a trade school, or a high school diploma with a high GPA or something). there are probably some instances where getting an undergrad degree isn't good, but those instances are strongly overrepresented to ben & saul, and the base rate is not that. also, it seems like society should give greater affordan...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A top-effective public outreach: small-farmed-animal reducetarian deep questioning, published by Stijn on August 13, 2024 on The Effective Altruism Forum. For two decades, I did a lot of public outreach: talking with people from the broad or general public (on the streets,…) to persuade them to do something good. Now I believe that most of such public outreach is not so effective, with one or a few exceptions. Here I want to give a concrete example of a public outreach that I believe to be one of the most cost-effective, generating the highest impact at the lowest costs: small-farmed-animal reducetarian deep questioning. This involves asking specific questions to make people think more deeply about the welfare of (small) animals used for food in order to encourage them to reduce their consumption of animal products (primarily from small animals such as chickens, fish and shrimp). I'll first discuss why I think this is a top-effective strategy, and then present a specific example of a list of questions that I ask people on the streets. Why is small-farmed-animal reducetarian deep questioning a top-effective public outreach? The term 'small-farmed-animal reducetarian deep questioning' has three parts. First, the focus area or problem that we want to tackle: the suffering and rights violations of small farmed animals. Second, the objective or personal choice that we want people to make: a behavior change that involves reducing the consumption of specific food products. Third, the method or approach that we want to apply: conversations with people on the street where we ask them deep questions. Concerning the focus area: probably the largest contributor to global suffering and loss of welfare, is the farming of small animals such as chickens, fish and shrimp. Evidence from a survey shows that most people already have opinions and judgments that lead to this conclusion. This focus area is also closely connected to the public, because it is the result of the personal behavior (consumption choice) of members of the public. Consuming animal products is not far from their bed. That makes it a good candidate for public outreach. Concerning the objective: there are many things that we can ask people on the streets to do. We can ask them to reduce eating some products, to boycott some companies, to reduce flying, to lower the personal carbon footprint, to install solar panels, to donate blood, to sign petitions, to join protest marches, to donate to charities, to switch their careers towards high-impact jobs, to vote for the best political candidates, and many more. I think most of these objectives are not so effective for public outreach. Boycotting a company, flying less, lowering one's personal carbon footprint, installing solar panels and donating blood have a relatively low impact compared to reducing meat consumption. Public outreach to sign petitions and join protest marches requires coordination: their effectiveness depends on what other people do. If other people do not collect signatures, you will end up with a petition that has only a few signatures and hence a low impact. Also, petitions and protest marches have a low success rate. They can have a high impact when successful, but the probability is high that they achieve nothing. That means these strategies are less suitable for public outreachers who have a difference making risk-aversion, i.e. a preference for actions that are more likely to make a difference. Donating to top-effective charities does have a huge impact, but based on my experience in public fundraising (I worked as a charity fundraiser for a few months) and deep questioning, I think this objective is less suitable for deep questioning public outreach. I didn't find a good conversation approach, with the right questions to ask, that motivates people ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Humanity isn't remotely longtermist, so arguments for AGI x-risk should focus on the near term, published by Seth Herd on August 13, 2024 on LessWrong. Toby Ord recently published a nice piece On the Value of Advancing Progress about mathematical projections of far-future outcomes given different rates of progress and risk levels. The problem with that and many arguments for caution is that people usually barely care about possibilities even twenty years out. We could talk about sharp discounting curves in decision-making studies, and how that makes sense given evolutionary pressures in tribal environments. But I think this is pretty obvious from talking to people and watching our political and economic practices. Utilitarianism is a nicely self-consistent value system. Utilitarianism pretty clearly implies longtermism. Most people don't care that much about logical consistency,[1] so they are happily non-utilitarian and non-longtermist in a variety of ways. Many arguments for AGI safety are longtermist, or at least long-term, so they're not going to work well for most of humanity. This is a fairly obvious, but worth-keeping-in-mind point. One non-obvious lemma of this observation is that much skepticism about AGI x-risk is probably based on skepticism about AGI happening soon. This doesn't explain all skepticism, but it's a significant factor worth addressing. When people dig into their logic, that's often a central point. They start out saying "AGI wouldn't kill humans" then over the course of a conversation it turns out that they feel that way primarily because they don't think real AGI will happen in their lifetimes. Any discussion of AGI x-risks isn't productive, because they just don't care about it. The obvious counterpoint is "You're pretty sure it won't happen soon? I didn't know you were an expert in AI or cognition!" Please don't say this - nothing convinces your opponents to cling to their positions beyond all logic like calling them stupid.[2] Something like "well, a lot of people with the most relevant expertise think it will happen pretty soon. A bunch more think it will take longer. So I just assume I don't know which is right, and it might very well happen pretty soon". It looks to me like discussing whether AGI might threaten humans is pretty pointless if the person is still assuming it's not going to happen for a long time. Once you're past that, it might make sense to actually talk about why you think AGI would be risky for humans.[3] 1. ^ This is an aside, but you'll probably find that utilitarianism isn't that much more logical than other value systems anyway. Preferring what your brain wants you to prefer, while avoiding drastic inconsistency, has practical advantages over values that are more consistent but that clash with your felt emotions. So let's not assume humanity isn't utilitarian just because it's stupid. 2. ^ Making sure any discussions you have about x-risk are pleasant for all involved is probably actually the most important strategy. I strongly suspect that personal affinity weighs more heavily than logic on average, even for fairly intellectual people. (Rationalists are a special case; I think we're resistant but not immune to motivated reasoning). So making a few points in a pleasant way, then moving on to other topics they like is probably way better than making the perfect logical argument while even slightly irritating them. 3. ^ From there you might be having the actual discussion on why AGI might threaten humans. Here are some things I've seen be convincing. People seem to often think "okay fine it might happen soon, but surely AI smarter than us still won't have free will and make its own goals". From there you could point out that it needs goals to be useful, and if it misunderstands those goals even slightly, it might be...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Californians, tell your reps to vote yes on SB 1047!, published by Holly Elmore on August 12, 2024 on The Effective Altruism Forum. See the event page here. Hello Californians! We need you to help us fight for SB 1047, a landmark bill to help set a benchmark for AI safety, decrease existential risk, and promote safety research. This bill has been supported by some of the world's leading AI scientists and the Center of AI Safety, and is extremely important for us to pass. As Californians, we have a unique opportunity to inspire other states to follow suit. Unfortunately, due to misinformation and lobbying by big tech companies, SB 1047 is currently stalled in the Assembly Appropriations Committee. It may be killed if it does not get advanced by August 19th. This would be a significant blow against safety and would continue the "race to the bottom" in AI capabilities without any guardrails. We need you to do the following to save the bill. This will take no more than 5 minutes: Email the Chair (Buffy Wicks, D) and Vice-Chair (Kate Sanchez, R) of the Appropriations Committee, and ask them to advance the bill without watering it down. Buffy Wicks: assemblymember.wicks@assembly.ca.gov Kate Sanchez: assemblymember.sanchez@assembly.ca.gov Email templates and examples can be found here. Call the offices of the Chair and Vice-Chair in support of the bill. The best day to do this is August 16. Buffy Wicks (D): (916) 319-2014 Kate Sanchez (R): (916) 319-2071 These calls can be very short (~1 minute) and straightforward. See here for more guidance. Here's a Google Calendar event you can use to mark the date on your calendar. This document has additional information about the bill and other ways to help. Please try to get this done as soon as possible, and let us know if you need any help. Your voice matters, and it is urgent that we push this before it's too late. Thank you so much for your support! Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Time preferences for impact in the EA community (data from the 2023 EA Survey Supplement), published by Jamie Elsey on August 12, 2024 on The Effective Altruism Forum. As part of ongoing research into EA community attitudes and priorities, a supplemental EA community survey was fielded from December 2023 to January 2024. This survey included a number of questions requested by EA decision-makers and researchers. One such question concerned EA community attitudes regarding time preferences for impact. The question requested was: Putting aside issues of tractability, what are your time preferences with respect to the altruistic impacts of your actions? I am at least roughly indifferent between benefiting someone today and benefiting someone to an equal extent indefinitely far in the future. I value benefiting someone today more than benefiting someone to an equal extent in the future, and the value I place on future benefits declines roughly exponentially per year by ___%. I value helping someone today less than helping someone to an equal extent in the future, and the value I place on future benefits rises roughly exponentially per year by ___%. I am at least roughly indifferent between current and future benefits among members of a given generation, but I value helping members of the current generation substantially more than members of future generations (holding the sizes, but not necessarily the identities, of future generations fixed). I am at least roughly indifferent between current and future benefits among members of a given generation, but I value helping members of the current generation substantially less than members of future generations (holding the sizes, but not necessarily the identities, of future generations fixed). My time preferences are not described by any of these. In total, 377 respondents provided answers to this question. The most endorsed attitude (44%) was to be roughly indifferent between benefiting someone today relative to indefinitely far in the future. This was followed by preferring to help current people, with a specified 'decay' of value into the future (endorsed by 25% of respondents). A further 17% reported being indifferent with respect to time when helping within a generation of people, but preferring helping the current generation of people more than future generations. Finally, 11% of respondents reported that none of the specified attitudes matched their preferences. All other views received less than 2% endorsement. Within those who reported preferring to help people now, we received 88 responses that provided a specific percentage of decay per year. The two plots below show, firstly, the number of people giving different percentage values, and secondly, the cumulative probability of giving different percentages. Over 50% of these respondents had a value at or less than 5%, and 90% of people reported a percentage decline of 10% or less. However, there was substantial variation. A handful of people reported very high percentages such as 80% or 100%. The mean of the distribution was 6.8, with the single most endorsed percentage decline being 5% (n = 21), followed by 1% (n = 19). Precise percentages given by each respondent are shown in the table below: Only three people who selected the 'increasing' value into the future option provided percentages, and one of these selected 0, so we do not display values for that response option. When interpreting the substantive meaning of these responses, it is important to remember that the question stipulated that responses should be 'putting aside issues of tractability'. Respondents' attitudes may differ when taking into account tractability. It should also be noted that the question was quite complex, and may have been challenging for respondents to answer. Rethink Priorities is a thi...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Extracting SAE task features for ICL, published by Dmitrii Kharlapenko on August 12, 2024 on The AI Alignment Forum. TL;DR We try to study task vectors in the SAE basis. This is challenging because there is no canonical way to convert an arbitrary vector in the residual stream to a linear combination of SAE features - you can't just pass an arbitrary vector through the encoder without going off distribution. We explored the algorithm of gradient pursuit suggested in Smith et al, but it didn't work for us without modifications. Our approach is to apply the SAE encoder to the task vector, and then apply a gradient-based cleanup. This exploits the fact that task vectors have a differentiable objective. We find that this gives a sparser and cleaner reconstruction, which is also highly interpretable, and also serves as a better task vector due to directly optimizing for log likelihood. This takes us from ~100 active features to ~10. Using our algorithm, we find two classes of SAE features involved in ICL. One of them recognizes the exact tasks or output formats from the examples, and another one encodes the tasks for execution by the model later on. We show that steering with these features has causal effects similar to task vectors. This work was produced as part of the ML Alignment & Theory Scholars Program - Summer 24 Cohort, under mentorship from Neel Nanda and Arthur Conmy. Prior work Task or function vectors are internal representations of some task that LLMs form while processing an ICL prompt. They can be extracted from a model running on a few-shot prompt and then be used to make it complete the same task without having any prior context or task description. Several papers (Function vectors in large language models, In-Context Learning Creates Task Vectors) have proposed different ways to extract those task vectors. They all center around having ICL examples being fed to a model in the form of "input output, … " and averaging the residuals on the "separator" token over a batch. This approach can reconstruct some part of the ICL performance but does not admit a straightforward conversion to the SAE basis. ITO with gradient pursuit can be used to do a sparse coding of a residual vector using SAE features. The post suggests using this algorithm for steering vector SAE decomposition. Since task vectors can be thought of as steering vectors, ITO may provide some insight into the ways they operate. Initial Phi-3 experiments Direct SAE task vector reconstruction In our study we trained a set of gated SAEs for Phi-3 Mini 3.8B using a model-generated synthetic instruction dataset. While offering a sparse dictionary decomposition of residuals, SAEs tend to introduce a reconstruction error that impacts the performance of the model. They also have no guarantee to be able to decompose out-of-distribution vectors, and task vectors being a product of averaging activations across prompts and tokens may be the case of such vectors. Thus, we first studied the performance of SAE reconstructions of task vectors in transferring the definition of two tasks: 1) antonym generation and 2) English to Spanish word translation. These and other tasks used to study task vectors were taken from the ICL task vectors paper github repository. These charts show the NLL loss of the model on the evaluation set of zero-shot prompts for both of the tasks depending on the layer of extraction/insertion. TV stands for the original task vector performance; Recon of TV stands for using the SAE reconstruction of the task vector instead of the task vector; TV on recon stands for first doing a SAE reconstruction of the residuals and then collecting a task vector on them; ITO stands for the ITO algorithm with 40 target l0 loss. It can be seen from charts that SAE reconstruction significantly decrea...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: In Defense of Open-Minded UDT, published by Abram Demski on August 12, 2024 on The AI Alignment Forum. A Defense of Open-Minded Updatelessness, with applications to alignment. This work owes a great debt to many conversations with Sahil, Martín Soto, and Scott Garrabrant. Sahil and I will have a public discussion about this this Wednesday (August 14) at 11am EDT; join via this link. You can support my work on Patreon. Iterated Counterfactual Mugging On a Single Coinflip Iterated counterfactual mugging on a single coinflip begins like a classic counterfactual mugging, with Omega approaching you, explaining the situation, and asking for your money. Let's say you buy the classic UDT idea, so you happily give Omega your money. Next week, Omega appears again, with the same question. However, Omega clarifies that it has used the same coin-flip as last week. This throws you off a little bit, but you see that the math is the same either way; your prior still assigns a 50-50 chance to both outcomes. If you thought it was a good deal last week, you should also think it is a good deal this week. You pay up again. On the third week, Omega makes the same offer again, and once again has used the same coinflip. You ask Omega how many times it's going to do this. Omega replies, "forever". You ask Omega whether it would have continued coming if the coin had landed heads; it says "Of course! How else could I make you this offer now? Since the coin landed tails, I will come and ask you for $100 every single week going forward. If the coin had landed heads, I would have simulated what would happen if it had landed tails, and I would come and give you $10,000 on every week that simulated-you gives up $100!" Let's say for the sake of the thought experiment that you can afford to give Omega $100 once a week. It hurts, but it doesn't hurt as much as getting $10,000 from Omega every week would have benefited you, if that had happened.[1] Nonetheless, I suspect many readers will feel some doubt creep in as they imagine giving Omega $100 week after week after week. The first few weeks, the possibility of the coin landing heads might feel "very real". Heck yeah I want to be the sort of person who gets a 50% chance of 10K from Omega for a (50% chance) cost of $100! By the hundredth week, though, you may feel yourself the fool for giving up so much money for the imaginary benefit of the "heads" world that never was. If you think you'd still happily give up the $100 for as long as Omega kept asking, then I would ask you to consider a counterlogical mugging instead. Rather than flipping a coin, Omega uses a digit of the binary expansion of π; as before, Omega uses the same digit week after week, for infinitely many counterlogical muggings. Feeling uneasy yet? Does the possibility of the digit of π going one way or the other continue to feel "just as real" as time passes? Or do you become more sympathetic to the idea that, at some point, you're wasting money on helping a non-real world? UDT vs Learning Updateless Decision Theory (UDT) clearly keeps giving Omega the $100 forever in this situation, at least, under the usual assumptions. A single Counterfactual Mugging is not any different from an infinitely iterated one, especially in the version above where only a single coinflip is used. The ordinary decision between "give up $100" and "refuse" is isomorphic to the choice of general policy "give up $100 forever" and "refuse forever".[2] However, the idea of applying a decision theory to a specific decision problem is actually quite subtle, especially for UDT. We generally assume an agent's prior equals the probabilities described in the decision problem.[3] A simple interpretation of this could be that the agent is born with this prior (and immediately placed into the decision problem). This isn't v...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Rowing vs steering, published by Saul Munn on August 12, 2024 on LessWrong. Alex Lawsen used a great metaphor on the 80k After Hours podcast:[1] [1:38:14] …you're rowing a boat on your own, and you're trying to get somewhere, and you've got some map that you need to look at to see where you're going, I imagine like a map and compass. […] When you're rowing, you're facing back; you can't see where you're going. You've just got to sit there and pull both of the oars, and do that a bunch of times, and then the boat goes forwards. […] You steer [… by pulling] harder with one side, something like that. I can imagine […] you sitting forwards in the boat, and trying to hold the map with your left hand while it's gripping one oar, and trying to hold the compass with your right hand while it's gripping the other; pushing them rather than pulling them while looking at where you're going; so you're always precisely on track, but my guess is you're just going to go super slowly, because that's not how to row a boat. Whereas you can imagine someone else, maybe someone that's racing you, who is going to point the boat in pretty much the right direction - they're not exactly sure it's the right direction, and they might go a bit off course. And then they go, "Cool. I'm going to row hard for a minute, and then I'm going to stop and check I'm pointing in the right direction, and then I'm going to row hard for another minute." [1:37:56] The metaphor is trying to point at … the strategy, [which] is pretty clear: gather some information, make a decision with that information, stick to that decision for some period of time that you've planned in advance, and then reevaluate, gather some more information, and then make a new decision. [1:35:58] … you [should] stick to some policy, which is like: "I'm going to look at a bunch of things, I'm going to actually seriously consider my options. And then, with all of the information I have, I'm going to make a decision. And I'm going to make the decision to do the thing that seems best for some fixed period of time. At the end of that fixed period of time, then I will consider other options." [1:47:43] … if you think expected value is a reasonable framework to use, … then I do actually want to say: I think having this kind of policy is actually the thing that seems best in expectation. [1:41:21] … I think some people … they start doing a thing, and then they're so worried about whether it's the best, that they're just miserable, and they never find out if it is the best thing for them because they're not putting all of their effort in, because they've got one foot out of the door because they think something else could be better. When you're in a rowboat, you don't want to be constantly rowing (and never steering), nor constantly steering (and never rowing). But there's also an in-between state that's still a failure mode, where you're trying to half-row and half-steer all at the same time. You'd be way better off by purely rowing for a bit, then purely steering for a bit, then back and forth again, but it causes anxiety to purely row without steering ("what if I'm rowing in the wrong direction!"), and it causes less forward progress to purely steer with no rowing ("I'm not even moving!"). So Alex's solution is to set a policy that looks something like: "For the next minute, I'm going to row hard. After sixty seconds, I'll turn around and steer. But for the next sixty seconds, I'm not even going to consider that I'm rowing in the wrong direction, because I'm in rowing mode, not steering mode." And importantly, having the knowledge that you'll be correcting your course sixty seconds from now makes it so much less anxiety-inducing to purely row for sixty seconds straight. I've used this in situations where it's costly to be thinking about how best ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: All The Latest Human tFUS Studies, published by sarahconstantin on August 10, 2024 on LessWrong. Transcranial focused ultrasound neuromodulation - altering the brain's activity with low-intensity ultrasound - is really exciting . It allows us to manipulate arbitrary regions of the brain without surgery, potentially replacing the (brain-damaging) electrode implants currently used for serious neurological conditions like epilepsy and Parkinson's, and potentially also expanding applications of brain stimulation to milder conditions not worth the risks of brain surgery, like mental illness, addiction, or chronic pain. The field is rapidly growing, and since I wrote my earlier post series there have been quite a few human studies published. Here's a systematic overview of all the human studies published in 2024, by target brain region. Headline Results This year's papers further confirm, to start with, that ultrasound does things to brain activity, if that was still in doubt, and that it is safe enough to run human experiments with (no adverse effects during experiments with small numbers of participants and brief exposures.) There are notably inconsistent results in whether targeting ultrasound to a given brain area increases or decreases neural activity in that area, even in some cases when the same area is targeted with the same sonication parameters! We clearly need to get a better sense of what ultrasound even does. Most studies don't do the obvious (but admittedly expensive) thing of confirming a change in neural activity via a noninvasive measure like fMRI. Those that do, show different results (more activity in the targeted region, less activity in the targeted region, or neither) depending on which region is targeted; this tells us that "tFUS" as a class doesn't have a globally consistent effect on targeted neural activity. Again, still more to learn. However, despite the primitive state of our understanding of this modality, we do already seem to have some strikingly useful results. Ultrasound stimulation of the thalamus seems to be helpful for essential tremor, stimulation of the posterior insula seems to reduce pain sensitivity, and stimulation of the anterior medial prefrontal cortex seems to have quite strong effects on depression. These are before vs. after results without a control group, not randomized controlled studies, but I think they at least warrant followup. I'm not as excited as I'd want to be about Jay Sanguinetti's default-mode-network-inhibition study. The effects seem subtle and game-able; and anecdotally the stories I hear from people who've tried the protocol from his lab are not "I was in a clearly altered state". But all in all, it continues to be a promising field; tFUS clearly does things, some of those things may be useful, and the more data we get, the closer we'll get to an actual model of what it does. Amygdala Chou, et al1 at Harvard Medical School tested tFUS2 on the left amygdalas of 30 healthy volunteers. Compared to sham stimulation, tFUS resulted in less fMRI-measured activity in the amygdala. The amygdala is involved in fear responses, so reducing amygdala activity could have uses in anxiety disorders and phobias. Hoang-Dang, et al3 at UCLA used tFUS4 on the right amygdala of 21 older adults, and found no effect on state anxiety after tFUS, but did show an increase in negative emotional reaction to viewing negative images. There was also a significant increase in heart rate between trials of this mildly stressful task. Since the amygdala is usually active during fear, this suggests that these stimulation parameters may have activated the amygdala…despite the other study using similar parameters and showing a direct decrease in amygdala activity. The UCLA study doesn't mention the duration of tFUS stimulation, which may be a re...