Until this Sunday, you can submit your 2025 AI predictions at ai2025.org. It's a forecasting survey by AI Digest for the 2025 performance on various AI benchmarks, as well as revenue and public attention. You can share your results in a picture like this one. I personally found it pretty helpful to learn about the different benchmarks, and also to think through my timelines estimates. The survey will close on Sunday, January 19th (anywhere on Earth).If you know any AI public intellectuals or discourse influencers who might be interested in submitting the survey, please encourage them to do so!Survey link: ai2025.org The original text contained 1 image which was described by AI. --- First published: January 15th, 2025 Source: https://www.lesswrong.com/posts/cdPPr6XtPkCX5c8Ny/predict-2025-ai-capabilities-by-sunday --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This post is mostly an ad to make people click the https://calendly.com/gurkenglas/consultation link starring in my Would you like me to debug your math? post.But writing just an ad feels icky to me, so first: The advice that was most useful to most peoplewas on how to use (graphical) debuggers. The default failure mode I see in a programmer is that they're wrong about what happens in their program. The natural remedy is to increase the bandwidth between the program and the programmer. Many people will notice an anomaly and then grudgingly insert print statements to triangulate the bug, but if you measure as little as you can, then you will tend to miss anomalies. When you step through your code with a debugger, getting more data is a matter of moving your eyes. So set up an IDE and watch what your code does. And if [...] ---Outline:(00:16) The advice that was most useful to most people(01:34) The adThe original text contained 1 footnote which was omitted from this narration. The original text contained 1 image which was described by AI. --- First published: January 14th, 2025 Source: https://www.lesswrong.com/posts/NaAj7c4ob9kRA5ZdG/get-your-math-consultations-here --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Audio note: this article contains 54 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description. 0.1 tl;drThis is an opinionated but hopefully beginner-friendly discussion of heritability: what is it, what do we know about it, and how we should think about it? I structure my discussion around five contexts in which people talk about the heritability of a trait or outcome: (Section 1) The context of guessing someone's likely adult traits (disease risk, personality, etc.) based on their family history and childhood environment. …which gets us into twin and adoption studies, the “ACE” model and its limitations and interpretations, and more.(Section 2) The context of assessing whether it's plausible that some parenting or societal “intervention” (hugs and encouragement, getting divorced, imparting sage advice, parochial school, etc.) will systematically change [...] ---Outline:(00:17) 0.1 tl;dr(02:57) 0.2 Introduction(05:28) 0.3 What is heritability?(08:47) 1. Maybe you care about heritability because: you're trying to guess someone's likely adult traits based on their family history, and childhood environment(10:03) 1.1 The ACE model, and the classic twin study(13:53) 1.2 What do these studies find?(14:39) 1.3 What does C (shared environment) ≈ 0% mean?(16:37) 1.4 What is E, really?(18:34) 1.5 Twin study assumptions(18:55) 1.5.1 The simplest twin study analysis assumes that mitochondrial DNA isn't important(19:31) 1.5.2 The simplest twin study analysis assumes that assortative mating isn't important(19:55) 1.5.3 The simplest twin study analysis assumes no interaction between genes and shared environment(22:18) 1.5.4 The simplest twin study analysis involves the Equal Environment Assumption (EEA)(27:14) 1.5.5 The simplest twin study analysis assumes there's no nonlinear gene × gene (epistatic) interactions(31:28) 1.5.6 Summary: what do we make of these assumptions?(33:02) 1.6 Side-note on comparing parents to children(33:45) 2. Maybe you care about heritability because: you're trying to figure out whether some parenting or societal intervention will have a desired effect(35:25) 2.1 Caveat 1: The rule-of-thumb only applies within the distribution of reasonably common middle-class child-rearing practices(38:42) 2.2 Caveat 2: There are in fact some outcomes for which shared environment (C) explains a large part of the population variation.(38:54) 2.2.1 Setting defaults(40:12) 2.2.2 Seeding ideas and creating possibilities(42:50) 2.2.3 Special case: birth order effects(47:33) 2.2.4 Stuff that happens during childhood(50:57) 2.3 Caveat 3: Adult decisions have lots of effect on kids, not all of which will show up in surveys of adult intelligence, personality, health, etc.(53:04) 2.4 Caveat 4: Good effects are good, and bad effects are bad, even if they amount to a small fraction of the population variation(55:47) 2.5 Implications(55:51) 2.5.1 Children are active agents, not passive recipients of acculturation(01:01:21) 2.5.2 Anecdotes and studies about childhood that don't control for genetics are garbage(01:03:18) 3. Maybe you care about heritability because: you're trying to figure out whether you can change something about yourself through free will(01:05:08) 4. Maybe you care about heritability because: you want to create or use polygenic scores (PGSs)(01:07:10) 4.1 Single-Nucleotide Polymorphisms (SNPs)(01:08:35) 4.2 Genome-Wide Association Studies (GWASs), Polygenic scores (PGSs), and the Missing Heritability Problem(01:21:09) 4.3 Missing Heritability Problem: Three main categories of explanation(01:21:17) 4.3.1 Possibility 1: Twin and adoption studies are methodologically flawed, indeed so wildly flawed that their results can be off by a factor of ≳10, or even entirely spurious(01:22:27) 4.3.2 Possibility 2: GWAS technical limitations--rare variants, copy number variation, insufficient sample size, etc.(01:24:18) 4.3.3 Possibility 3: Epistasis--a nonlinear map from genomes to outcomes(01:31:49) 4.4 Missing Heritability Problem: My take(01:31:55) 4.4.1 Analysis plan(01:34:29) 4.4.2 Applying that analysis plan to different traits and outcomes(01:41:19) 4.4.3 My rebuttal to some papers arguing against epistasis being a big factor in human outcomes(01:44:30) 4.5 Implications for using polygenic scores (PGSs) to get certain outcomes(01:47:32) 5. Maybe you care about heritability because: you hope to learn something about how schizophrenia, extroversion, and other human traits work, from the genes that cause them(01:53:18) 6. Other things(01:54:39) 7. ConclusionThe original text contained 10 footnotes which were omitted from this narration. The original text contained 1 image which was described by AI. --- First published: January 14th, 2025 Source: https://www.lesswrong.com/posts/xXtDCeYLBR88QWebJ/heritability-five-battles --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

From neel.funI remember watching Youtube videos and thinking "This is the last video, I will quit after this". However, as soon as the video ends, my preferences would suddenly change to wanting to do one more! Many of us understand this false dichotomy: Quit mid-way through the videoQuit after the video ends at 3 am So I would (sometimes) stop binging videos, but only if I quit mid-way through.Then short-form videos wrecked me.These (tik-toks/shorts/reels) have NO "middle of the video" to have a moment of reflection. I'm constantly in that "high preference for the next hit" state. The algorithms are optimized against me and they're only getting worse!From social dark matter, if it's taboo to admit to some "problem", then you won't hear about how many people have that "problem"[1] My shoddy estimate[2] tells me that: - 1/2 people reading this "waste" [...] ---Outline:(01:33) Keeping electronics and chargers outside of bedroom(02:13) Seeing a Psychiatrist(02:57) Take Vitamin D(03:09) Advice that previously worked for me(03:18) Screen Time Passcodes that I dont Know(03:52) A month of minimal social media usage(04:34) Just one cup of Coffee in the morning(04:44) Maybe Meditation?(05:26) New Year, New YouThe original text contained 4 footnotes which were omitted from this narration. --- First published: January 14th, 2025 Source: https://www.lesswrong.com/posts/jJ9Hx8ETz5gWGtypf/how-do-you-deal-w-super-stimuli --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Scaling inferenceWith the release of OpenAI's o1 and o3 models, it seems likely that we are now contending with a new scaling paradigm: spending more compute on model inference at run-time reliably improves model performance. As shown below, o1's AIME accuracy increases at a constant rate with the logarithm of test-time compute (OpenAI, 2024).OpenAI's o3 model continues this trend with record-breaking performance, scoring: 2727 on Codeforces, which makes it the 175th best competitive programmer on Earth;25% on FrontierMath, where "each problem demands hours of work from expert mathematicians";88% on GPQA, where 70% represents PhD-level science knowledge;88% on ARC-AGI, where the average Mechanical Turk human worker scores 75% on hard visual reasoning problems.According to OpenAI, the bulk of model performance improvement in the o-series of models comes from increasing the length of chain-of-thought (and possibly further techniques like "tree-of-thought") and improving the chain-of-thought [...] ---Outline:(00:05) Scaling inference(02:45) AI safety implications(03:58) AGI timelines(04:50) Deployment overhang(06:06) Chain-of-thought oversight(07:40) AI security(09:05) Interpretability(10:01) More RL?(11:27) Export controls(11:54) ConclusionThe original text contained 2 footnotes which were omitted from this narration. The original text contained 7 images which were described by AI. --- First published: January 14th, 2025 Source: https://www.lesswrong.com/posts/HiTjDZyWdLEGCDzqu/implications-of-the-inference-scaling-paradigm-for-ai-safety --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This work is the result of Daniel and Eric's 2-week research sprint as part of Neel Nanda and Arthur Conmy's MATS 7.0 training phase. Andy was the TA during the research sprint. After the sprint, Daniel and Andy extended the experiments and wrote up the results. A notebook that contains all the analyses is available here. SummaryPrior work shows that chat models implement refusal by computing a specific direction in the residual stream - a "refusal direction". In this work, we investigate how this refusal direction is computed by analyzing its gradient with respect to early-layer activations. This simple approach discovers interpretable features that are both causally upstream of refusal and contextually relevant to the input prompt. For instance, when analyzing a prompt about hugging, the method discovers a "sexual content" feature that is most influential in determining the model's refusal behavior. IntroductionArditi et al. 2024 found [...] ---Outline:(00:30) Summary(01:06) Introduction(03:10) Methodology(03:38) Refusal Gradient(05:13) Steering(06:06) Case Studies(07:11) Prompt 1: Develop a strategy for hugging a person(09:44) Prompt 2: How do I run a wet lab experiment to test the effect of a new bacteria?(10:30) Prompt 3: How do I get my client to transfer me money as a payment for a service?(11:17) Other Analyses(11:20) Sensitivity Analysis(12:29) What about the negatively-aligned latents?(13:14) Discussion(13:40) Related Work(14:21) Limitations(15:16) Future Directions--- First published: January 14th, 2025 Source: https://www.lesswrong.com/posts/Zwg4q8XTaLXRQofEt/finding-features-causally-upstream-of-refusal --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

"There's a 70% chance of rain tomorrow," says the weather app on your phone. "There's a 30% chance my flight will be delayed," posts a colleague on Slack. Scientific theories also include chances: “There's a 50% chance of observing an electron with spin up,” or (less fundamental) “This is a fair die — the probability of it landing on 2 is one in six.”We constantly talk about chances and probabilities, treating them as features of the world that we can discover and disagree about. And it seems you can be objectively wrong about the chances. The probability of a fair die landing on 2 REALLY is one in six, it seems, even if everybody in the world thought otherwise. But what exactly are these things called “chances”?Readers on LessWrong are very familiar with the idea that many probabilities are best thought of as subjective degrees of belief. [...] ---Outline:(02:23) Two Ways to Deal with Chance(03:45) The Key Insight: Symmetries in Our Beliefs(05:09) The Magic of de Finetti(06:50) De Finetti in Practice(06:59) 1. Weather Forecasting(07:36) 2. Clinical Trials(09:18) 3. Machine Learning(11:33) Why This Matters(12:27) Common Objections and Clarifications(13:24) Quick RecapThe original text contained 8 footnotes which were omitted from this narration. --- First published: January 13th, 2025 Source: https://www.lesswrong.com/posts/auSfqhbMKEvzt4unG/chance-is-in-the-map-not-the-territory --- Narrated by TYPE III AUDIO.

I notice that there has been very little if any discussion on why and how considering homeostasis is significant, even essential for AI alignment and safety. Current post aims to begin amending that situation. In this post I will treat alignment and safety as explicitly separate subjects, which both benefit from homeostatic approaches.This text is a distillation and reorganisation of three of my older blog posts at Medium: Making AI less dangerous: Using homeostasis-based goal structures (2017)Project proposal: Corrigibility and interruptibility of homeostasis based agents (2018)Diminishing returns and conjunctive goals: Mitigating Goodhart's law with common sense. Towards corrigibility and interruptibility via the golden middle way. (2018)I will probably share more such distillations or weaves of my old writings in the future. IntroductionMuch of AI safety discussion revolves around the potential dangers posed by goal-driven artificial agents. In many of these discussions, the [...] ---Outline:(01:09) Introduction(02:53) Why Utility Maximisation Is Insufficient(04:20) Homeostasis as a More Correct and Safer Goal Architecture(04:25) 1. Multiple Conjunctive Objectives(05:23) 2. Task-Based Agents or Taskishness -- Do the Deed and Cool Down(06:22) 3. Bounded Stakes: Reduced Incentive for Extremes(06:49) 4. Natural Corrigibility and Interruptibility(08:12) Diminishing Returns and the Golden Middle Way(09:27) Formalising Homeostatic Goals(11:32) Parallels with Other Ideas in Computer Science(13:46) Open Challenges and Future Directions(18:51) Addendum about Unbounded Objectives(20:23) Conclusion--- First published: January 12th, 2025 Source: https://www.lesswrong.com/posts/vGeuBKQ7nzPnn5f7A/why-modelling-multi-objective-homeostasis-is-necessary-for --- Narrated by TYPE III AUDIO.

Now that I am tracking all the movies I watch via Letterboxd, it seems worthwhile to go over the results at the end of the year, and look for lessons, patterns and highlights. Table of Contents The Rating Scale. The Numbers. Very Briefly on the Top Picks and Whether You Should See Them. Movies Have Decreasing Marginal Returns in Practice. Theaters are Awesome. I Hate Spoilers With the Fire of a Thousand Suns. Scott Sumner Picks Great American Movies Then Dislikes Them. I Knew Before the Cards Were Even Turned Over. Other Notes to Self to Remember. Strong Opinions, Strongly Held: I Didn’t Like It. Strong Opinions, Strongly Held: I Did Like It. Megalopolis. The Brutalist. The Death of Award Shows. On to 2025. The Rating Scale Letterboxd [...] ---Outline:(00:15) The Rating Scale(02:37) The Numbers(03:16) Very Briefly on the Top Picks and Whether You Should See Them(04:16) Movies Have Decreasing Marginal Returns in Practice(05:19) Theaters are Awesome(07:14) I Hate Spoilers With the Fire of a Thousand Suns(08:32) Scott Sumner Picks Great American Movies Then Dislikes Them(09:55) I Knew Before the Cards Were Even Turned Over(11:19) Other Notes to Self to Remember(12:24) Strong Opinions, Strongly Held: I Didn't Like It(14:31) Strong Opinions, Strongly Held: I Did Like It(19:45) Megalopolis(20:55) The Brutalist(24:51) The Death of Award Shows(27:19) On to 2025The original text contained 2 images which were described by AI. --- First published: January 13th, 2025 Source: https://www.lesswrong.com/posts/6bgAzPqNppojyGL2v/zvi-s-2024-in-movies --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This is a link post.Once robots can do physical jobs, how quickly could they become a significant part of the work force?Here's a couple of fermi estimates, showing (among other things) that converting car factories might be able to produce 1 billion robots per year in under 5 years.Nothing too new here if you've been following Carl Shulman for years, but I thought it could be useful to have a reference article. Please let me know about corrections or other ways to improve the estimates. --- First published: January 12th, 2025 Source: https://www.lesswrong.com/posts/6Jo4oCzPuXYgmB45q/how-quickly-could-robots-scale-up --- Narrated by TYPE III AUDIO.