LessWrong (30+ Karma)

Delve into the intriguing world of nonconsent fantasies with surprising statistics revealing a significant number of individuals harboring such thoughts. Explore how these preferences often transcend mere fantasies and shape broader dating behaviors. The discussion uncovers the paradox of consent—often perceived as a turn-off—in romantic pursuits. Discover how subtle courtship dynamics and indirect signaling play a role in the dating landscape. Ultimately, the conversation ties these elements together, shedding light on the peculiarities of modern mating practices.

2.1 Summary In the last post, I introduced model-based RL, which is the frame we will use to analyze the alignment problem, and we learned that the critic is trained to predict reward. I already briefly mentioned that the alignment problem is centrally about making the critic assign high value to outcomes we like and low value to outcomes we don’t like. In this post, we’re going to try to get some intuition for what values a critic may learn, and thereby also learn about some key difficulties of the alignment problem. Section-by-section summary: 2.2 The Distributional Leap: The distributional leap is the shift from the training domain to the dangerous domain (where the AI could take over). We cannot test safety in that domain, so we need to predict how values generalize. 2.3 A Naive Training Strategy: We set up a toy example: a model-based RL chatbot trained on human feedback, where the critic learns to predict reward from the model's internal thoughts. This isn't meant as a good alignment strategy—it's a simplified setup for analysis. 2.4 What might the critic learn?: The critic learns aspects of the model's thoughts that correlate with reward. We analyze whether [...] ---Outline:(00:16) 2.1. Summary(03:48) 2.2. The Distributional Leap(05:26) 2.3. A Naive Training Strategy(07:01) 2.3.1. How this relates to current AIs(08:26) 2.4. What might the critic learn?(09:55) 2.4.1. Might the critic learn to score honesty highly?(12:35) 2.4.1.1. Aside: Contrast to the human value of honesty(13:05) 2.5. Niceness is not optimal(14:59) 2.6. Niceness is not (uniquely) simple(16:02) 2.6.1. Anthropomorphic Optimism(19:26) 2.6.2. Intuitions from looking at humans may mislead you(21:12) 2.7. Natural Abstractions or Alienness?(21:35) 2.7.1. Natural Abstractions(23:15) 2.7.2. ... or Alienness?(25:54) 2.8. Value extrapolation(27:49) 2.8.1. Coherent Extrapolated Volition(32:48) 2.9. Conclusion The original text contained 11 footnotes which were omitted from this narration. --- First published: January 2nd, 2026 Source: https://www.lesswrong.com/posts/vv6QojgvprM4jYJLr/advanced-intro-to-ai-alignment-2-what-values-may-an-ai-learn --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Previously, I wrote about how I was considering going back to San Francisco State University for two semesters to finish up my Bachelor's degree in math. So, I did that. I think it was a good decision! I got more out of it than I expected. To be clear, "better than I expected" is not an endorsement of college. SF State is still the same communist dystopia I remember from a dozen years ago—a bureaucratic command economy dripping in propaganda about how indispensible and humanitarian it is, whose subjects' souls have withered to the point where, even if they don't quite believe the propaganda, they can't conceive of life and work outside the system. But it didn't hurt this time, because I had a sense of humor about it now—and a sense of perspective (thanks to life experience, no thanks to school). Ultimately, policy debates should not appear one-sided: if things are terrible, it's probably not because people are choosing the straightforwardly terrible thing for no reason whatsoever, with no trade-offs, coordination problems, or nonobvious truths making the terrible thing look better than it is. The thing that makes life under communism unbearable is the fact [...] ---Outline:(01:45) Doing It Right This Time (Math)(02:31) Real Analysis II (Fall 2024)(11:48) Probability Models (Fall 2024)(19:15) The 85th William Lowell Putnam Mathematical Competition(20:51) Measure and Integration (Spring 2025)(25:04) Theory of Functions of a Complex Variable (Spring 2025)(33:16) Modern Algebra I (Spring 2025)(38:59) Not Sweating the Fake Stuff (Non-Math)(39:41) Queer Literatures and Media (Fall 2024)(49:31) Philosophy of Animals (Spring 2025)(55:12) Self, Place, and Knowing: An Introduction to Interdisciplinary Inquiry (Spring 2025)(01:01:31) Cynicism and Sanity --- First published: January 1st, 2026 Source: https://www.lesswrong.com/posts/hHEvfp7kwvaHwgcdH/college-was-not-that-terrible-now-that-i-m-not-that-crazy --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This discussion dives into AI predictions for 2025, analyzing claims from past years. Participants explore the trend of overestimating AI advancements. A notable prompt from 2023 is evaluated, revealing the speaker's misconceptions. The conversation touches on Elon Musk's ambitious forecasts, which largely fall flat. Future implications of AI’s role in economy and work hours by 2030 are considered. Additionally, there’s a fascinating mix of optimism and skepticism about the timeline for achieving AGI and its potential impacts.

Late in 2019 I, like many of my rationalist friends purchased the parts for and assembled a genuine, bona fide LUMENATOR™️ - a device for greatly increasing the brightness of your home - according to the original specification. To me, lumenators are the quintessetial application of the More Dakka mindset: when you face a problem that responds positively to a little bit of X and responds even more positively to larger amounts of X, you don't just stop applying X once you feel you've done a reasonable amount of it, you add more and more and more until your problem goes away or X stops working. I built a lumenator not for seasonal affective disorder, but because it helps me wake up feeling refreshed in the morning and I feel happy when my house is very bright inside. In 2019 I lived in a small group house in Waterloo, ON and we'd often give people directions to our house like "turn the corner and then look for the one with ridiuclously bright light streaming from the windows". They'd show up without trouble and remark: "Wow, I didn't actually expect I'd be able to find your place based on those [...] --- First published: December 31st, 2025 Source: https://www.lesswrong.com/posts/p869mKDGL3onYsGYt/lumenator-2-0 --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

TL;DR: Most AI forecasts generally assume that if a conflict over Taiwan occurs, it will largely be about AI. I think there's a decent chance for a conflict before either side becomes substantially AGI-pilled. Thanks to Aaron Scher for comments on a draft of this post. I'm no China expert, but a lot of China experts seem pretty concerned about the possibility of a conflict over Taiwan. China is currently engaged in a massive military buildup and modernization effort, it's building specialized invasion barges like the Mulberry harbors used in the WWII Normandy landings, and it's conducting amphibious landing exercises with civilian roll-on/roll-off vehicle ferries, many of which China modifies for potential military use. Increasingly frequent military exercises around Taiwan could let China rapidly transition to a full blockade. Its internal propaganda suggests that Taiwanese "provocations" could justify military action, and its leadership continually talk about Taiwan's "return to China", with some even openly discussing "reeducation". By some cosmic coincidence, 2027, the PLA's centennial, is sometimes identified as the year when the PLA hopes to be ready for a conflict over Taiwan. This doesn't mean China will immediately pull the trigger, but they might want to be prepared by [...] ---Outline:(04:25) Impacts on compute(07:41) Securitization(08:20) Conclusion The original text contained 6 footnotes which were omitted from this narration. --- First published: January 1st, 2026 Source: https://www.lesswrong.com/posts/ozKqPoA3qhmrhZJ7t/taiwan-war-timelines-might-be-shorter-than-ai-timelines --- Narrated by TYPE III AUDIO.

Herre are the results of the New Year Write-Like-Lsusr Masquerade. 3% Meditations on Suffering Meditations on Suffering by MeditationsOnShrimp technically satisfies all of the competition criteria. Congratulations on beating the ~8 billion people who did not participate in this tournament. You are in the top 0.00000002%. MeditationsOnShrimp wins this participation trophy. 🏆 8% [Book Review] "Reality+" by David Chalmers This post felt familiar. Something about the facing pacing felt like the kind of thing I would write in the early phase of my writing, when I had only 2-3 years of blogging experience rather than the 6 years I have now, back before I learned how to wield indirection and deliberate ambiguity. I think this post was written by someone who has read a lot of my writings and even watched some of the videos. 8% The Benefits of Meditation Come From Telling People That You Meditate This post is obviously not written by the real lsusr because it demonstrates a level of self-awareness and humility far beyond the real lsusr. I don't think this was a serious competition entry. I think this was is a savage takedown that uses the competition as a pretext to make fun of [...] ---Outline:(00:17) 3% Meditations on Suffering(00:45) 8% \[Book Review\] Reality+ by David Chalmers(01:13) 8% The Benefits of Meditation Come From Telling People That You Meditate(01:49) 9% Rules clarification for the Write like lsusr competition(02:18) 10% The Ones who Feed their Children(02:27) 19% How To Create A Lsusr Golem(02:50) 23% Human Values(03:22) 48% Orpheus Basilisk(03:44) 97% \[Book Review\] * → 🚹 → * --- First published: January 1st, 2026 Source: https://www.lesswrong.com/posts/nDiijZnPaLZdTttS3/usd500-write-like-lsusr-competition-results --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

There's many debates about "what counts as AGI" or "what counts as superintelligence?". Some people might consider those arguments "goalpost moving." Some people were using "superintelligence" to mean "overwhelmingly smarter than humanity", and using it to mean "spikily good at some coding tasks while still not really successfully generalizing or maintaining focus" feels like watering it down. I think there's just actually a wide range of concepts that need to get talked about. And, right now, most of the AIs that people will wanna talk about are kinda general and kinda superintelligent and kinda aligned. If you have an specific concept you wanna protect, I think it's better to just give it a clunky name that people don't want to use in casual conversation, rather than pumping against entropy to defend a simple term that could be defined to mean other things. Previously OpenPhil had used "Transformative AI" to mean "AI that is, you know, powerful enough to radically transform society, somehow." I think that's a useful term. But, it's not exactly what If Anyone Builds It is cautioning about. The type of AI I'm most directly worried about is "overwhelmingly superhuman compared to humanity." (And, AIs that [...] --- First published: January 1st, 2026 Source: https://www.lesswrong.com/posts/64RAfxjgxnHfFhbh5/overwhelming-superintelligence --- Narrated by TYPE III AUDIO.

Prior work has examined 2-hop latent (by "latent" I mean: the model must answer immediately without any Chain-of-Thought) reasoning (e.g., "What element has atomic number (the age at which Tesla died)?") and found that LLM performance was limited aside from spurious successes (from memorization and shortcuts). I find that recent LLMs can now do 2-hop and 3-hop latent reasoning with moderate accuracy. I construct a new dataset for evaluating n-hop latent reasoning on natural facts (as in, facts that LLMs already know). On this dataset, I find that Gemini 3 Pro gets 60% of 2-hop questions right and 34% of 3-hop questions right. Opus 4 performs better than Opus 4.5 at this task; Opus 4 gets 31% of 2-hop questions right and 7% of 3-hop questions right. All models I evaluate have chance or near chance accuracy on 4-hop questions. Older models perform much worse; for instance, GPT-4 gets 9.7% of 2-hop questions right and 3.9% of 3-hop questions right. I believe this new dataset I've created is the best existing dataset for evaluating n-hop latent reasoning. According to Balesni et al., prior datasets based on natural facts had issues with spurious successes while synthetic facts (introduced to [...] ---Outline:(06:22) Appendix: resolving Leo Gaos Manifold questions about fact composition(10:40) Appendix: The effect of problem repeats and few-shots on these fact composition questions(12:43) Appendix: adding the result of N 1-hop questions(17:41) Appendix: Effects of using a different fact distribution(18:31) Appendix: what 3-hop questions is Gemini 3 Pro getting right?(21:51) Appendix: Dataset description(30:02) Dataset sanity checks(30:37) Appendix: AI usage The original text contained 7 footnotes which were omitted from this narration. --- First published: January 1st, 2026 Source: https://www.lesswrong.com/posts/aYtrLhoZtCKZnfBvA/recent-llms-can-do-2-hop-and-3-hop-latent-no-cot-reasoning --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Seeing this post and its comments made me a bit concerned for young people around this community. I thought I would try to write down why I believe most folks who read and write here (and are generally smart, caring, and knowledgable) will be OK. I agree that our society often is under prepared for tail risks. As a general planner, you should be worrying about potential catastrophes even if their probability is small. However as an individual, if there is a certain probability X of doom that is beyond your control, it is best to focus on the 1-X fraction of the probability space that you control rather than constantly worrying about it. A generation of Americans and Russians grew up under a non-trivial probability of a total nuclear war, and they still went about their lives. Even when we do have some control over possibility of very bad outcomes (e.g., traffic accidents), it is best to follow some common sense best practices (wear a seatbelt, don't drive a motorcycle) but then put that out of your mind. I do not want to engage here in the usual debate of P[doom]. But just as it makes absolute sense [...] --- First published: December 31st, 2025 Source: https://www.lesswrong.com/posts/fwQburGDyGoSSweT9/you-will-be-ok --- Narrated by TYPE III AUDIO.