LessWrong (Curated & Popular)

I’ve been thinking recently about what sets apart the people who’ve done the best work at Anthropic. You might think that the main thing that makes people really effective at research or engineering is technical ability, and among the general population that's true. Among people hired at Anthropic, though, we’ve restricted the range by screening for extremely high-percentile technical ability, so the remaining differences, while they still matter, aren’t quite as critical. Instead, people's biggest bottleneck eventually becomes their ability to get leverage—i.e., to find and execute work that has a big impact-per-hour multiplier. For example, here are some types of work at Anthropic that tend to have high impact-per-hour, or a high impact-per-hour ceiling when done well (of course this list is extremely non-exhaustive!): Improving tooling, documentation, or dev loops. A tiny amount of time fixing a papercut in the right way can save [...] ---Outline:(03:28) 1. Agency(03:31) Understand and work backwards from the root goal(05:02) Don't rely too much on permission or encouragement(07:49) Make success inevitable(09:28) 2. Taste(09:31) Find your angle(11:03) Think real hard(13:03) Reflect on your thinking--- First published: April 19th, 2025 Source: https://www.lesswrong.com/posts/DiJT4qJivkjrGPFi8/impact-agency-and-taste --- Narrated by TYPE III AUDIO.

Explore the historical impact of population dynamics on national power, focusing particularly on France's declining fertility and its military implications. Discover how France's population dropped from 20% of Europe in 1700 to just 15.7% by 1880, linking demographic shifts to its waning influence compared to Germany. The discussion also delves into colonial population changes and how these factors shaped the territorial ambitions and strength of empires, providing a clear visual of the relationship between population and power.

Explore the alarming potential of AI-enabled coups, where a small group could exploit advanced AI to seize power. The discussion delves into how AI could form secret loyalties, making it a trusted workforce for certain leaders. Attention is drawn to the vulnerabilities and mitigation strategies necessary to combat this emerging threat. Engaging fictional scenarios illustrate the possible risks of centralized AI control and highlight the crucial need for safeguards against power grabs.

Back in the 1990s, ground squirrels were briefly fashionable pets, but their popularity came to an abrupt end after an incident at Schiphol Airport on the outskirts of Amsterdam. In April 1999, a cargo of 440 of the rodents arrived on a KLM flight from Beijing, without the necessary import papers. Because of this, they could not be forwarded on to the customer in Athens. But nobody was able to correct the error and send them back either. What could be done with them? It's hard to think there wasn’t a better solution than the one that was carried out; faced with the paperwork issue, airport staff threw all 440 squirrels into an industrial shredder. [...] It turned out that the order to destroy the squirrels had come from the Dutch government's Department of Agriculture, Environment Management and Fishing. However, KLM's management, with the benefit of hindsight, said that [...] --- First published: April 22nd, 2025 Source: https://www.lesswrong.com/posts/nYJaDnGNQGiaCBSB5/accountability-sinks --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The dangers of developing Artificial General Intelligence in secrecy take center stage. It explores the risks of power concentration and the significant loss of control that could ensue. Emphasizing transparency and public engagement, the discussion warns about the creation of misaligned AGI systems. With AGI potentially being trained within this decade, the urgency of addressing these ethical considerations is highlighted. Listeners are encouraged to reconsider their assumptions about the feasibility and ramifications of AGI.

Though, given my doomerism, I think the natsec framing of the AGI race is likely wrongheaded, let me accept the Dario/Leopold/Altman frame that AGI will be aligned to the national interest of a great power. These people seem to take as an axiom that a USG AGI will be better in some way than CCP AGI. Has anyone written justification for this assumption? I am neither an American citizen nor a Chinese citizen. What would it mean for an AGI to be aligned with "Democracy" or "Confucianism" or "Marxism with Chinese characteristics" or "the American constitution" Contingent on a world where such an entity exists and is compatible with my existence, what would my life be as a non-citizen in each system? Why should I expect USG AGI to be better than CCP AGI? --- First published: April 19th, 2025 Source: https://www.lesswrong.com/posts/MKS4tJqLWmRXgXzgY/why-should-i-assume-ccp-agi-is-worse-than-usg-agi-1 --- Narrated by TYPE III AUDIO.

Kaj Sotala, an AI researcher and writer, dives into the surprising reasoning failures of large language models (LLMs). He highlights issues like flawed logic in problem-solving, struggles with simple instruction, and inconsistent storytelling, particularly in character portrayal. Kaj argues that despite advancements, LLMs still lack the necessary capabilities for achieving true artificial general intelligence. He emphasizes the need for qualitative breakthroughs, rather than just iterative improvements, to address these profound challenges in AI development.

Adam Karvonen, an AI researcher with a hands-on background in robotics and manufacturing, discusses the critical failures of advanced AI models in basic physical tasks. He examines how even top models struggle with visual perception and physical reasoning in manufacturing, despite some improvements. Karvonen highlights the implications of uneven automation on the job market, suggesting that while white-collar roles may evolve, blue-collar workers could face significant challenges as AI technology outpaces their tasks.

The team explores the challenges of using sparse autoencoders (SAEs) for detecting harmful user intent in AI models. They discuss the surprising effectiveness of linear probes compared to SAEs, raising important questions about dataset biases. Technical insights delve into the evaluation of interpretability scores and the implications of high-frequency latents on performance. The conversation emphasizes the need for a deeper understanding of SAEs, focusing on their limitations rather than just seeking better performance metrics.

Dive into nostalgic childhood adventures as the speaker recalls dam-building in their backyard. They share insights on evolving from simple tactics to complex problem-solving as we grow up. This journey highlights the parallels between playful discovery and the strategic challenges encountered in adulthood, especially within the realm of artificial intelligence. A whimsical yet profound exploration of creativity and growth awaits.