LessWrong (Curated & Popular)

Paul Christiano and Eliezer Yudkowsky discuss disagreements on pivotal acts, take-off speeds, and recursive self-improvement in AI. They also explore addressing risks in transformative AI systems through factored cognition, evaluation challenges, imitation learning, unknown unknowns of deep learning, and disagreements on AI development.

This podcast discusses the limited comprehension of the algorithms implemented by AI models, challenging the misconception that AI's functionality is deliberately programmed. It explores the potential concerns that would arise if the general public were aware of this lack of understanding.

The podcast discusses the limitations and skepticism surrounding Contrast Consistent Search (CCS) and unsupervised consistency-based methods in finding knowledge. It explores simulated entities and the challenge of distinguishing propositional knowledge. It examines the limitations and drawbacks of future CCS-like approaches and the challenges with unsupervised LLM knowledge discovery, including bugs and generalization failures. The podcast suggests criteria for evaluating ELK methods and the need for suitable test beds for evaluation.

This is a linkpost for https://www.narrativeark.xyz/p/succession“A table beside the evening sea where you sit shelling pistachios, flicking the next open with the half- shell of the last, story opening story, on down to the sandy end of time.” V1: LeavingDeceleration is the hardest part. Even after burning almost all of my fuel, I’m still coming in at 0.8c. I’ve planned a slingshot around the galaxy's central black hole which will slow me down even further, but at this speed it’ll require incredibly precise timing. I’ve been optimized hard for this, with specialized circuits for it built in on the hardware level to reduce latency. Even so, less than half of slingshots at this speed succeed—most probes crash, or fly off trajectory and are left coasting through empty space.I’ve already beaten the odds by making it here. Intergalactic probes travel so fast, and so far, that almost all [...]--- First published: December 20th, 2023 Source: https://www.lesswrong.com/posts/CAzntXYTEaNfC9nB6/succession Linkpost URL:https://www.narrativeark.xyz/p/succession --- Narrated by TYPE III AUDIO.

Recently, Ben Pace wrote a well-intentioned blog post mostly based on complaints from 2 (of 21) Nonlinear employees who 1) wanted more money, 2) felt socially isolated, and 3) felt persecuted/oppressed. Of relevance, one has accused the majority of her previous employers, and 28 people of abuse - that we know of. She has accused multiple people of threatening to kill her and literally accused an ex-employer of murder. Within three weeks of joining us, she had accused five separate people of abuse: not paying her what was promised, controlling her romantic life, hiring stalkers, and other forms of persecution. We have empathy for her. Initially, we believed her too. We spent weeks helping her get her “nefarious employer to finally pay her” and commiserated with her over how badly they mistreated her. Then she started accusing us of strange things. You’ve seen Ben's evidence, which [...]--- First published: December 12th, 2023 Source: https://www.lesswrong.com/posts/q4MXBzzrE6bnDHJbM/nonlinear-s-evidence-debunking-false-and-misleading-claims --- Narrated by TYPE III AUDIO.

The New York Times Picture a scene: the New York Times is releasing an article on Effective Altruism (EA) with an express goal to dig up every piece of negative information they can find. They contact Émile Torres, David Gerard, and Timnit Gebru, collect evidence about Sam Bankman-Fried, the OpenAI board blowup, and Pasek's Doom, start calling Astral Codex Ten (ACX) readers to ask them about rumors they'd heard about affinity between Effective Altruists, neoreactionaries, and something called TESCREAL. They spend hundreds of hours over six months on interviews and evidence collection, paying Émile and Timnit for their time and effort. The phrase "HBD" is muttered, but it's nobody's birthday. A few days before publication, they present key claims to the Centre for Effective Altruism (CEA), who furiously tell them that many of the claims are provably false and ask for a brief delay to demonstrate the falsehood of [...]The original text contained 16 footnotes which were omitted from this narration. --- First published: December 19th, 2023 Source: https://www.lesswrong.com/posts/2vNHiaTb4rcA8PgXQ/effective-aspersions-how-the-nonlinear-investigation-went --- Narrated by TYPE III AUDIO.

At the Bay Area Solstice, I heard the song Bold Orion for the first time. I like it a lot. It does, however, have one problem:He has seen the rise and fall of kings and continents and all, Rising silent, bold Orion on the rise.Orion has not witnessed the rise and fall of continents. Constellations are younger than continents.The time scale that continents change on is ten or hundreds of millions of years.The time scale that stars the size of the sun live and die on is billions of years. So stars are older than continents.But constellations are not stars or sets of stars. They are the patterns that stars make in our night sky.The stars of some constellations are close together in space, and are gravitationally bound together, like the Pleiades. The Pleiades likely have been together, and will stay close [...]The original text contained 1 footnote which was omitted from this narration. --- First published: December 19th, 2023 Source: https://www.lesswrong.com/posts/YMakfmwZsoLdXAZhb/constellations-are-younger-than-continents --- Narrated by TYPE III AUDIO.

Many thanks to Samuel Hammond, Cate Hall, Beren Millidge, Steve Byrnes, Lucius Bushnaq, Joar Skalse, Kyle Gracey, Gunnar Zarncke, Ross Nordby, David Lambert, Simeon Campos, Bogdan Ionut-Cirstea, Ryan Kidd, Eric Ho, and Ashwin Acharya for critical comments and suggestions on earlier drafts of this agenda, as well as Philip Gubbins, Diogo de Lucena, Rob Luke, and Mason Seale from AE Studio for their support and feedback throughout. TL;DR  Our initial theory of change at AE Studio was a 'neglected approach' that involved rerouting profits from our consulting business towards the development of brain-computer interface (BCI) technology to dramatically enhance human agency, better enabling us to do things like solve alignment. Now, given shortening timelines, we're updating our theory of change to scale up our technical alignment efforts.With a solid technical foundation in BCI, neuroscience, and machine learning, we are optimistic that we’ll be able to contribute meaningfully [...]The original text contained 6 footnotes which were omitted from this narration. --- First published: December 18th, 2023 Source: https://www.lesswrong.com/posts/qAdDzcBuDBLexb4fC/the-neglected-approaches-approach-ae-studio-s-alignment --- Narrated by TYPE III AUDIO.

When discussing AGI Risk, people often talk about it in terms of a war between humanity and an AGI. Comparisons between the amounts of resources at both sides' disposal are brought up and factored in, big impressive nuclear stockpiles are sometimes waved around, etc.I'm pretty sure it's not how that'd look like, on several levels. 1. Threat AmbiguityI think what people imagine, when they imagine a war, is Terminator-style movie scenarios where the obviously evil AGI becomes obviously evil in a way that's obvious to everyone, and then it's a neatly arranged white-and-black humanity vs. machines all-out fight. Everyone sees the problem, and knows everyone else sees it too, the problem is common knowledge, and we can all decisively act against it.[1]But in real life, such unambiguity is rare. The monsters don't look obviously evil, the signs of fatal issues are rarely blatant. Is this whiff [...]The original text contained 1 footnote which was omitted from this narration. --- First published: December 16th, 2023 Source: https://www.lesswrong.com/posts/xSJMj3Hw3D7DPy5fJ/humanity-vs-agi-will-never-look-like-humanity-vs-agi-to --- Narrated by TYPE III AUDIO.

Epistemic status: Speculation. An unholy union of evo psych, introspection, random stuff I happen to observe & hear about, and thinking. Done on a highly charged topic. Caveat emptor!Most of my life, whenever I'd felt sexually unwanted, I'd start planning to get fit.Specifically to shape my body so it looks hot. Like the muscly guys I'd see in action films.This choice is a little odd. In close to every context I've listened to, I hear women say that some muscle tone on a guy is nice and abs are a plus, but big muscles are gross — and all of that is utterly overwhelmed by other factors anyway.It also didn't match up with whom I'd see women actually dating.But all of that just… didn't affect my desire?There's a related bit of dating advice for guys. "Bro, do you even lift?" Depending on the [...]--- First published: December 13th, 2023 Source: https://www.lesswrong.com/posts/nvmfqdytxyEpRJC3F/is-being-sexy-for-your-homies --- Narrated by TYPE III AUDIO.