Astral Codex Ten Podcast

https://astralcodexten.substack.com/p/practically-a-book-review-yudkowsky   I. The story thus far: AI safety, which started as the hobbyhorse of a few weird transhumanists in the early 2000s, has grown into a medium-sized respectable field. OpenAI, the people responsible for GPT-3 and other marvels, have a safety team. So do DeepMind, the people responsible for AlphaGo, AlphaFold, and AlphaWorldConquest (last one as yet unreleased). So do Stanford, Cambridge, UC Berkeley, etc, etc. Thanks to donations from people like Elon Musk and Dustin Moskowitz, everyone involved is contentedly flush with cash. They all report making slow but encouraging progress. Eliezer Yudkowsky, one of the original weird transhumanists, is having none of this. He says the problem is harder than everyone else thinks. Their clever solutions will fail. He's been flitting around for the past few years, Cassandra-like, insisting that their plans will explode and they are doomed.

https://astralcodexten.substack.com/p/theres-a-time-for-everyone Last week I got married. I met her two years ago, at one of (our mutual friend) Aella’s weird parties. Not this one, a different one. I was at this one too though. It was great. Our first date, we talked about Singapore’s child tax credits, which gave me advanced notice of where her mind was at. Our second date, we talked about category formation in borderline personality disorder, which later became this post. Our third date, we talked about why Inuit suicide rates were so high, which later became this post. Then COVID hit. We switched our dates to a Minecraft virtual world, where we built a house together. At the time, I completely missed the kabbalistic significance of this. I don’t usually talk about my personal life on here. But I feel like I owe you guys this one, because, well, some of you have been reading this blog a long time. And some of my earliest posts (eg) were me complaining about the dating world, and how tough it was to meet anybody or even to stay sane. And you guys were kind to me, and commiserated with me, and shared your own experiences. I feel an obligation to check in with the rest of you, to celebrate those of you who have also succeeded and empathize with those of you who haven’t yet. Maybe I’m not a success story here, exactly. I’m getting married at 37, a lot later than I would have liked. And my story involved parts that probably don’t replicate well, like becoming a niche Internet microcelebrity whose readers sometimes invite him to things despite his many social inadequacies.

https://astralcodexten.substack.com/p/highlights-from-the-comments-on-dont Lots of people thought I was being unfair to the movie. G. Retriever writes: I TOTALLY disagree with your reading of the movie. To me it was a description of a social dynamic that makes even very straightforward problems impossible to focus on collectively, a tragedy of the commons where "the commons" is basically "attention". Even the experts get sucked into the vortex, nobody comes out clean, and in the end everyone gets killed. Batislu: Hmm .. I didn't come away from Don't Look Up with the message of "Trust The Experts". Rather I came away with a sense of futility that we're doomed as a species due to our inability to discover and form consensus around the truth. I thought the movie did a great job of relaying that, given that humanity is completely wiped out by the end. Erik Hoel: If we broaden our scope from the obvious mappings (Female President onto Trump) and admit that pure satires don't make the best cinema, at its broadest, it's a movie about institutional failure. Across party lines (though it skewers one more than the other, sure). It's for this reason it felt fresh to me and that I liked it. Institutional failure, even human failure, is becoming more and more obvious, as it's undeniable that our institutions, from academia to the White House, are more sclerotic and incapable and, well, foolish, than they either were in the past or appeared to be. And to me this movie was like an expression of America's Id realizing that over the past several years. Steph:

https://astralcodexten.substack.com/p/movie-review-dont-look-up I. Don’t Look Up is primarily a movie about existential risk, and many great people have already reviewed it as such. I’m going to be less virtuous and use it as a springboard to talk about politics. But first, the plot in a nutshell: Male Scientist and Female Scientist discover a comet will hit Earth in six months. They contact the relevant authorities, Black Scientist and Asian Scientist, and go to meet the President (who, despite being a woman, is Donald Trump). The President says scientists are always doomsaying, if people get too panicked she’ll lose the midterm election, and she’ll get around to dealing with this later. (the Earth, at this point, has five months and however many days left)

https://astralcodexten.substack.com/p/lewis-carroll-invented-retroactive Retroactive public goods funding is one of those ideas that’s so great people can’t stop reinventing it. I know of at least five independent inventions under five different names: “social impact bonds” by a New Zealand economist in 1988, “certificates of impact” by Paul Christiano in 2014, “retroactive public goods funding” by Vitalik Buterin a few years ago, “EA loans” by a blogger who prefers to remain anonymous, and “venture grants” by Mako Yass. These aren’t all exactly the same idea. Some are slightly better framed than others and probably I’m being terribly disrespectful to the better ones by saying they’re the same as the worse ones. But I think they all share a basic core: some structure that lets profit-seeking venture capitalist types invest in altruistic causes, in the hopes that altruists will pay them back later once they’ve been shown to work. Upon re-reading some old SSC comments, I found a gem I’d missed the first time around: Julie K says that the actual first person to invent this idea was Lewis Carroll (aka author of Alice in Wonderland) back in 1894. She quotes from his book Sylvie and Bruno:

https://astralcodexten.substack.com/p/links-for-december   [Remember, I haven’t independently verified each link. On average, commenters will end up spotting evidence that around two or three of the links in each links post are wrong or misleading. I correct these as I see them, and will highlight important corrections later, but I can’t guarantee I will have caught them all by the time you read this.] 1: List Of Games That Buddha Would Not Play. 2: Claim via NPR: When Brazil had high inflation in the 1990s, some economists developed a plan: price everything in inflation-adjusted units, so that people felt like things were “stable”, then declare that the Inflation Adjusted Unit was the new currency. How Fake Money Saved Brazil. Also interesting: they tried it because the new finance minister knew no economics, recognized his ignorance, and was willing to call up random economists and listen to their hare-brained plans. 3: In the 19th century, a group of Tibeto-Burman-speaking former headhunters along the India/Burma border declared themselves the descendants of Manasseh (one of the Ten Lost Tribes) and converted en masse to Judaism. In 2005, the Chief Rabbinate of Israel accepted their claim and expedited immigration paperwork for several thousand of them.

https://astralcodexten.substack.com/p/acx-grants-results Thanks to everyone who participated in ACX Grants, whether as an applicant, an evaluator, or a funder. Before I announce awardees, a caveat: this was hard in lots of ways I didn't expect. I got 656 applications addressing different problems and requiring different skills to judge. I'll write a long post on it later, but the part I want to emphasize now is: if I didn't grant you money, it doesn't mean I didn't like your project. Sometimes it meant I couldn't find someone qualified to evaluate it. Other times a reviewer was concerned that if you were successful, your work might be used by terrorists / dictators / AI capabilities researchers / Republicans and cause damage in ways you couldn't foresee. Other times it meant it was a better match for some other grant organization and I handed it off to them. Still other times, my grant reviewers tied themselves up in knots with 4D chess logic like "if they're smart enough to attempt this project, they're smart enough to know about XYZ Grants which is better suited for them, which means they're mostly banking on XYZ funding and using you as a backup, but if XYZ doesn't fund these people then that's strong evidence that they shouldn't be funded, so even though everything about them looks amazing, please reject them." I have no idea if things really work this way, but I needed some experienced grant reviewers on board and they were all like this. I took these considerations seriously and in some marginal cases they prevented funding.

https://astralcodexten.substack.com/p/mantic-monday-dogs-in-wizard-hats   I found this YouTube explainer about prediction markets on the subreddit. It’s pretty good! My small nitpicks are that it overestimates their accuracy relative to traditional forecasters (it focuses on markets beating forecasters in 2008, but I don’t think this is consistent) and underestimates their resilience against bad actors trying to skew the probabilities. Still, this will be my go-to source when someone wants a short explanation of what these are and why I’m so excited. Futuur Soon Last week I mentioned a new prediction market called Futuur . Today we’ll look at it in more depth. Futuur sends non-Americans to their real money markets and Americans to their play money markets (because of the US’ unique anti-prediction-market regulations). Their play money markets are awful:

https://astralcodexten.substack.com/p/highlights-from-the-comments-on-diseasonality   The main highlight was an email I got from a reader who prefers to remain anonymous, linking me to Projecting The Transmission Dynamics Of SARS-CoV2. This paper is head and shoulders above anything I found during my own literature review and just comes out and says everything painfully tried to piece together. Either my research skills suck, the epidemiology literature is a bunch of disparate subthreads with wildly differing levels of competence, or both. The authors (including Marc Lipsitch who some of you might know from Twitter) are writing in May 2020, trying to predict the future course of COVID. To that end, they investigate the past course of two other coronaviruses called OC43 and HKU1, which cause mild colds. These show a seasonal pattern. Why?

https://astralcodexten.substack.com/p/addendum-to-no-evidence-post The day after I wrote The Phrase “No Evidence” Is A Red Flag For Bad Science Communication, FT published this article: Like many uses of “no evidence”, they meant that one particular study of this complicated question had failed to reject the null hypothesis. Here’s what happened to Metaculus’ prediction tournament when the same study came out: The consensus prediction dropped from 72% chance that it was less lethal, to 63% chance. But it quickly recovered, and is now up to 80%. This is an unusually clear example of the difference between classical and Bayesian ways of thinking.