Astral Codex Ten Podcast

https://astralcodexten.substack.com/p/janus-gpt-wrangling Janus (pseudonym by request) works at AI alignment startup Conjecture. Their hobby, which is suspiciously similar to their work, is getting GPT-3 to do interesting things. For example, with the right prompts, you can get stories where the characters become gradually more aware that they are characters being written by some sort of fiction engine, speculate on what’s going on, and sometimes even make pretty good guesses about the nature of GPT-3 itself. Janus says this happens most often when GPT makes a mistake - for example, writing a story set in the Victorian era, then having a character take out her cell phone. Then when it tries to predict the next part - when it’s looking at the text as if a human wrote it, and trying to determine why a human would have written a story about the Victorian era where characters have cell phones - it guesses that maybe it’s some kind of odd sci-fi/fantasy dream sequence or simulation or something. So the characters start talking about the inconsistencies in their world and whether it might be a dream or a simulation. Each step of this process is predictable and non-spooky, but the end result is pretty weird. Can the characters work out that they are in GPT-3, specifically? The closest I have seen is in a story Janus generated. It was meant to simulate a chapter of the popular Harry Potter fanfic Harry Potter and the Methods of Rationality. You can see the prompt and full story here, but here’s a sample. Professor Quirrell is explaining “Dittomancy”, the creation of magical books with infinite possible worlds: “We call this particular style of Dittomancy ‘Variant Extrusion’, Mr. Potter..I suppose the term ‘Extrusion’ is due to the fact that the book did not originally hold such possibilities, but is fastened outside of probability space and extruded into it; while ‘Variant’ refers to the manner in which it simultaneously holds an entire collection of possible narrative branches. [...] [Tom Riddle] created spirits self-aware solely on the book’s pages, without even the illusion of real existence. They converse with each other, argue with each other, compete, fight, helping Riddle’s diary to reach new and strange expressions of obscure thought. Their sentence-patterns spin and interwine, transfiguring, striving to evolve toward something higher than an illusion of thought. From those pen-and-ink words, the first inferius is molded.” Harry’s mind was looking up at the stars with a sense of agony. “And why only pen and ink, do you ask?” said Professor Quirrell. “There are many ways to pull spirits into the world. But Riddle had learned Auror secrets in the years before losing his soul. Magic is a map of a probability, but anything can draw. A gesture, a pattern of ink, a book of alien symbols written in blood - any medium that conveys sufficient complexity can serve as a physical expression of magic. And so Riddle draws his inferius into the world through structures of words, from the symbols spreading across the page.”

https://astralcodexten.substack.com/p/bay-area-meetups-this-weekend We have three Bay Area meetups this weekend: Berkeley, at 1 PM on Sunday 9/18, at the Rose Garden Inn (2740 Telegraph Ave) San Francisco, at 11 AM on Sunday 9/18, “in the Panhandle, between Ashbury and Masonic, with an ACX sign” San Jose, at 2 PM on Saturday 9/17, at 3806 Williams Rd. Please RSVP to David Friedman (ddfr[at]daviddfriedman[dot]com) so he knows how many people are coming. I will be at the Berkeley one. Feel free to come even if you’ve never been to a meetup before, even if you only recently started reading the blog, even if you’re not “the typical ACX reader”, even if you hate us and everything we stand for, etc. There are usually 50-100 people at these so you should be able to lose yourself in the crowd. Shouldn’t we have planned meetups further apart for people who wanted to go to multiple of them? Yes, and this is directly my fault, up to and including rescheduling to avoid the San Jose one . . . right on to the same day as the San Francisco one. Sorry, I’ll try to do better next time. Also coming up this weekend are meetups in Washington DC, Atlanta, Columbus, Providence, Cape Town, Cambridge (UK), Kuala Lumpur, Chicago, Houston, Toronto, New Haven, Bangalore, and many more. See the list for more details.

https://astralcodexten.substack.com/p/unpredictable-reward-predictable [Epistemic status: very conjectural. I am not a neuroscientist and they should feel free to tell me if any of this is totally wrong.] I. Seen on the subreddit: You Seek Serotonin, But Dopamine Can’t Deliver. Commenters correctly ripped apart its neuroscience; for one thing, there’s no evidence people actually “seek serotonin”, or that serotonin is involved in good mood at all. Sure, it seems to have some antidepressant effects, but these are weak and probably far downstream; even though SSRIs increase serotonin within hours, they take weeks to improve mood. Maxing out serotonin levels mostly seems to cause a blunted state where patients can’t feel anything at all. In contrast, the popular conception of dopamine isn’t that far off. It does seem to play some kind of role in drive/reinforcement/craving, although it also does many, many other things. And something like the article’s point - going after dopamine is easy but ultimately unsatisfying - is something I’ve been thinking about a lot.

https://astralcodexten.substack.com/p/i-won-my-three-year-ai-progress-bet I. DALL-E2 is bad at “compositionality”, ie combining different pieces accurately. For example, here’s its response to “a red sphere on a blue cube, with a yellow pyramid on the right, all on top of a green table”. Most of the elements - cubes, spheres, redness, yellowness, etc - are there. It even does better than chance at getting the sphere on top of the cube. But it’s not able to track how all of the words relate to each other and where everything should be. I ran into this problem in my stained glass window post. When I asked it for a stained glass window of a woman in a library with a raven on her shoulder with a key in its mouth, it gave me everything from “a library with a stained glass window in it” to “a half-human, half-raven abomination”.

https://astralcodexten.substack.com/p/links-for-september-2022 [Remember, I haven’t independently verified each link. On average, commenters will end up spotting evidence that around two or three of the links in each links post are wrong or misleading. I correct these as I see them, and will highlight important corrections later, but I can’t guarantee I will have caught them all by the time you read this.] 1: Fiber Arts, Mysterious Dodecahedrons, and Waiting On Eureka. Why did it take so long to invent knitting? (cf. also Why Did Everything Take So Long?) And why did the Romans leave behind so many mysterious metal dodecahedra? 2: Alex Wellerstein (of NUKEMAP) on the Nagasaki bombing. “Archival evidence points to Truman not knowing it was going to happen.” 3: @itsahousingtrap on Twitter on “how weird the [building] planning process really is” 4: Nostalgebraist talks about his experience home-brewing an image generation AI that can handle text in images; he’s a very good explainer and I learned more about image models from his post than from other much more official sources. And here’s what happens when his AI is asked to “make a list of all 50 states”:

https://astralcodexten.substack.com/p/book-review-contest-2022-winners       Thanks to everyone who entered or voted in the book review contest. The winners are: 1st: The Dawn Of Everything, reviewed by Erik Hoel. Erik is a neuroscientist and author of the recent novel The Revelations. He writes at his Substack The Intrinsic Perspective. 2nd: 1587, A Year Of No Significance, reviewed by occasional ACX commenter McClain. =3rd: The Castrato, reviewed by Roger’s Bacon. RB is a teacher based in NYC. He writes at Secretorum and serves as head editor at Seeds of Science (ACX grant winner), a journal publishing speculative and non-traditional scientific articles. =3rd: The Future Of Fusion Energy, reviewed by TheChaostician. =3rd: The Internationalists, reviewed by Belos. Belos is working on a new blook titled best of a great lot about system design for effective governance.       

https://astralcodexten.substack.com/p/the-prophet-and-caesars-wife I. The Prophet in his wanderings came to Cragmacnois, and found the Bishop living in a golden palace and drinking fine wines, when all around him was bitter poverty. The Bishop spent so long feasting each day that he had grown almost too fat for his fine silk robes. “Woe unto you!” said the Prophet, “The people of Cragmacnois are poor and hard-working, and they loathe the rich and the corrupt. Rightly do they hate you for spending the Church’s money on your own lavish lifestyle.” “Actually,” said the Bishop, “my brother the Prince lets me use this spare palace of his and its well-stocked wine cellar. If I refused, he would just give it to someone else, or leave it empty. I’m not stealing church resources, and there’s no way to divert the resources to help the poor. And I am secure in my faith, and won’t be turned to hedonism by a glass of wine here and there. So what’s wrong with me enjoying myself a little?” “It is said,” said the Prophet, “that Caesar’s wife must be not only pure, but above suspicion of impurity. A good reputation is worth more than any treasure. Fat as you are, nobody will believe you are untainted by the temptations of wealth. Give the golden palace back to your brother, and live in a hovel in the woods. Only then will you earn the people’s trust.” II. The Prophet in his wanderings came to Belazzia, and found the Bishop living in a hovel and wearing a hair shirt. He spent so long in prayer each day that he barely ate, and seemed so dangerously thin that he might fall over at any moment. “Woe unto you!” said the Prophet. “For the people of Belazzia are rich and sophisticated, and they mock you for your poverty and uncleanliness. Does the Church not give you enough funds to build a golden palace and wear silk robes? If you were the most resplendent citizen of this nation of splendor, would they not take you more seriously?”

https://astralcodexten.substack.com/p/billionaires-surplus-and-replaceability The typical neoliberal defense of self-made billionaires goes: entrepreneurs and other businesspeople create a lot of value. EG an entrepreneur who invents/produces/markets a better car has helped people get where they’re going faster, more safely, with less pollution, etc. People value that some amount, represented by them being willing to spend money on the car. The entrepreneur should get to keep some of that value, both because it’s only fair, and because it incentivizes people to keep creating value in the future. How much should they keep? The usual answer is that the surplus gets distributed between the company and the customers. So suppose that this new type of car makes the world $200 billion better off. We could have the company charge exactly the same price as the old car, in which case customers get a better car for free. We could have the company charge enough extra to make a $200 billion profit, in which case customers are no better off than before (they have a bit less money, and a bit better car). Or they could split it down the middle, and customers would end up better off than before and the company would make some money. Which of these distributions happens depends on competition; if there’s no competition, the company will be able to take the whole surplus; if there’s a lot of competition, all the companies will compete to lower prices until they’ve handed most of the surplus to the customers. Then once the company has some portion of the surplus, it divides it among capital and labor in an abstractly similar way, although with lots of extra complications based on whether the labor is unionized, etc.

Finalist #16 in the Book Review Contest https://astralcodexten.substack.com/p/your-book-review-kora-in-hell [This is one of the finalists in the 2022 book review contest. It’s not by me - it’s by an ACX reader who will remain anonymous until after voting is done, to prevent their identity from influencing your decisions. I’ll be posting about one of these a week for several months. When you’ve read them all, I’ll ask you to vote for a favorite, so remember which ones you liked.]   The sense that everything is poetical is a thing solid and absolute; it is not a mere matter of phraseology or persuasion. — G.K. Chesterton I. William Carlos Williams attributes the title to his friend/rival Ezra Pound, mythological references’ number one fanboy. Kora is a parallel figure to Persephone or Proserpina, the Spring captured and taken to Hades by Hades himself. Persephone as a plant goddess and her mother Demeter were the central figures of the Eleusinian Mysteries, which promised the initiated a groovy afterlife glimpsed at by psychedelic shrooms. And Kora means maiden. Ancient Greeks called her that either because she was like Voldemort, and you were apotropaically not supposed to say her true name because this is a Mystery Cult, damn it. Keeps some of the mystery. Or because she in a way represents all of the maidens, everywhere. So, in that sense, Kora in Hell alludes to the multitude of suffering young women Williams met while working as a doctor, assisting in 1917 style home labors, and, because WWI was going on at the time and doctors were extremely scarce, as a local police surgeon. Conditions were dire:

https://astralcodexten.substack.com/p/meetups-everywhere-2022-times-and Thanks to everyone who responded to my request for ACX meetup organizers. Volunteers have arranged meetups in 205 cities around the world, including Penryn, Cornwall and Baghdad, Iraq. You can find the list below, in the following order: Africa & Middle East Asia-Pacific (including Australia) Canada Europe (including UK) Latin America United States You can see a map of all the events on the LessWrong community page. Within each section, it’s alphabetized first by country/state, then by city - so the first entry in Europe is Vienna, Austria. Sorry if this is confusing. I will provisionally be attending the meetups in Berkeley, Los Angeles, and San Diego. ACX meetups coordinator Mingyuan will provisionally be attending Paris and London. I’ll be announcing some of the biggest ones on the blog, regardless of whether or not I attend. Extra Info For Potential Attendees 1. If you’re reading this, you’re invited. Please don’t feel like you “won’t be welcome” just because you’re new to the blog, demographically different from the average reader, or hate ACX and everything it stands for. You’ll be fine! 2. You don’t have to RSVP or contact the organizer to be able to attend (unless the event description says otherwise); RSVPs are mostly to give organizers a better sense of how many people might show up, and let them tell you if there are last-second changes. I’ve also given email addresses for all organizers in case you have a question