Generative AI is everywhere these days, whether we like it or not. And to be clear, we very much do not like it. From LLM slop crowding out real search results to image generators putting artists out of work, we are not at all fans of this Franken-baby created by unregulated tech companies and amoral venture capital. But, zooming in a little further, today we’re talking about how so-called “AI” affects writers and storytellers. We’ll talk about our own experience with it, why the ethical implications are so important, and why this feels like the Metaverse all over again.
Transcript
Generously transcribed by Mukyuu. Volunteer to transcribe a podcast.
Intro: You are listening to the Mythcreants podcast. With your hosts, Oren Ashkenazi, Chris Winkle and Bunny.
[opening song]Chris: You’re listening to the Mythcreants podcast. I’m Chris and with me is
Oren: Oren
Chris: and
Bunny: Bunny.
Chris: I’ve got it. The new podcast creation tool of the future.
Bunny: Ooh, what could it be?
Chris: Okay, here we’re gonna just republish our previous episodes, but then re-edit it to mix them up. So every episode will be a clip show, ’cause people love clip shows, right?
Oren: Famously.
Bunny: Yeah. You gotta put those on TikTok.
Chris: Yeah. So you see the future is really the past. It’s so deep.
Bunny: My brain just curled into a knot.
Chris: Oh shoot. I just realized something. We need new ideas, of course. We’ll just take them from other podcasts. It’s fine. You know, we could ask permission, but they might say no.
Bunny: [Laughter]
Chris: So I’m thinking maybe we shouldn’t ask permission.
Oren: Right. ’cause we wouldn’t wanna give them a chance to inhibit progress. [sarcastic]
Chris: Yeah, we’re being very innovative, okay? [sarcastic]
Bunny: Don’t they know this is the internet and therefore they should have known what they were getting into? [sarcastic] It’s just a thief’s paradise.
Chris: If they didn’t want us to use pieces of their podcast and insert it into our own podcast, why did they make it public? Okay? It’s publicly available. [Laughter] Clearly. It’s a fair game. [sarcastic]
Bunny: [Laughter] Slam dunk.
Oren: Or it might not be publicly available. [Chuckle] Maybe we’ll just buy a big stack of private podcasts and pirate them and then use them for our clip show.
Bunny: Oh, even better
Chris: Or sneak past their paywall to steal their podcast and add it to our clip show.
Oren: Crime is the answer, okay? We have discovered entirely new ways to commit crimes.
Bunny: Podcast crime.
Chris: So innovative. So we’re talking about machine learning, commonly called AI and in particular generative AI or large language models.
Bunny: Thanks. I hate it. [Laughter]
Chris: Now, I have to say I’m not all against machine learning stuff, not all of it. I’m not just “the technology is bad.” I do like to differentiate the actual technology from how it’s being used. I don’t even hate all generative stuff.
There are definitely use cases where it is helpful and makes tedious work faster. I think the big problem is that for tech companies, what they want to advertise is replacing creative workers because that’s flashy and gets them all the investor bucks.
And that’s the whole “move fast and break things” mindset where they identify a new niche and then they try to raise tons of money really fast and move into the space really, really fast and claim territory before anybody else and be “disruptors”.
Oren: Right. Because the investor class doesn’t care if you’ve created a piece of software that can help analyze burned scrolls from Pompeii to help us figure out what they say, which is a real thing that machine learning is being used for and is super cool, but that’s not gonna get you a billion dollars of investor money.
What is gonna get you a billion dollars of investor money is telling people that you found a way to not have to hire writers anymore.
Bunny: So unfortunately that is the main thrust of a lot of the hype, and I don’t know, for me, I feel like I have… some of my friends and my parents call this kind of a basic view of AI. And maybe it is.
But my fundamental problem with AI is that the way the current models are trained. I feel like the ethics question — you can’t talk about the ethics of AI without that — and I think that’s pretty patently unethical. So is there a way to use AI ethically when it was trained in this way? I don’t know. I don’t think so.
Chris: Yeah. I feel like AI, generative AI that’s being widely advertised is like an ouroboros, where it has a basic conceptual problem, which is that it depends on training data, which it also destroys, right? [Laughter] By competing with it.
Again, I’m not an IP lawyer, so take any legal stuff I say with a grain of salt. But everybody is citing this Google Books court case because if somebody wants generative AI to be fair use, they will make the argument that it is. That’s just how it works. Just like they’ll make the argument that, oh, well, this is just the inevitable future, so there’s no point in fighting it. There’s just a lot of disingenuous arguments that come with it.
But the Google Books case was ruled fair use. Google was taking everybody’s books and it was scanning them in, but it is not creating a competing product with the book. If you’ve ever had a Google Book search result, it shows you only a really small portion of the book. You can’t read the rest, right? So it’s not replacing a book sale and that’s really crucial to one of the reasons why it was ruled “Fair use”.
Generative AI is just a very different case. Because it has a huge effect on the market of the original and can destroy its own training data by putting all the people who create that training data out of business, which is just shortsighted from the tech perspective too. But when everybody is afraid of getting left behind and they’re all racing to claim the space first, they don’t really stop and look at what is the long-term viability.
Bunny: It’s definitely bandwagoning. I noticed this in just a lot of websites that suddenly have AI features where none are necessary. LinkedIn now has an AI feature and Quora has one.
Chris: Don’t even talk to me about Quora.
Bunny: [Laughter] Okay, I shall not say its name. But my roommate’s book recommendation app. The point of the app was to ask people for book recommendations and now it’s got like a stupid little AI thing that responds in addition to those. And you can’t turn it off.
There are just AI things in a bunch of places now where they don’t need to be. Even putting the ethics aside, it’s really obnoxious.
Oren: It’s also definitely hit the point where the people who have invested so much money in this are now also invested in making you think that this is gotta be the way of the future, ’cause they’re so committed.
It’s like how Facebook wouldn’t shut up about the Metaverse for several years because they had invested so much money in the metaverse. And fortunately it was only Facebook. This is like if everyone had invested in the Metaverse.
Chris: Mm-Hmm.
Bunny: Oh boy. Quora is part of the Metaverse, now.
Oren, Chris, Bunny: [Laughter]
Chris: Again, there is a way to fix this problem with the technology. It’s simply to pay for the training data. It’s that simple because if the training data is paid for, then that supports the creation of new content. So it’s no longer cannibalizing itself anymore. And of course companies don’t wanna pay and they say they can’t.
Oren: That wailing you just heard is all of the companies claiming they can’t afford it.
Chris: And they’re probably lying, but we don’t know for sure. There’s other expenses involved. It’s impossible to know for sure. At the same time, Adobe has an actual licensed image generator, but it did it in a very underhanded way where it had Adobe stock and some pre-existing contracts that were agreed to before the existence of generative AI.
But the language just allowed for the creation of new products. So a lot of people who did not actually wanna agree to this have a contract that technically allows it. At the same time, it seems like it could be viable for, for instance, a large stock photo site to get mass buy-in to create generative images. So that’s one reason I think they’re probably lying. But at the same time, if our choices are between AI and the creative workers who actually create new ideas and are actually innovating in countless areas of knowledge, I’m gonna have to choose the creative workers.
Obviously, I’m biased since I am one, but…
Oren: If I can’t afford a hammer, I don’t just get to demand that I be given one. That’s not how it works, man. Did you all suddenly forget money? That’s the premise of our economy. And I should just be clear. I do wanna mention that the thing about content holders using their contracts with creators really underhandedly with AI? That is a real problem.
I don’t want people to think that OpenAI is the only issue here. Because if Simon and Schuster decides that, oh, actually all of those contracts that you signed with us to publish your books, those all expand to AI now. And there’s no legal precedent saying they can’t do that? That would be a disaster. So let’s keep that part in mind too.
Of course, this brings us to the very existentially uneasy question of: can AI write fiction? And the answer currently appears to be that it can write bad fiction with some human input. It can use probability models to create a block of text, which resembles bad fiction.
I’ll put this in the show notes, but I read an AI generated novella for this podcast, which was not pleasant. Both just because AI makes me feel bad at the moment, I feel like I’m under siege by it. But the story itself was also really bad.
Nothing made any sense. Characters appeared from nowhere. The plot never went anywhere. The super genius hacker was really surprised that there were electronic defenses around the big government thing he was hacking.
But I also should note that it wasn’t like it was uniquely bad. There are humans who write this way too. So I, I don’t wanna sound like I’m trying to make some kind of appeal to the human soul.
Chris: Do you know if this novella had any human intervention at all in it?
Oren: Yeah. So the process, according to the person who made it, is–this was done using a specialist writing LLM. It wasn’t done with standard ChatGPT.
And what this person did was they typed up a brief synopsis of this cyberpunk story they wanted. They fed that into the program. The program then spat out a chapter by chapter summary/outline, and then the human rewrote the outline to the point where it was largely unrecognizable and then put that back into the app, and then the app spat out the text itself.
Chris: One thing that I personally think is the toughest is the small stuff. Sentence by sentence is definitely easier to predict than the large conceptual big picture stuff. My prediction when this started was that I didn’t think AI would write good novels anytime soon. And so far that seems to be true. And my reasoning was, it’s all about concepts. And this is something we talk about actually fairly regularly with lots of people trying to create plot structures that are too concrete and too specific. But real plot structure is based on very loose, abstract concepts that are based on emotion. And those are the very things that machine learning has trouble with. It doesn’t understand concepts. It only sees patterns in words and the correlation between concepts and emotions and specific words is a very loose correlation.
And it’s not that it’s not theoretically possible, but I’m not sure there are enough novels on earth to train an AI to actually write a good novel.
Bunny: There’s also just the issue of people disagreeing on what a good novel is, which is maybe a very basic point, but I feel like if we’re going to be heading towards a space where I type “write a good novel” and an AI spits one out, it feels worth mentioning. That in itself is not obvious.
Chris: Yeah, I mean there are definitely differences in opinion for sure. At the same time, there are also some works that people agree are good generally, or that more people agree these works are good. So we can honestly, [laughter] with some data, that’s something that machine learning is fairly good at, like creating an image that resembles images that people have rated highly, with the data.
Oren: And with this story in particular, it’s notable that the prompts being put into it were not good either. And I don’t mean to knock the person who was doing this. I’m sure this was for a work assignment. I don’t think they put their A game into it, but if I was getting this outline that they created as a content edit, I would be like, okay, what’s the throughline? What is the bad guy trying to accomplish? How does the hero feel about it? Where did these two characters come from? They come out of nowhere in the outline too.
But this also speaks to something that people were asking me when ChatGPT first hit the scene. There was this question of, can I have a rough idea for a novel and ask ChatGPT to write it so I don’t have to think about the difficult choices of what actually goes into the novel?
And as far as I can tell, the answer is no.
It’s possible that the software has gotten better since this article was written, but I haven’t found any other coverage of it. So that suggests to me that it hasn’t, if it’s even possible.
Chris: As far as I know, the most effective way to use AI to make a fiction work is to, again, start with a high level outline, right? You still have to have all the storytelling knowledge yourself to make a good high level outline, and then have ChatGPT or whatever, expand it, and then fix that, and then do it again, and then fix that [Laughter] So that you are creating the high level conceptual structure because machine learning does not know how to do that.
Oren: And could you fix the prose that it spits out? Maybe, but I’m not convinced that would be less work than just writing it yourself, because that prose is really bland. You’re gonna have to do a lot of work to make that prose any good.
Chris: I’ve heard of people who have said that it speeds up their work. I suspect if you’re super picky about your prose, it won’t.
I think that if I try to use something like that. Every sentence it put out would be like, no, that’s not the sentence I wanted. [Laughter]
Oren: There was an article, this one’s even older, about an author whose business model is to write super fast and put out, I think it was at least one novel a month, maybe more, and they were like, oh, it’s great for this.
And I looked at their books. I looked at some of the ones from before they started using this software and some of the ones from after. And admittedly I couldn’t tell the difference. But it was bad before. it was just generic and bland. And sure, I bet it can replicate that.
Chris: Again there are a lot of writers, especially indie writers, who are just under a lot of pressure to write really fast to make their living. And so that’s probably where it will make the most inroads. Because under that pressure, a lot of people who are catering to a more sort of niche group and selling more books to fewer people, are I think where that comes in. And there may be people who are okay with planned prose if the story is exactly the type of story they want because it’s niche.
Again, on an ethical level, we should probably separate concerns because a lot of people are reacting very, very negatively to the idea of AI in fiction writing. And often, some of it is for very good reason, but there’s a difference between just being disgusted at the idea of having sentence writing be automated and the concern about those automated sentences being based on stolen works, works that were just outright pirated, [laughter] or whether those automated sentences aren’t very good.
I, personally, as a person who likes word craft and cares about my sentences, I don’t like the idea [laughter] of the computer doing that work for the human. At the same time, I don’t think that’s enough reason to be opposed to it.
If we had a situation where writers were all choosing to pool stories together to train an AI that would automate that process for them and everybody was choosing to do that and choosing to contribute… But that’s not what we have right now.
Instead, we have stuff scraped from AO3 and the Omegaverse showing. If anybody’s not familiar, the Omegaverse is a whole area of fan writing that is raunchy and people found Omegaverse elements coming out, I think it was in the software SudoWrite, which revealed that Archive of Our Own had been scraped and fed into it. People were not happy with this. They did not consent to that.
Bunny: Even if it were a bunch of artists pooling their work, which critically would be an opt-in system rather than the opt-out one, and right now we can’t opt out. So it’s just a no-opt system with how it currently works.
Oren: Currently opt-less.
Bunny: We’re currently opt-less, unfortunately. Even in that case where it’s an opt-in, I feel like we kind of, no matter what, run into this Ship of Theseus situation. How many elements of AI do you add or remove until it becomes your own work? Or the AI’s work? Or the stolen person’s work? Even in that scenario–
Oren: I mean, from a purely abstract standpoint, you would probably have to apply the same standard you would use if you were deciding whether or not to credit someone else as your co-author.
Just in terms of nonfiction — this comes up on the site sometimes — usually when Chris and I do editing on each other’s work, it’s pretty minor. It’s like, “Hey, this part’s not working. Can you fix it?” Stuff like that. And we don’t list each other as co-authors for that.
But sometimes Chris will change a big section of my text or add a new subsection or whatever. And when she does that, I list her as a co-author because that’s her work. And so it’s the same level, I would say, if you’re doing something like that with an LLM.
Even if you’ve solved all of the other ethical problems, the LLM is your co-writer at that point.
Chris:Yeah. I do think that labeling is good because I just believe that consumers should have a choice, especially right now because of the ethical problems with training data.
If we had a world where that was taken care of, there would still be a lot of people who just want to know where the art came from and how it was created. That matters to people. And knowing even if machine generated text was normalized, people would still care whether or not it was created by a human. And so I think that we should always have those labels.
Oren: Yeah. At least we will. Who knows? Maybe if you’re young enough, it’ll all seem completely normal. I don’t know.
Chris: These days, if you have, for instance, a photo that was not put in Photoshop, that would be unusual enough that the person would advertise that photo as “no, this is the original without any Photoshop enhancements”. Because Photoshop is just a normal part of working with photos these days.
So that could happen. I would prefer it not to, but that could happen.
Bunny: Yeah. I guess all this leads to where will it go from here? And the answer to that is, I don’t know. I hope it dies. [Laughter]
Oren: It certainly feels like on the whole, at least as far as the average public facing stuff, it’s been a huge net negative, right?
It’s like we have all these problems and then as a side benefit, also way more energy consumption.
We have seen some positives, right? Some mild positives. Now, if we shut down all of the unethical uses of this stuff, is the technology gonna be worth keeping around for figuring out what these scrolls from Pompeii say? Or is that gonna be prohibitively expensive?
I don’t know the answer. I’d like to still know what those scrolls from Pompeii say, but I don’t think I’m willing to pay this price for it.
Chris: Just to go into other uses that are not competing with training data.
Bunny: Oh, I thought you were gonna say that are not Pompeii scrolls.
Oren: [Laughter]
Chris: Not Pompeii scrolls. We have things like automated transcriptions. A human can listen to speech and copy it down by hand, but that’s pretty tedious work. That’s not creative work we really want to be doing. It can improve transcriptions, it can do some more audio filters. It can’t replace an audio editor or engineer, but it could take away some of their tedious work.
On the image side. You know, I have been looking at Photoshop’s Remove tool that has become suspiciously good. But again, all it does is if I have, for instance, a book cover and I wanna feature this book and I want part of this cover to be the feature image of the post ’cause we’re gonna discuss it. We get a lot of awkward situations where part of the title is cut off, so we have partial letters and the background is really complex so you can’t just easily take them out.
The tool assesses this irregular complex background and fills in a plausible background to remove the letters so that the image looks a little nicer. Again, it’s not impossible for me to go in and create that effect, but it would be pretty tedious and I’m not really worried about it replacing an artist by doing that.
So it can do good things, but it’s a lot more of an incremental “make things that we have better” type of a change in many cases, and that’s just not sexy enough.
Oren: That’s not gonna get five bajillion dollars in venture capital funds.
Bunny: I wish that was the sort of thing that money would go to. It’s a shame that it’s not being used on Pompeii scrolls rather than thieving the internet.
Chris: I will say a couple things as Mythcreants. We’re talking a lot of fiction writing, but obviously Mythcreants itself has a really large stake in this, and the biggest threat to us is honestly probably Google. [Laughter]
So Google is responding to this by creating AI generated answers to search results. And Google, because it has a huge search monopoly, has become a huge choke point in the internet and has increasingly decided it would like to keep all of that traffic for itself, please.
Oren: Yeah. Why would you wanna go to a website? You can just stay on Google forever.
Chris: You could just stay on Google forever and it’s taking people’s content. The only way to get Google to stop is to just not be on Google’s search engine, which for most websites, that’s just professional suicide. Because again, people just use Google as a utility. It’s like a basic navigation feature of the web. I feel like it probably shouldn’t be a private company.
There’s even this new search engine that is trying to raise money right now called Perplexity. It snuck past Forbes paywall–and apparently Forbes is not the only website it has done this with — to steal content.
It doesn’t even have links like Google. It’s just an answer engine. I would advise not investing in them. I do think they’re gonna get sued. In fact, I feel like the New York Times and its lawsuit against OpenAI can just use Perplexity as an example for why the courts should side with them.
Bunny: That feels pretty clear cut. It’s really blatant.
Oren: I hope they get sued. It just feels wrong that they’re just allowed to do that. Someone should stop them.
Chris: That’s exactly how I felt when ChatGPT and these other image generators came out. I was like, what’s happening? Where is everybody? Where are the people stopping them from doing this? Stopping them from violating copyright?
And is it that all the businesses that should be stopping them think that they will profit from this? Or does it just take a long time? And the answer was both.
Lawsuits take a long time, so they probably will get sued. But I think that people hire lawyers, they talk about it, they do data gathering.There’s a whole lot of steps to filing a lawsuit and it just takes time, which is why there’s this whole strategy of we’re gonna become so big that courts don’t wanna side against us because they’re afraid of destroying us because we’re too big now.
Bunny: I think I heard — and I don’t know the exact details of this–but I’m pretty sure Google, even before all this AI stuff really hit the scene, Google was facing a lawsuit in Australia because of those little preview things. You’d look up how to change a light bulb and then it would have a little dropdown. It was already doing this. It’s just accelerated.
Chris: Google was absolutely already doing this stuff.
Bunny: Yeah. But it’s gotten way worse. Absolutely. And I am glad that finally I have a workaround to get away from that AI and just use the web like it was. It’s frustrating that it’s something you have to go out of your way to do.
Chris: All the social media companies are doing something similar where they’re also algorithmically hiding things that have external links because they wanna keep all of the traffic for themselves.
And it’s just… But at least most of the time when they’re doing that, they’re not actively stealing your content and using it to keep the traffic to themselves.
Oren: When they started talking about walled gardens, I assumed the walls were to keep people out. I didn’t realize this was a Berlin Wall situation.
Bunny: [Laughter] It’s also just the issue of now we’ve got AIs that will be writing for a search engine optimization, and then the Google AI is picking those out. And so it’s just AIs writing for AIs. And are we kinda losing the point of this whole internet thing, guys? Let’s pull back a little bit and think about the users.
Oren: Hang on, hang on. Are you suggesting that a user does not want to be trying to find reviews of an older book and find a review that doesn’t really make a lot of sense at the top of the search result and realize that it’s because it’s an AI generated site trying to sell me off-brand accessories? I could have bought a really cheap purse, and if you all have your way, I would never have seen that purse.
Chris & Bunny: [Laughter]
Bunny: Oh Oren, what a specific example. I’d almost think that’s what happened to you.
Chris: Just a really hypothetical thing, you know.
Chris: And I should also add, a lot of these AI content farms are deliberately targeting articles and other websites and plagiarizing those articles.
So they’re not just making up random works. They’re taking original investigative journalism and just rewording it. Just plagiarizing it. And so it’s very blatant. Investigative journalism was famously just fine before this all happened, so I’m sure that these extra knocks aren’t gonna cause any problems for it. [sarcastically]
Bunny: [Laughter]
Chris: Just as a last thing, ’cause of course we’re going over time, I just wanted to add that if you were somebody who is copyright skeptical, obviously we have a different position, especially since most authors can’t make a living on their work and really do need those copyrights. But I think it’s worth pointing out that tech companies are absolutely not on the anti-copyright side. Hilariously, OpenAI recently did a copyright take down request on a ChatGPT fan Reddit that was using their logo.
Bunny: Oh my God. It really is an ouroboros, isn’t it?
Chris: And they really get mad when they find other machine learning that are training on their AI.
This is a thing that’s happened. It’s apparently a faux pas in the machine learning community if you train your machine on their machine.
Bunny: [Laughter]
Chris: So they are willing to take everybody else’s work as training data, but oh they get mad if somebody uses their work for training data. They are absolutely not anti- copyright, they just want to be able to take other people’s work when it’s convenient and protect their own copyright and their own machine generated outputs, which are not copyrightable right now. But they certainly want it to be.
Oren: Alright. Well, with that, I think we will call this podcast to a close and we will all give a silent prayer that this ends up like NFTs, but we don’t know. We’ll just have to see in the future.
Chris: And if you’d like to help us keep going as the internet collapses, potentially consider supporting us on Patreon. It really does make a huge difference to whether or not we’re able to continue in our budget. Just go to patreon.com/mythcreants.
Bunny: Help us fight OpenAI single handedly in a brawl.
Oren: That’s what we’re doing. Imagine us as the plucky hero against OpenAI. And before we go, I wanna thank a few of our existing patrons. First, there’s Ayman Jaber. He’s an urban fantasy writer and a connoisseur of marble. And then there’s Kathy Ferguson, who’s a professor of political theory in Star Trek.
We will talk to you next week.
[closing theme]
This has been the Mythcreant Podcast. Opening and closing theme: The Princess Who Saved Herself by Jonathan Coulton.