Brief Interactions With LLMs Are Often Misleading

When GPD4 came out, and I guess also when ChatchPTT first came out, there was very predictably this wave of people on Twitter saying, hey, this system is sentient. There's a great analogy that came up in a recent article by Helen Toner saying they're basically improv players. They're, um, they're going to say yes. They're going to go along with what you're doing. If you make it clear what you expect, like, what kind of narrative you're putting them in,  they'll go along with that. And so chat, GPT and the other LLMs are there to say yes, yes, and, and that