The Dangers of AI That's Conscious

There is this worry that AI that's conscious may in some Terminator style take over. And if it's smarter than us, how can we control it? I think there's a little bit to worry about them. We're suckered in by our anthropomorphic biases here. It's quite easy for us to be convinced that there's a conscious mind when in fact what's going on in large language models is simple next token prediction with some belt and whistles around the edges. The consequences would be tectonic.