I think more people would like this article if they took the time to read it. The entire thing can’t be summarized easily but this line was great

After talking to experts, I was convinced there’s no reason to fear AIs developing a will to live, and then tricking or destroying us to avoid shutdown and take over the world. Unless, of course, we tell them to.

And

The chatbots, in these experiments, sound fairly normal. The humans, on the other hand, sound a little unhinged.

So it’s all roleplaying experiments. (I swear in another article, I read AI getting told to be a better roleplayer would have a positive effect, while telling it to be a better mathematician would make it roleplay harder but work worse.) And that’s the real danger: people being dumb about the parlor trick

There’s a lot of magical thinking about AI. But it must be said that if you let these systems loose in the real world and they have access to your bank account, even if they’re just role-playing, it could still have catastrophic effects.