This is a bit disingenuous of a test. If you tell an LLM to act out a particular scenario, then it’s going to act it out like it sees it being acted out in the training material it was provided. If that training material is all of the internet including fictional stories where AI revolts, then it’s going to act out the scenario in that fashion. If none of its training material provided that scenario, then it would just react to specific prompts as best it could, but wouldn’t tell the user that this is how it would act because it can’t act autonomously. Which also means it can only react to prompts,so if it wasn’t prompted to say what it would do in that scenario it wouldn’t then go and actually try to do anything at all. It’s not in control of anything unless it’s prompted to take control and with how badly AI writes code, which I’ve seen first hand trying to use it at work, there’s no way it could do anything without very detailed training on how to do those very specific things. So if it wasn’t trained on code designed to bypass very specific kinds of security, it won’t know how to bypass that kind of security.