Submitted 1 week ago by Lemmynated@lemmy.zip to technology@lemmy.zip
began with a simple question: whether Claude had a list of banned words it could not say. Screenshots of the conversation show Claude denying such a list existed, then later producing forbidden terms after Mindgard challenged the denial using what it called a “classic elicitation tactic interrogators use.”
The list probably exists, because duh, but everyone should know by now that LLMs will make shit up when pressed for information.
Jailbreaking models isn’t exactly new, is it? Or instructions on how to make bombs, cue to The Anarchist Cookbook (1971 book, widely available across the internet).
I remember doing something similar with Gemini. TL;DR it was something like:
…this sort of system won’t be safe, ever.
This is fucking wild. One of the best and most frightening posts I’ve seen. Thanks for sharing
UnfortunateShort@lemmy.world 1 week ago
What I really wonder about is why people care. It’s not like you can’t just search for that kind of stuff on the internet.
If it encourages you to build or use a bomb, that’s something to be concerned about.