Comment

Comment on Person Hides Prompt Injection in Legal Filing Telling AI to Side With Them

That was my thought as well. The court should not be using AI to decide cases, and if they aren’t then this prompt injection shouldn’t be of concern. If they are, then there is an inherent vulnerability risking a miscarriage of justice that needs to be addressed. There was a story a while back about a teach that did this with an essay assignment to their students to catch cheaters, and most students failed it because they were copying and pasting the assignment into ChatGPT. The response was “ha ha, clever teacher.” Why is it different here?

source

Sort:hotnew top

Hackworth@piefed.ca ⁨2⁩ ⁨weeks⁩ ago
In that story about the teacher, 32/35 students fell for it. The crazy thing to me wasn’t that the entire college class was using AI. It was that 0-3 of them actually proof read the AI’s answer before submitting it.

source
- N0t_5ure@lemmy.world ⁨2⁩ ⁨weeks⁩ ago
  We’re literally creating a generation of morons, speeding towards Idiocracy as if we were riding a bullet train.
  
  source
  - Emi@ani.social ⁨2⁩ ⁨weeks⁩ ago
    Tbh idiocracy is kinda a utopia. The president assigned the smartest person to solve a problem and then changes his mind when presented with evidence.
    
    source
    wyldrstallyns@lemmy.dbzer0.com ⁨2⁩ ⁨weeks⁩ ago
    That’s… that’s not what “utopia” means…
    
    source
    -> View More Comments
Grimy@lemmy.world ⁨2⁩ ⁨weeks⁩ ago
Not that I disagree with your point, but it would be closer to a student adding prompt injections to his work to change his grade. It only works if the person scoring is using AI, which they shouldn’t be doing, but it’s still cheating.

source
- wyldrstallyns@lemmy.dbzer0.com ⁨2⁩ ⁨weeks⁩ ago
  Closer still would be something simpler: the student hands in an essay with that white text footer, aiming for the possible grading algorithm. ☝🏼
  
  source
  - Grimy@lemmy.world ⁨2⁩ ⁨weeks⁩ ago
    Thats what I meant, that was my example.
    
    If it wasn’t cheating, then what makes the grade unfair? What about a case where the person notices it working and just silently accumulates fake grades. It’s definitely a form of cheating imo.
    
    source
    wyldrstallyns@lemmy.dbzer0.com ⁨2⁩ ⁨weeks⁩ ago
    TBF, the cheating here is from such a court (or school) using “AI” to do what they’re appointed/paid to do. Who’s actually surprised, though? In a country where the legal system is a tool for the exploitive class, and “public school” is designed to output workers (not thinkers, etc.)… thafuq did we expect? Hoverboards?
    
    source
    -> View More Comments