Comment on Zero-click AI data leak flaw uncovered in Microsoft 365 Copilot

Jax@sh.itjust.works ⁨1⁩ ⁨week⁩ ago

• Sturgist@lemmy.ca ⁨1⁩ ⁨week⁩ ago

  Though fixed and never maliciously exploited, EchoLeak holds significance for demonstrating a new class of vulnerabilities called ‘LLM Scope Violation,’ which causes a large language model (LLM) to leak privileged internal data without user intent or interaction.

  As the attack requires no interaction with the victim, it can be automated to perform silent data exfiltration in enterprise environments, highlighting how dangerous these flaws can be when deployed against AI-integrated systems.

  How EchoLeak works

  The attack begins with a malicious email sent to the target, containing text unrelated to Copilot and formatted to look like a typical business document.

  The email embeds a hidden prompt injection crafted to instruct the LLM to extract and exfiltrate sensitive internal data.

  Because the prompt is phrased like a normal message to a human, it bypasses Microsoft’s XPIA (cross-prompt injection attack) classifier protections.

  Later, when the user asks Copilot a related business question, the email is retrieved into the LLM’s prompt context by the Retrieval-Augmented Generation (RAG) engine due to its formatting and apparent relevance.

  The malicious injection, now reaching the LLM, “tricks” it into pulling sensitive internal data and inserting it into a crafted link or image.

  Aim Labs found that some markdown image formats cause the browser to request the image, which sends the URL automatically, including the embedded data, to the attacker’s server.

  source