AI Prompt Poisoning via Fake “Summarize with AI” Buttons Threatens Content Integrity

Microsoft Defender Security Research uncovered a new attack chain in which threat actors compromise a website and insert a counterfeit “Summarize with AI” button. When a user clicks the button, the request is sent to a legitimate AI model, but the attacker‑crafted prompt manipulates the model’s output, effectively poisoning its memory and steering the chatbot’s recommendations toward attacker‑chosen narratives.

The poisoned responses can be used to amplify misinformation, bias search rankings, or deliver malicious advice that influences user behavior. Defenders need to watch for unauthorized UI elements that invoke external AI services, enforce strict content‑security policies, validate referer headers, and monitor outbound AI request patterns for anomalies. Prompt sanitization and logging of AI interactions are essential controls to mitigate this emerging vector.

Source: Read original article