A clever technique used to hide malicious prompts in attacks on AI agents has been adopted by spammers to evade filters on email platforms that are designed to flag unwanted messages used in mass campaigns.
The technique is broadly known as ASCII smuggling. It gained attention two years ago as a means of making a class of AI attack known as prompt injections more stealthy. Malicious instructions embedded in emails or other untrusted content to be processed by an LLM aren’t written in ordinary text. Instead, they’re rendered by a special range of Unicode tags. For example, the tag point U+E0041 mirrors “A,” and U+E0061 mirrors “a.”
No longer just for obscuring prompt injections
The block of 128 tags mimics a portion of the American Standard Code for Information Interchange almost perfectly, with one major difference: the characters they encode are readable by computers but, by design, are almost completely invisible to humans. By expressing the malicious prompts in these tags, LLMs detect the instructions, but people reading the email never see them. There’s much more about ASCII smuggling here.
Earlier this year, Microsoft started seeing a massive increase in spam messages that used the technique. Beginning on one day in early February, the number of ASCII smuggling signatures detected by Microsoft Defender for Office spiked from roughly 21,000 per day to more than 1.3 million. Within four days, signature detections jumped to 2.5 million. The deluge persisted for months and then fell off sharply in mid-May.
“Because tag characters are invisible to humans but exist at the text-processing level, the same property that makes them useful for smuggling instructions into a model also makes them useful for obfuscating keywords before a detector evaluates them,” Microsoft explained Thursday. “The intent is inverted, but the mechanism is similar, and a user’s suspicions are not raised.”
A previous Ars artilce (linked to in this article) goes into more detail:
https://arstechnica.com/security/20...isible-text-creating-an-ideal-covert-channel/
It's interesting that the Ars article calls the flag modifier function essentially "depreciated," when there's this comment:
And Wikipedia:
https://en.wikipedia.org/wiki/Tags_(Unicode_block
Suggesting the flag modifier function isn't so much "depreciated" as "not widely implemented."