WordAdjust - A Deobfuscation Frontend to Content-Aware Anti-Spam Tools
Authors
Sørensen, Lars Tabro ; Larsen, Martin Møller
Term
4. term
Education
Publication year
2008
Abstract
Spam-mails er en vedvarende udfordring, og afsendere finder hele tiden nye måder at slippe forbi filtre. Denne afhandling fokuserer på tre udbredte sløringsteknikker: brug af lignende Unicode-tegn, ombytning af bogstaver (scrambling) og bevidste stavefejl. Vi introducerer WordAdjust – et frontend deobfuskationsfilter (et forfilter, der afkoder slørede ord) til anti-spam-værktøjet SpamAssassin. WordAdjust afkoder disse tre typer sløring, så teksten bliver mere læsbar for SpamAssassin. For bevidste stavefejl anvender WordAdjust N-gram fuzzy-søgning, som deler ord op i små bogstavsekvenser og finder omtrentlige matches. SpamAssassin har svært ved at håndtere e-mails, der er sløret med disse metoder. I vores forsøg øgede WordAdjust den gennemsnitlige SpamAssassin-score med 56% på bevidst slørede spam-mails. Yderligere forsøg viser, at WordAdjust i gennemsnit gør det muligt for SpamAssassin at opfange 10% af den spam, der ellers ville være endt i brugernes indbakke. Resultaterne peger på, at et enkelt forfilter til afkodning markant kan styrke eksisterende spamfiltre.
Spam emails remain a persistent problem, and senders constantly invent new ways to bypass filters. This thesis focuses on three common obfuscation techniques: using look‑alike Unicode characters, scrambling letters, and intentional misspellings. We introduce WordAdjust, a frontend deobfuscation filter (a pre-filter that recovers obscured words) for the anti-spam tool SpamAssassin. WordAdjust reverses these three types of obfuscation so the text is easier for SpamAssassin to interpret. For misspellings, it uses N-gram fuzzy search, which breaks words into short letter sequences to find approximate matches. SpamAssassin struggles with emails obfuscated in these ways. In our experiments, WordAdjust increased the average SpamAssassin score by 56% on intentionally obfuscated spam emails. Further tests show that, on average, WordAdjust enables SpamAssassin to catch 10% of the spam that would otherwise reach users’ inboxes. These results suggest that a simple deobfuscation pre-filter can significantly strengthen existing spam filters.
[This abstract was generated with the help of AI]
Documents