Spam checking content urls

As outlined in previous chapters, RBL use can be problematic. They tend to be very inaccurate - even the best RBL sites include legitimate email sources and exclude sources of spam. But there's another tool our box of software tools, that is very effective at spam filtering.

Most spammers have gotten very clever at beating spam filters. Encoding techniques, mispellings, spoofing, image messages, all kinds of techniques. But the one thing they must include the email is a url. Without a url, the recipients have no easy link to purchase the spammers offerings. Even if there's nothing but an image in a spam, there's probably a link associated with that image that will take your browser directly to a pill peddler or morgage broker.

These embedded urls can be checked just like RBLs are checked.

But why are content url lookups so much more effective than RBL lookups? - When a spam is sent to you, the sending domain name is usually spoofed. We've all received phishing scams from 'PayPal.com', asking us to login and verify account information. They sure look official until you notice the url provided in the email does not point to paypal.com. It usually points to an IP address.

Any email with a link using an IP address is suspect to begin with. If the url includes a domain name, then it's quite a simple matter to perform a lookup on that domain name to see if it's a spammer. The domain name in a url cannot be spoofed so easily (note we're talking about the actual link url, not the blue clickable text). A domain name needs to be registered, and DNS records created for the click to work. DNS records alone can take up to 72 hours to propagate, so spammers cannot just setup a site for a few days and then move on without incurring the cost associated with registering a domain and setting up the DNS records.

This means that spam checking the urls in the contents of an email is a very effective means of filtering spam. Content url spam filtering is a feature added to SpamAssassin version 3.x and is well worth an upgrade if you are sill using version 2.x

In conclusion, a spam email needs a link for the recipient to click on. For technical reasons, spammers cannot simply setup shop, spoof, and move along, so content url spam filterng becomes a very effective software based tool for blocking spam.