Spam Proofing Your Web Site

Spam sucks.

I think we're all in agreement with that sentiment. But did you know that you're literally giving spammers your email address whenever you post it in plain view on the internet?

Yup, they send their little automated spambot email harvesters all over the web looking for email addresses. Once collected, they're sent on to the mailing houses who then proceed to deliver all sorts of useful information on how you can save money on your mortgate.

I receive about 200 spams a day to my work address. And while I've developed methods of dealing with all the junk, it would still be nice to just not have to deal with it.

So for this month's article I've decided to take the suggestion of a reader and describe ways you can keep your email address hidden from the spambot email harvesters.

How Spambot Harvesters Find Your Email Address

An email harvester is an automated software program that visits web pages and searches through the HTML source code of the page in search of patterns of text that match the format of email addresses. Since email addresses must be in a certain form, it's really easy to write software to do this.

So the key to defeating these harvesters is to make it so a text string matching the pattern of an email address does not appear on the page. That's not to say you can't have your email address on the page, it just can't look like it.

Of course, you still need your visitors to contact you. It's convenient for them to click on your mailto: link and have their email client pop up with the To: address already filled in.

Don't worry, it's still possible to do this while protecting your address at the same time.

How To Deal

There are a few different categories of methods for keeping your email addresses inaccessible to spambots while still allowing your legitimate customers to contact you. Some work better than others. Here they are, in summary form.

1. Javascript
2. Email Obfuscation
3. Contact Forms
4. Hard-core back-end server stuff that's probably more trouble than it's worth

1. Javascript

Since harvesters look for email addresses, you can use Javascript to separate the different parts of your email address, and then recombine them in the browser using Javascript.

The disadvantage to this method is that if the visitor's browser does not support Javascript, they will not be able to see the email address.

Here are some example scripts that make this work:

Example 1: The most basic example

... would look like ...

Example 2: Another basic example

... would look like ...

Neither of the above examples use a mailto: link. But that's still possible:

Example 3: A clickable mailto link with link text

... would look like ...

Example 4: A clickable mailto link displaying your email address

... would look like ...

Example 5: Using inline javascript

... would look like ...

Example 6: Escrambler Generated Javascript

There are online javascript email scramblers... Here's one:

http://innerpeace.org/escrambler.shtml

That scrambler creates code which looks like this:

... which then looks like this...

As you can see there are a number of ways to use javascript to hide your email addresses. So which should you use? The answer: all of them.

As more people begin hiding their addresses using these methods, the people designing email harvesters will begin to incorporate detection of the most popular methods. If everyone used example 3, for instance, the email harvesters would quickly include support for detecting that type of code.

The solution is to use a variety of the methods above, and even come up with some of your own.

Disadvantages of the Javascript method

As mentioned above, the primary disadvantage to using Javascript is browser compatibility. The vast majority of people are using Javascript-capable browsers. However, some people choose to surf the web with Javascript disabled with the aim of protecting themselves from malicious web pages. If such a person visits your site, your email address will not be displayed.

2. Email Obfuscation

Another way of making your email address not look like an email address is to obfuscate it. Here's an example of what an obfuscated email address might look like:

That translates to...

Here's a site which offers an obfuscation script...

http://www.healyourchurchwebsite.com/obfuscator/

Google has lists of other email obfuscators...

http://www.google.com/search?q=email+obfuscator

Disadvantages

Again, the key is to use a variety of methods to hide your address. As the script below demonstrates, any technique can be broken given a concerted effort.

This site detects a number of commonly used obfuscation techniques. Try it out on your page to see if you're really protected:

http://willmaster.com/possibilities/demo/RetrieveEmails.cgi

3. Use A Contact Form

This technique is guaranteed to work regardless of browser settings. It protects your email address because it never appears in the web pages source code. Instead, the email address is stored in the protected source code of the script. Spambots do not have access to this code, so are unable to access your email address.

Here's a simplified example of a contact form which utilizes the NMS FormMail Script:

Go ahead, send me a message!

I've only included two fields in the above example, but NMS FormMail (and any other decent contact form script) allows any number of fields, and is otherwise highly configurable. Visit the NMS FormMail site for complete details.

There are a few benefits to using a contact form on your websites. The first and most obvious is that it keeps your email from the spambots. If you view the source code of the page, you'll see this entry in the form code above:

Instead of placing my email addresses in the recipient field, I can simply use a number which coincides with a list of email addresses in the source code of the script. Choosing "1" sends the email to Customer Service, choosing "2" sends the email to my personal in box, and so on. In the above example the email is sent to both of those addresses. I can set up any number of email address / number combinations.

The other less-obvious benefit of using a contact form is that it can help messages sent through the system traverse any spam filters you have set up. Since you can hard-code the subject line of the generated email, you can include a special string of characters which you can then include in your spam filters. Whenever the filter encounters this string, it can assume that the message is okay and send it directly to your email client. This could all but eliminate delayed or lost email due to your spam filtering.

NMS FormMail is free software available under the GNU Public License, or the Perl Artistic License, which means you are free to modify and re-distribute it so long as it's made available under the same license.

Download NMS FormMail here: http://nms-cgi.sourceforge.net/

Disadvantages

The draw-backs to this system are minimal. Some people are hesitant to enter information into forms, doubting that they'll ever get a response. There also isn't a copy of the sent message left in their out box as when sending a regular email, however, the script can be configured to email a copy of the message back to the sender for their records.

Note: There are older versions of FormMail available, notably from Matt's Script Archive, that are insecure and can be hijacked by spammers to send mail. NMS FormMail, using it's default configuration, is not susceptible to this hijacking.

As always, read the documentation for any script you install on your servers to ensure you are configuring it in a secure manner. You are responsible for the security of your server.

Protecting Your Contact Directory Behind A Form

One thing that email harvesters are not currently doing is submitting forms. This means that you can require the visitor to click a "View Contact Directory" button which will then display the contact directory page containing your email addresses.

Here's an example of this in action:

http://simon.incutio.com/contact/

5. Now We're Getting Hardcore

The following techniques are probably a bit beyond mere mortals such as you and I. Venture at your own risk...

Using Linux, Apache, mod_perl, Perl, MySQL, ipchains and Embperl to Trap Spambots: http://www.neilgunton.com/spambot_trap/

Using mod_rewrite To Block Spambots: http://diveintomark.org/archives/2003/02/26/
how_to_block_spambots_ban_spybots_and
tell_unwanted_robots_to_go_to_hell

These guys have way too much time on their hands, but I must admit it's an interesting project: http://www.kungfugrippe.com/previously/002462.php

Well, that just about covers it... As you can see, there's a wide variety of possibilities. The key to continued email stealth is the use of a number of different techniques.

Best regards,
Ian Cook