SPAM - Unsolicited e-mail is a problem for Australian business.

So tell me something I don’t know, you’re thinking.

Spam uses up valuable IT resources and reduces the effectiveness of e-mail. It can offend and distress employees through the propagation of pornography, trap gullible staff members in financial scams, and significantly reduce the productivity of employees. Spam can bear malicious code, resulting in security incursions and downtime for IT systems.

Yes. Been there, done that, you say.

Less measurable is the fact that spam can negatively impact on trade, as e-mail users are reluctant to publish their e-mail addresses openly in the fear of getting on yet another spammers list.

A January 2003 report from Ferris Research (www.ferris.com) — “Spam Control: Problems and Opportunities” — found that in the US spam accounted for between 15 and 20 per cent of inbound mail at typical corporations and 30 per cent of inbound mail for ISPs. Ferris further claimed that the average cost due to lost productivity, consumption of IT resources and help desk costs would increase to $US14 per month per user in 2003.

Just last month, in a UK study, “Spam: Report of an Inquiry by the All Parliamentary Internet Group (APIG)”, the Radicati Group (www.radicati.com) claims that a “ . . . company of 10,000 users with no antispam solutions will spend on average $US49 per year per mailbox in processing spam messages”.

Also in the APIG report, a UK university claimed that the “ . . . direct costs of their spam-filtering system were £78,000. However, it [filtering] is still costing them an estimated £1.1 million per annum, assuming that staff can deal with the spam that gets through the filters in a mere two minutes each per day.”

Hmmm . . . now you’ve got my attention, you’re saying.

Money Down the Drain

The direct and lost opportunity costs from spam can be divided into four key areas.

Reduced employee productivity. With spam volumes up to 50 per cent, employees are spending a significant amount of time sorting through unwanted spam e-mail to find legitimate business e-mail. This task is only made worse by spam’s growing use of misleading subject lines. There is also a cost from the time wasted by employees who read spam e-mails, click on spam links, and discuss particularly entertaining spam with colleagues.

Even when a spam filtering solution is put in place, there may be productivity costs for staff checking their quarantine message store and maintaining their own lists of e-mail addresses to always let through.

Increased IT infrastructure capital expenditure. Costs include additional e-mail and networking equipment to maintain e-mail service quality, bandwidth costs from unwanted spam data across Internet links, and the staff costs to maintain and administer these additional loads. Extra storage space may be necessary due to corporate governance rules for e-mail archiving.

Reduction in effectiveness of e-mail as a communications channel. More than just a disruption to business, as spam mail volumes head towards the one in every two e-mail level, spam begins to undermine the effectiveness of e-mail as a communications channel. This reduces the advantages provided to business by e-mail and negates the significant investment made in e-mail infrastructure.

Potential for human resources problems. The distribution of pornographic mail is an unwanted side effect of spam. A significant proportion of spam e-mails contain links to pornography sites or html calls to load pornographic images with links to pornography sites.

For some employees this unwanted pornographic intrusion could offend. A continual barrage of pornographic spam has the potential for causing stress to such employees.

Legislation to the Rescue?

The federal government’s “spam Bill 2003” passed the House of Representatives in October. While the legislation will provide the ability to deal with spam originating in Australia, it will do little to actually stop spam hitting the desktops of local workers because most of it originates in other countries.

In other words, CIOs looking to reduce the impact of spam on their organisations in both the short and long term still need to look to technical solutions. At the fore of these solutions is the use of a companywide spam filter, which can at least moderate the pain of spam. While filter technology is not perfect, Nucleus Research recently reported that the “use of such a device reduced the average cost per employee by 26 per cent to $US650, or 5.0 minutes per day, per employee”.

The two key locations for filtering are at the perimeter of the corporate network and at the desktop.

For mid- to large-size organisations, filtering before spam enters the organisation’s e-mail infrastructure is far preferable. It reduces load on internal networks, traps potentially malicious code before it enters the organisation, allows for easy centralised management and minimises the data storage impact of spam. In addition, many spam filtering solutions also now provide virus scanning, usually using an antivirus solution from a leading vendor.

These solutions can either reside within the corporate network, generally at the network perimeter, or be supplied by a service provider and located outside the corporate network. Using a service provider and locating the solution outside the corporate network offers the ability to reduce congestion on Internet links, eliminates the need for capital equipment purchases and does not require the cost of hiring and training staff.

However, it does result in filtered mail passing through, and often being stored by, a third party. And there are the usual service provider issues to look out for such as service level guarantees as well as the ability to fine-tune the filtering solution. Solutions located inside the corporate network can be appliance- or server-based. These solutions generally feature frequent updates reducing the time between the vendor’s identification of spam and filtering of the spam by the customer’s installation.

CIOs interested in spam filtering within the corporate network but who don’t want to invest time and money in staff and training can investigate managed service options.

Whatever the solution, CIOs need to be confident that the filtering system can adequately scale to meet the expected volume of e-mail for the organisation at acceptable service levels.

Different Approaches

There are a large number of techniques used by spam filtering software to detect spam.

Blocklists, blacklists or real-time blackhole lists: These are either public or private (user or vendor-maintained) address lists that contain lists of IP addresses or domain names of known spam sending systems. Public blacklists are susceptible to the listing of legitimate mail servers.

Whitelists: Again, these can be either public or private (user or vendor-maintained) and identify legitimate mail sources. While the whitelist participants may be a source of spam (for example, through error in mail server configuration) they are still legitimate senders of mail.

Honeypot or unpublished e-mail address methodology: Spam filter vendors set up e-mail addresses on servers but do not sign up to e-mail services or use the addresses for commercial purposes. E-mails received to the e-mail addresses in this way are almost certainly spam. The e-mails are “fingerprinted” and the information distributed from the vendor to the spam software at the customer site. This is similar to distributing virus definitions.

Content analysis: This is used for both e-mails found in the honeypots as well as e-mails as they come into the organisation through the spam filter. As spammers become more sophisticated they work their way around simple content filtering techniques — consequently spam filtering vendors must continually update content filtering techniques. Techniques used include lexical analysis, artificial intelligence, neural networks, Bayesian probability, statistical analysis and heuristics.

Where Did My Message Go?

Whether you select spam filtering as a service provider solution, an appliance, server-based software or a managed service, your users will judge the effectiveness of the solution by the number of legitimate business e-mails that the spam filter identifies as spam, commonly known as false positives.

False positives cause users consternation due to important e-mails not being received and time spent checking message quarantine areas.

It’s important to accept that with current tools it’s pretty much impossible to eliminate spam altogether and that the closer you get to total elimination the higher the risk of false positives.

Look for a solution that uses multiple identification methods and which has granular settings that can be made by the administrator. Particularly important is the ability to set private whitelists to ensure that e-mail from genuine sources gets through no matter what.

Ben Gerholt is CIO of IDG Communications (publisher of CIO magazine) and is also responsible for IDG’s Web sites and e-mail services. He has over 10 years experience analysing and writing about information technology products, services and solutions. You can contact him at ben_gerholt@idg.com.au

This article appeared in Essential Technology, a new technology-oriented section of CIO magazine.