Reader ROI
» The four aspects of good e-mail management: storage, archiving, indexing and policy enforcement
» How proper e-mail management can boost regulatory compliance

E-mail's usage and scope is exploding. IDC (a sister company to CSO's publisher) forecasts that the average number of e-mails sent each day worldwide will hit 36.2 billion in 2006, and Gartner predicts the volume of business e-mail will grow 25 percent to 30 percent a year through 2009. (Gartner's figures exclude spam, which currently accounts for around three-fourths of inbound e-mail.) This growth reflects an important shift in how e-mail is employed. The Enterprise Strategy Group (ESG) reports that as much as 75 percent of most companies' intellectual property is contained in the messages and attachments they send through their e-mail systems.

"E-mail has become the primary medium for how we communicate," says Jeffrey Schwarz, a partner at McDermott, Will & Emery. "Four years ago we used paper and FedEx. Now almost everything is done over e-mail." The consequence is that e-mail has become a de facto record repository, a burden that e-mail systems as we know them can barely handle. "We are trying to make a system do something that it wasn't designed to do," says Schwarz, who is also the top IT officer for the $668 million firm. "E-mail wasn't designed to be a document repository. It was meant to be send, read, delete. But now you can't delete. There are regulations that don't let you do that."

Many CIOs thought they had nailed e-mail systems in the 90s and could move on to more important things, but the kind of search required by the new regulations is beyond the capability of most current e-mail systems. Simply adding more storage isn't nearly enough. Consider that over the next seven years, a company with 20,000 employees will have to save approximately 4.5 billion e-mails, and it must be able to search through them all to find messages relevant to a request for information in a matter of days or hours.

"These new [regulatory] obligations require you not just to save more e-mails, but to be able to access them promptly," says Carl Metzger, a partner specializing in securities litigation at Testa, Hurwitz & Thibeault. "CIOs who have ignored these requirements need to take their heads out of the sand." It's high time for all CIOs to re-examine their e-mail management systems.

Setting aside the question of regulations, a good e-mail management policy is a good business practice. Qualcomm senior vice president and CIO Norm Fjeldheim says his company saves every e-mail sent or received to fend off potential patent violation lawsuits. Yet a 2003 study by the Association for Information and Image Management and Kahn Consulting found that 60 percent of companies have no formal e-mail retention policy.

Storing and searching messages on a large scale requires a new approach. This approach has four different but interrelated components: storage, archiving, indexing and policy enforcement. For the most part, it is a seamless change for users and a straightforward initiative for CIOs. "This isn't reinventing the wheel," says Vincent Cottone, vice president and director of infrastructure service for mutual fund company Eaton Vance. The key to the new e-mail management is several technologies that are coming of age - and consequently coming down in price. Cheaper disk storage lets CIOs store e-mails in a searchable format, and archiving and indexing software gives these messages the meta-data that makes searching possible on the required scale. And it all happens on the back end.

Why the Old Way Doesn't Work

Exchange and Lotus Notes, the two dominant e-mail platforms today, were not designed with today's e-mail management needs in mind. These systems were made to communicate messages, not to become a company's primary document repository - and certainly not to give CIOs control of all their companies' e-mails. In fact, Exchange's personal folder storage system is in a sense the opposite of what a compliance-minded policy calls for, in that it allows users to remove messages from the server and store them locally.

In the past, e-mail management was a matter of buying more servers and backing up onto tape. But tape is an insufficient medium in a regulated environment. First, it breaks. According to Peter Gerr, an analyst at ESG, only 70 percent of companies have a tape recovery rate greater than 80 percent. Second, it takes too long. Qualcomm's Fjeldheim says the standard turnaround time to find e-mails requested by his legal department on his tape backup is three or four weeks. That may be acceptable for legal discovery or an internal investigation, but it will get you into trouble with regulators. In the US, for example, Bank of America was fined $10 million in March 2004 when it failed to turn over e-mails to the Securities and Exchange Commission (SEC) in a timely manner (currently interpreted as only 36 to 72 hours).

Switching to disk storage technology is part of the answer, and it is easy enough now to buy disk storage instead of tape. Prices are coming down; a terabyte of disk storage today costs a sixteenth of its price in 2001, according to Gerr. But simply switching from tape to disk doesn't solve the more fundamental problem of search and recovery. Gerr says that just as with tape, e-mails on disk are hard to search unless they are indexed. "Exchange and Lotus don't have native tools to index all the incoming and outgoing messages," he says. For the time being, that capability needs to come from third-party software that can intercept e-mails as soon as they hit the mail server, index them and send them to an archive.

The New Method of E-mail Management

When David Taylor became CIO of the US state of Florida's Department of Health in March 2003, he started telling anyone who would listen that the organization had to change the way it managed e-mails. There were 17,000 users who together produced about 3 terabytes of e-mail a year. And because of Florida's Sunshine Law, which lets any citizen obtain a copy of any government document, Taylor had to save every single e-mail. The department met the challenge, for the most part, but it was doing so in an undependable way.

For starters, the department's document retention policy relied on individuals for enforcement. "Every employee had to understand the records requirement," Taylor says, which meant that employees applied different interpretations and levels of diligence. "There is no need to save an e-mail that says, 'Hey, let's go to lunch.' But it was up to the individual to make that decision," he says. One of the most glaring problems with this approach was that users tended to keep e-mails in their inboxes, which sometimes grew to 20,000 e-mails and 4GB to 5GB. Trying to find any single e-mail in this setup placed a tremendous strain on the e-mail server. "A lot of people would feel it if a person with a 4-gig mailbox did a sort by name," says Taylor.

Whenever the legal department needed to retrieve requested communications about something complicated, such as information that led to a procurement decision, the system handicapped the department. "It could be a major fishing expedition," says Taylor. Someone in IT would have to find and restore the backup tapes and then search through all the e-mails to find the right ones. But sometimes e-mails might get inadvertently deleted from users' inboxes before the backup was performed. Even figuring out if they were searching the right tapes was a difficult matter; the burden fell to the requester of the e-mails to provide very detailed information, a date range and who the senders or recipients were. There was also the question of how many tapes to restore. "We did due diligence, but that was all you could do," says Taylor. "There was no guarantee you could find something."

To break his dependence on the users, in 2003 Taylor set up what he refers to as a vault. As soon as an e-mail arrives at the Department of Health's Exchange server, a copy is automatically sent to the vault, which is actually a Centera storage device from EMC. The change is seamless for users; they still send and receive e-mails, and can store them anywhere they want. It's just that the department now has a master copy as well.

Storing e-mails is just the beginning. A comprehensive approach to e-mail management - one that will not only meet regulatory requirements but will also actively prevent violations - includes indexing and policy enforcement.

INDEXING. Having an e-mail archive helps relieve the burden on servers, and it's easier to search than tape backups. However, good e-mail management doesn't stop there. To fully enjoy the benefits of an archive, CIOs need to create an index that captures key information about each e-mail. In the past, the simple search tools that came with an e-mail program would suffice; administrators could enter keywords and search the server. But McDermott, Will & Emery's Schwarz says the usefulness of that kind of searching is long past. "Consider your closet. That is what e-mail was in 1999," he says. E-mail volumes were small enough that searching for any particular item was simply a matter of sorting through a few alternatives. Nowadays, says Schwarz, a company's e-mail is like the inventory for Wal-Mart: "It's too big to go through by hand. You need to have a logistics system to manage it."

For this, CIOs need software that analyzes messages bound for the archive, creates meta-data tags that identify the sender and recipient, and performs a context-based analysis of the message. John Hegner, vice president of technology services at Liberty Medical Supply, deployed such a system in July 2004. His business is subject to US Health Insurance Portability and Accountability Act (HIPAA) requirements and Medicare audits. "If a customer complains that something happened, we can search through e-mails, based on a name or a number, to see if there is a record of it," says Hegner. "Same if an employee claims he never got an e-mail from a customer or a business partner. We can determine if that is true or not." A search through a terabyte's worth of e-mails takes just a couple of minutes. The search results return a list of e-mails from the indexing directory. If Hegner wants to check out a particular e-mail to see if it's what he is looking for, he simply clicks the link to the archive.

POLICY ENFORCEMENT. Proper e-mail management requires an early warning system for violations. After all, it's ridiculous to go to the effort and expense of indexing and archiving e-mails if you're just making it easier to find incriminating evidence after the fact.

One of the benefits of reviewing e-mails is that inbound and outbound messages that test positive for certain illegal terms or context can be flagged and routed to a compliance officer for review. To protect their companies, CIOs need to be able to stop e-mails that violate policies before they go out. "If something goes wrong, you don't want to have to explain to your board how difficult it was to detect," says Metzger of Testa, Hurwitz & Thibeault.

After the SEC cracked down on brokerages in late 2002, the unnamed e-mail manager of a large company installed a monitoring tool to make sure that all e-mails sent and received by employees complied with regulatory requirements. "We had had communications policies [in place] for five years, but we could never implement them," he says. The new policy tool searched for certain words, terms or meanings in e-mails that would trigger an alert, and then routed the e-mail to a compliance officer who needed to give approval before the e-mail would be sent.

The results were shocking, says the e-mail manager. Despite the looming presence of the SEC, employees were still sending several hundred e-mails a week that violated US federal regulations. "The majority were honest mistakes," says the manager, such as a research analyst sending a report to an old distribution list that included traders (in contravention of the "Chinese Wall" separation of research and trading functions). But some of the violations appeared intentional, he says. With the new e-mail system, the company's compliance officer is able to confront the offender before it becomes a criminal matter.

How Much E-Mail Management Is Right for You?

Some organizations - especially small, private, unregulated companies - can go without new e-mail management software, or at least can wait before investing. And there are benefits to waiting; disk storage and e-mail indexing and monitoring software are both becoming commodities. While setting up a system that can archive, index and monitor costs between $5 and $25 per user per year today (the exact cost depends on the size of the company), those prices are expected to be substantially lower within two years, says ESG's Gerr.

Most companies, however, are subject to at least one of the myriad regulations that necessitate a new approach to e-mail management. And while prices will invariably come down, the risks of waiting are too high. "Compliance is the catalyst that makes it more urgent," says Schwarz. "If HIPAA and Sarbanes-Oxley and other regulations hadn't come out, we could have taken more time to address e-mail management."

And while the price for management systems can run upwards of $US500,000, the potential avoided cost for regulated companies is worth it. A single fine from the SEC or another regulator can easily outstrip that.

But CIOs looking for a traditional ROI can probably find that too. Archiving e-mails onto searchable disk storage means you can take them off more expensive servers. And while the quantity of e-mail that companies need to save is soaring, most archiving software saves only one copy of each e-mail and corresponding attachment - whether it was sent to two, 10 or 50 people in the company. That means the overall number of e-mails being saved might actually go down. Also, recovering e-mails from tape takes a long time. And time, of course, is money. In the case of a regulated industry, that time can lead to millions of dollars in fines.

What's more, the e-mail management crisis offers a rare chance for the CIO to be the hero. Eaton Vance's Cottone, for example, brought the issue to the attention of his compliance officer. "[Regulatory compliance] is not a traditional IT job," he says. "But this is big for us. We need to be aware of everything." By extending his scope, Cottone enhanced his standing and protected that of his company. "Your reputation is at stake," he says, "and you can't put a price on reputation."