Learning records management discipline Document management has helped governments manage more information than ever — but how do you use that information once it’s archived? The answer lies in convergence between records and document management solutions.

Document management has become part of everyday life at the City of Melbourne, which has digitised and indexed more than a million paper and digital documents since it began using the Hummingbird DOCS knowledge management system half a decade ago. Yet despite its successes so far, the agency is facing a new challenge as it tries to apply more discipline to an ever-growing volume of information that has exploded as e-mail is increasingly considered an essential government record.

Growth in the everyday use of email has presented a major challenge for government organisations around the world, since conventional records management (RM) legislation has addressed the management of paper documents but has offered few answers when it comes to electronic records. The general trend today has been towards the treatment of e-mail and other digital records as business documents with the same legislative weight as paper documents — and that means those documents must be retained and categorised.

However, such information is intrinsically ephemeral: potentially critical documents can be wiped out with a single keystroke. Given that the City of Melbourne has worked hard to make its knowledge management system as unobtrusive as possible — so users do not feel they are being forced into something they do not want to participate in — the organisation has had to balance the need for comprehensive archiving with the desire not to put users offside.

The result, here as at government organisations across the country, has been a lot of lost information.

“Whilst we’ve got the ability to save an e-mail directly into DOCS, we’re finding that it’s not used a lot because it’s not a mandatory feature,” says Mike Healey, manager of information and technical services with the City of Melbourne. “We’re losing another set of corporate documentation that’s now sitting in the e-mail system. There’s a whole lot of junk in e-mail so you wouldn’t want to save everything — but spread amongst the junk is a lot of important business dialogues that need to be saved.”

The council’s solution is an upgrade of its DOCS environment to a system combining Hummingbird’s KM (knowledge management — indexing, searching and repurposing information across the organisation), DM (imaging and storing paper and digital documents) and RM (for applying formal records storage and life cycle management policies) tools.

Set to go live by October, the system will hook into users’ Microsoft Outlook e-mail systems to provide a simple way of archiving important e-mails by dragging and dropping. It is hoped that, by teaching users the importance of RM discipline, they will actively contribute to building an even more useful and structured knowledge base.

DM, meet RM

Across all layers of government and business, departments face increasing pressure to formalise their record keeping. Technology, however, has lagged far behind their needs.

On the one hand, RM vendors like QRMS, Accutrac, IMR, Zasio, Versatile, RecFind, Triad and InfoLinx targeted formal and well-trained librarians with powerful but esoteric solutions that enforced rigid hierarchies on innumerable boxes of paper communications, customer records, forms, microfilm, microfiche and other physical records.

On the other, DM vendors like OpenText, Hummingbird, Documentum, FileNet, Objective and Tower Technology have enjoyed success doing the heavy lifting of information management — the scanning of paper pages, archiving and version control of electronic documents, and sorting, retrieval and presentation of both types of information.

Designed for ease of accessibility and indexing, however, such solutions have lacked the highly granular discipline mandated within a comprehensive RM framework like the US DoD 5015.2-STD, the UK Public Records Office (PRO) and Australia’s own Victorian Electronic Records Strategy (VERS), administered by the Public Record Office of Victoria.

This quickly presents problems for those trying to manage content within a structured hierarchy. Even though they support the entry of metadata, for example, DM solutions may not specify what that metadata includes. That leaves users, none of whom are typically trained in the whys and wherefores of responsible records management, responsible for the integrity of the RM structure.

“Most departments have struggled with how they can take this [classification] function that used to be a departmental function, and suddenly ask users to become record keepers,” says Hummingbird sales director Rob Whiter, who concedes vendors have struggled to improve the situation.

“You had [small] records management companies with the know-how but not the ability to execute. But RM systems were never meant to be DM systems: they have conceptual logic in them, but to the uninitiated they’re confusing. The thing that DM brought to the table was usability. None of the DM companies had any clue what RM meant, but they were already producing a system that had 90 per cent of its requirements. The only thing they needed was the ability to file against a classification; they were so close but didn’t know it.”

Resolving this disparity has taken time as DM vendors work through crash courses on RM hierarchies. Various vendors have resolved the problem either through acquisition or partnership: Open Text, for example, bought RIMS, Documentum bought TrueArc, and last November IBM bought Tarian Software to add RM capabilities to its Content Manager suite and Lotus applications. FileNET, iManage, Optica, Stellent and Hummingbird took the partnership approach.

The result has been a gradual convergence of the DM and RM worlds that is this year culminating in what analyst firm META Group believes will be resurgence in investment in enterprise content management (ECM, a category into which DM and RM are folding).

There are signs this resurgence is already happening. Objective Corporation, for example, was recently appointed to the GSAS ITS 2323 RIMS panel contract to supply RM systems to the NSW government, with the first taker the NSW Office of the Director of Public Prosecutions. Objective’s combined RM and DM system has even been chosen by no less than the UK Public Record Office, which advises UK government agencies on RM best practice and defines the standards they should use.

Noting that ECM topped a recent survey of IT spending priorities in around 400 large businesses’ IT budget priorities, META believes the worldwide market will grow to be worth $US10 billion next year as increased legislative and governance pressure forces both private sector and public sector organisations to improve information management across the board.

Chasing the Virtual Paper Trail

Given that e-mails must now be factored into knowledge management strategies, government departments are recognising the need to apply RM to what has become a major vehicle for corporate knowledge. E-mail management must be addressed early on, and given the massive volumes of e-mails sweeping through government departments it is important to formulate a clear strategy for minimising e-mail clutter while maximising the retention of important e-mails.

“We’re aware we need to keep a complete record of the organisation’s business transactions, and at the moment we have a gap in that e-mail area,” says the City of Melbourne’s Healey, who sees closing that gap as a serious problem. “Because of the way e-mail is now used, and the extent of the noise in the e-mail inbox, it’s going to be very hard to siphon off the noise and decide what’s real.”

It is possible to close this gap by setting up shunts that automatically copy, index and archive all e-mails sent and received within the company.

In this approach, solutions such as Legato Systems’ E-mailXtender, IXOS eCONserver, Veritas NetBackup Storage Migrator and UK company ArchiveIt intercept e-mails and subject them to centrally defined archiving policies. This could include, for example, keeping a copy of e-mails on online disk for several months before deleting them or moving them onto permanent storage for archiving and indexed searching. That way, at least, the need to archive e-mails has been met, albeit with little opportunity to add value through metadata.

With trillions of e-mails circulating the globe annually, that can easily become a very storage-intensive process — and managing that storage necessarily involves expenditure on both disk space and relevant management tools. Osterman Research, for one, recently reported that the average Microsoft Exchange user’s mailbox consumed 72MB of disk space. One in six users, the survey found, has a mailbox larger than 100MB.

Dedicated e-mail storage management tools include a number of tricks for keeping disk consumption down: for example, many can replace file attachments with pointers to a single copy of each attachment, so that forwarded attachments do not consume undue amounts of disk space. Disk space consumption can also be limited by filtering spam at the e-mail gateway.

While dedicated e-mail management applications provide the means for government agencies to maintain running archives of e-mails, they lack the filing discipline and highly granular control necessary for RM to be properly extended into the e-mail world. In the short term, some organisations have filled the gap by having employees print important e-mails, then treating the printout like any other paper record. But with e-mail volumes continuing to increase, this practice is unnecessarily complicated and untenable in the long term.

Eventually, e-mail management will be integrated into conventional information management systems through the combination of specialised e-mail storage management with formal RM and DM processes. This integration was a key focus of Tower Technology’s Seraph application, launched in March and targeted specifically at government users.

Seraph is built around a browser-based interface that integrates e-mail, instant messaging and Web sessions along with conventional document and fax imaging. Combining capture, collaboration, case management and compliance functions, the system has been designed from the ground up to cater for government environments by facilitating collaboration around case management disciplines, according to Tower product marketing manager Adrian Foote.

“A lot of people have taken the step to go to an RM system but they’re too difficult to use,” Foote explains. “It’s because of their heritage, and they’re not scaling [to the volumes present in electronic environments]. But RM principles are at the very foundations of Seraph. And because we’re not cobbling together a bunch of different technologies, we can set auditing, access to records, movement and processing of cases, and so on. To provide that access, you need to have all the information captured and available.”

Integrating the two types of solutions might be a good direction for systems integrators in the future, but for now their lack of integration presents a difficult situation for government departments keen to find a balance between archiving everything and archiving nothing.

Getting Users in the Know

Ultimately, whatever approach a government department takes will have little utility if it is not supported by the users within that department. User acceptance has been an ongoing problem for both DM and RM implementations, and remains a key issue despite the maturation of the technology solutions themselves. It is users, after all, who will make or break the integrity of any information management system by choosing to what level of detail they catalogue the information they create.

WorkCover SA, which administers occupational health, safety, rehabilitation and compensation for more than 700,000 South Australian workers and employers, found user involvement to be critical during its implementation of Open Text Livelink several years ago.

The organisation addressed the issue through extensive user surveys, in which more than 60 per cent of employees participated. Follow-up interviews with more than 120 employees sought to identify cultural barriers that might be an impediment throughout the project. That information proved to be instrumental in engendering user acceptance, and helped set the stage for ongoing expansion of the system. RM is currently handled through a separate system, although the organisation is considering whether to push for integration of the two disciplines.

“This is a long-term program that’s built into the way we work,” says Jürgen Hobbs, program manager for knowledge management with WorkCover SA. “It was clear from the outset that it was not just a technical project, and we gathered a whole bunch of information from people at the coalface about how they travelled around the organisation and how they used their network to share knowledge. Now we’re moving towards a more open, cross-functional structure and we’re trying to encourage a distinction between what is corporate and private e-mail. It’s a bit of a cultural shift.”

The fact that departments struggle to get users meaningfully involved in data organisation at all — much less to the extent required by rigorous archiving standards — illustrates the depth of the challenge that faces organisations keen to embrace RM across the board.

Current users might be used to having the DM application ask them for meta information when they save a document, but they may struggle to remain interested if they are forced into the rigid confines of RM — particularly if they are required to enter classifying information about every e-mail that they feel is interesting. Making the process too hard for users results in incomplete data, which impacts later usability of that data.

“If you’re searching for content on a subject, sometimes you have to be fairly original in your searching algorithms to find everything on the subject,” says Healey. “Rather than relying on people being fairly lateral in their searches, it’s obviously much better if we’ve got a file discipline running through this as well. We’re hoping that tighter integration [with e-mail], together with more training and awareness in our organisation, is going to improve the recording of our e-mails. And as far as I’m concerned, the more of it we can mandate or automate, the better.”