Scrutinising every aspect of a finished project and acting on the lessons learned can be controversial and difficult. But post-implementation audits are an important way to avoid repeating costly mistakes.

Reader ROI

• Why post-implementation audits are valuable
• Learn from companies doing it right
• Overcome resistance to PIAs in your organisation

As soon as Michael Baker Corporation's IT department installing a Web-based procurement system from ePlus, Bruce Higgins, CIO of the $US405 million engineering and construction company, was bombarded with enquiries from colleagues about the system's effectiveness. He had ample anecdotal evidence suggesting that orders were getting turned around more quickly, but he didn't have tangible proof that the system was improving efficiency.

So he decided to conduct his first ever post-implementation audit (PIA) of the new system. A PIA is a top-to-bottom evaluation of the hard and soft benefits derived from a strategic information system, the security of that system and the project management process for deploying it. From the PIA, Higgins learned that because IT miscalculated the number of people needing to use the new system, the ROI was driven down by the cost of ordering additional licences. The PIA also showed that the system was saving the company more than $US150,000 yearly, however, so Higgins was able to prove the system's worth to his colleagues. And he learned some valuable lessons on what not to do on subsequent projects.

CIOs would do well to follow Higgins's lead in realising that a PIA is a worthwhile way to prove the value of IT. But many don't. In fact, Barbara Gomolski, a research director with Gartner, estimates that a mere 20 per cent of companies take the time to conduct PIAs.

Companies avoid post-implementation audits for many reasons: They take too much time and drain away valuable personnel resources - two things currently in short supply. They require reams of documentation so that processes and results can be validated. Finally, project sponsors and implementers fear that the results of an audit, if unfavourable, will be used against them.

The CIOs at companies successfully performing audits have identified critical success factors, including getting the right people involved, timing the audit properly, and collecting enough documentation to facilitate the smooth execution of a PIA. They pick the right projects to audit (for more information on this, see "How to Select the Right Project for Your First Post-Implementation Audit", page 108). And these CIOs share several key traits on how they ensure that PIAs become a sustained practice in their organisations: They are all committed to continuous improvement, they've made PIAs a part of their project management methodologies, and they have their CEOs' support.

But companies not performing PIAs are missing out on the important benefits such data provides. PIAs provide a thorough approach for proving the value of high-cost, mission-critical IT investments and for gleaning project management best practices, which CIOs can then apply to keep subsequent projects on track. At a time when the value of CIOs, IT departments and IT investments are under increased scrutiny, PIAs are now more critical to CIOs' success and survival than ever before. "With the pressure on business today and the responsibility of IT to help the business units understand where their dollars are being spent, I really have trouble with anyone saying you shouldn't be doing [PIAs] in this [hard economic] time," says Jim Smith, CIO of Sun Life Financial and a strong advocate of PIAs.