CIOs know that project implementation success rates are woefully low. So once a project comes in on time and under budget, CIOs think they've won the battle and can move on. Wrong. Here are some strategies for wresting value from systems long after they've gone live.

READER ROI

• Why CIOs need to monitor the value of projects long after they are completed
• How failing to do so can mean abandoning value vital to the enterprise
• What tools and tactics can help continuously wring value from completed projects

Caring for diabetics seems to be a natural place for IT to add value for a hospital. Diabetics need regular cardiovascular checkups and cholesterol tests. Annual eye exams are a must, as eye disease is a sign that the diabetes may be progressing. All these factors lend themselves to systems tracking.

So when Partners HealthCare System (which runs 10 hospitals in eastern Massachusetts, including two large academic medical centres) found that patients weren't routinely getting their eye exams, CIO John Glaser authorized an update to the electronic medical records system, adding a reminder feature that would pop up and tell doctors and nurses to make sure their diabetes patients got eye exams. He figured that should take care of the problem, and IT could rack up some value kudos.

So did some of the hospitals' doctors, who received a grant from the National Institutes of Health to research and write what they thought would be a glowing review of the impact the system had on diabetes care. But then they found that the number of patients getting annual eye exams went from 10 percent to only 17 percent.

"Which meant we sucked," Glaser says bluntly. So he dedicated new resources to finding out why the system wasn't generating the expected results.

It took almost two years of refining to get to the desired level of usage. Discussions with doctors and nurses revealed that the alerts weren't working because the hospital billing process rewarded them for seeing more patients, not for stopping to take care of alerts from a system. Ultimately what drove up the usage was an IT change spawned by a reworking of the medical centre's insurance rules that changed the way insurers paid doctors for treating diabetics.

The irony for Glaser is that the system update had such a clear-cut, perceived value that IT hadn't intended to follow up on the project. He saw the experience as a wake-up call for IT to redouble its efforts to make sure it was getting value out of existing implementations as well as new ones. So Glaser now does several things after a project is implemented: He looks for clearly defined project goals from the business side; makes sure someone, whether on the business side or the IT side, is accountable for key projects; and insists that those project goals are discussed, during and after implementation. He also pursues post-implementation audits on key projects.

Even so, he says, "We're uneven - and I think most places are - at following up to see if things are working."

Glaser's experience is not unique. It's well known that fewer than 20 percent of projects are successfully completed. What's less known is just how much value gets left on the table, even for successful projects. There are plenty of processes and tools to measure post-implementation value, and if CIOs don't use them, they're basically guaranteeing that they're not getting what they should from their IT investments.