How a $US25 million payment system created a $US300 million backlog in six short months. A lesson in how not to run a Web services project.

Reader ROI

• The importance of having a contingency plan
• The consequences of skipping end-to-end testing
• 10 rules for successful project management

On Friday, January 21, 2005, the US state of Maine cut the ribbon on its new, Web-based Maine Medicaid Claims System for processing $US1.5 billion in annual Medicaid claims and payments. The new $US25 million program, which replaced the state's old Honeywell mainframe, was hailed as a more secure system that would clear claims faster, track costs better and give providers more accurate information on claims status.

But within days of turning on the new system, Craig Hitchings knew that something was seriously wrong.

There had been problems right from the start - an unusually high rate of rejected claims - but Hitchings, director of information technology for the state of Maine's Department of Human Services (DHS), had assumed they were caused by providers using the wrong codes on the new electronic claim forms. By the end of the month, he wasn't so sure. The department's Bureau of Medical Services, which runs the Medicaid program, was being deluged with hundreds of calls from doctors, dentists, hospitals, health clinics and nursing homes, angry because their claims were not being paid. The new system had placed most of the rejected claims in a "suspended" file for forms that contained errors.

Tens of thousands of claims representing millions of dollars were being left in limbo.

Hitchings's team - about 15 IT staffers and about 4 dozen employees from CNSI, the contractor hired to develop the system - were working 12-hour days, writing software fixes and performing adjustments so fast that Hitchings knew that key project management guidelines were beginning to fall by the wayside. And nothing seemed to help.

Day after day, the calls kept coming. The bureau's call centre was so backed up that many providers could not get through. And when they did, they had to wait on the phone for a half hour to speak to a human.

By the end of March, the number of Medicaid claims in the suspended bin had reached approximately 300,000, and the state was falling further and further behind in its ability to process them. With their bills unpaid, some of Maine's 262,000 Medicaid recipients were turned away from their doctors' offices, according to the Maine Medical Association. Several dentists and therapists were forced to close their doors, and some physicians had to take out loans to stay afloat. With the Medicaid program accounting for one-third of the entire state budget, Maine's finances were in shambles, threatening the state's financial stability and its credit rating. Yet Hitchings was at a loss to explain what was causing all the suspensions.

And every day brought hundreds more.

Today, more than a year later, it's fair to say that the Maine Medicaid Claims System project has been a disaster of major proportions. Since the new system went live, it has cost the state of Maine close to $US30 million. The fallout has been broad and deep. In December 2005, Jack Nicholas, the commissioner of the DHS who oversaw the project, resigned.

As of press time, Maine is the only state in the union not in compliance with the US Health Insurance Portability and Accountability Act of 1996 (HIPAA) - a striking irony given that the new system was designed to facilitate that compliance. Although US federal authorities have said they will work with the state in extending the deadline, the failure has been a black eye on Maine's ability to manage the health of hundreds of thousands of its residents. And it has become an issue in this year's race for governor.

State IT officials say they have fixed most of the bugs in the new Web services system and that it is now processing 85 percent of claims (although physician groups dispute this). With 20/20 hindsight, they can now look back and see where the project went wrong. Hiring a vendor, CNSI, that had no experience in developing Medicaid claims systems was the first mistake. And that was compounded by the decision to build a new and relatively unproven technology platform for the entire system rather than, as other US states have done, integrating a Web-based portal with back-end legacy systems. Thirdly, IT switched over to the new system overnight with no backup system in case something went wrong. And making matters worse, no end-to-end testing or training was conducted before the switch-over. Indeed, the story of the Maine Medicaid Claims System is a classic example of how not to develop, deploy and manage an advanced Web services system.

"By the first of March, it was clear that we were missing any sort of basic management of this project and were in complete defensive mode," recalls Dick Thompson, then head of procurement for the state of Maine and now its CIO.

"We could not see our way out of this."