Impressive Deception

Wishful thinking - and practices that lead project teams to "lie" in estimation tools like earned value management (EVM) - can lead projects into serious strife. RNC Global Projects managing director Diane Dromgold has seen plenty of projects fail or need resuscitating because the project manager was relying on misleading metrics and measures to assess progress.

In one company making a high-tech product, for example, all the metrics and measures looked great even as the project slid ever faster backwards. Dromgold says the project manager would get everyone together every week and they would record the update. No one noticed that while they were spending much time and regularly updating percentages of completion, there was no actual progression.

"In fact in one month the project slipped by the entire month and the metrics couldn't tell that," she says. "The moral of the story? There is no better harbourer of delusions than the estimates of 'percentages complete'. An entire team can rejoice as the project inches ever closer to 100, despite any evidence the actual work has progressed."

However, Dromgold's favourite story concerns EVM, thanks to which, one project manager proudly told her, his team at any given time knew exactly where they were a month previous. Now, that might not have been too much of a worry, she says, except that they based their assessment on the qualitative data of progress applied against the quantitative data of money spent. In one project where her review eventually triggered project cancellation, the project manager was able to show her "the most beautiful metrics", which showed progress perfectly matched the time and money spent.

The bottom line was a $6 million gap between money spent and what the project manager was reporting, and finally, devastation for the project manager and cancellation of the project.

"It's quite sad that we have allowed project management to become an accounting practice without the benefit of sound accounting principles or the ability to recognize and report on reality," she says.

Lie Like a Dog

It is so easy for project members to deceive themselves and others partly because seemingly watertight methodologies for software estimation and resultant metrics or measures are anything but.

Take earned value analysis (EVA). While the naive treat its readings as gospel, EVA is not a science, or even necessarily the most appropriate tool for every IT project. As research firm Gartner points out in a report entitled What Every Government IT Professional Should Know About Earned Value Management, effective use of EVA needs understanding, experience and commonsense application.

Applied effectively, EVA yields earlier and better visibility into program performance than non-integrated methods of planning and control. But Gartner finds a proven track record in large government capital projects (such as construction, weapon systems and aerospace systems development) has made EVA neither well known nor understood among government IT professionals, for instance.

Cutter Consortium senior consultant Steven Kursh also promotes EVA as enabling managers to track - and respond suitably to - metrics on cost and performance about the calendar, milestones and budgets. However, the simple fact is, earned value can lie like a dog. Kursh says the tool, like all tools, can be manipulated and used improperly. He says the other tools available as a substitute all suffer from the same weakness: "The difference is between what works in theory and what can be screwed up in execution."

"You can manipulate EV," agrees Manfred Thurow, CIO at Telstra's KAZ subsidiary. "I have seen a software development and integration project coded green the first week and green every week after that for the entire life of the project. Yet, the project was a failure. It was late; it blew the budget, and delivered nothing usable to the internal business customer."

Thurow says it is easy to fudge EV by:

• Rebaselining the project. The project manager waits until a scope change or other change request, and uses that as an excuse to redo the project schedule. The project instantly turns green because actual progress now matches expected progress.

• Pushing problem tasks forward. Putting the easiest tasks at the beginning of the project and the hardest tasks at the end can keep a project green for a long time.

• Padding the schedule.

• Bumping up the task completion percentages. For example, when measures say a task is 20 percent or 80 percent complete, who can objectively tell? Changing the completion percentage of any subjective task will help the earned value numbers.