In a 2005 article"Why Software Projects Fail", Cutter Consortium Fellow Robert Charette narrates an infamous anecdote about a disappearing warehouse.

A software glitch caused the warehouse to vanish, not from physical view, but from the watchful eyes of a well-known retailer's automated distribution system. Goods destined for the warehouse were rerouted elsewhere while goods at the warehouse languished, yet employees said nothing because the company was in financial trouble and had been shuttering other warehouses to save money.

"For three years, nothing arrived or left," Charette wrote. "Employees were still getting their paycheques, however, because a different computer system handled the payroll. When the software glitch finally came to light, the merchandise in the warehouse was sold off, and upper management told employees to say nothing about the episode."

Floating around the information technology industry for about 20 years, the story, apocryphal or otherwise, is entirely believable because episodes like it happen all the time, Charette says.

Charette likens software project failures to plane crashes: software developers no more aim to fail than pilots aim to crash, and organizations should be just as forensic in determining the cause of software failures as investigators are when figuring out the causes of a plane crash. " . . . We need to look at the business environment, technical management, project management and organizational culture to get to the roots of software failures," he says.

Entire forests must have died as pundits laboured to nail the root causes of software failure. Explanations abound, and the temptation to rely on formulaic explanations and even more formulaic remedies is clearly very strong. But life is never so simple. In fact most failures, as Charette points out, grow out of a combination of technical, project management and business decisions, each interacting in complicated ways that exacerbate project risks and make failure more likely.

In a paper in the July issue of Communications of the ACM (the journal of the Association for Computing Machinery), David Avison, Shirley Gregor and David Wilson define a category of such failings as "managerial unconsciousness". The authors analyze three Australian projects of the past six years that proved either catastrophic or near catastrophic: Sydney Water's CRM and billing system, the Royal Melbourne Institute of Technology's (RMIT's) attempt to implement an academic management system using the PeopleSoft ERP, and the failure of former telco One.Tel's billing system, which they characterize as "largely responsible for the company's downfall".

They found three common themes emerging: (unnecessary) complexity of application system software, poor IT governance, and relatively inexperienced and/or powerless IT staff lacking clout among corporate decision makers.

"These cases suggest that where software applications are large and complex, experienced IT and IS staff are needed, and there must be tight governance of the project, including good project management. What seems to have happened is that governance was neglected or even withdrawn under the mistaken belief that 'IT doesn't matter'," the authors say.

"All such issues are the direct responsibility of senior management; only it could have changed these aspects of the projects for the better from the start, rather than, in the case of RMIT and Sydney Water, after crises were evident and an external audit had been conducted to identify problems and deficiencies. In the case of One.Tel, such an audit was never carried out, and the company went under."

Overall the authors found managerial IT unconsciousness was rife, characterized by lack of awareness of the importance of the IT projects being undertaken and a reluctance to tackle complex matters and ask tough questions. That little timely action was taken when things went wrong was more proof of the lax attitude, they say. They conclude management may be seduced by the abstract nature of software, the ubiquity of PCs on every desktop, and the availability of generic applications (such as word processing and spreadsheets) into thinking IT doesn't matter. However, the experiences of RMIT, Sydney Water and One.Tel show that such management thinking can have disastrous consequences.

"Software is flexible, and IS specialists can develop systems to support almost any business application. But [these systems] are complex and need rigorous design, careful construction and exhaustive testing to ensure they actually do what they are intended to do. Management must understand, track, review and control their progress, particularly their impact on the rest of the organization," the authors say.

"Not doing so is an abrogation of management's responsibility. Project success can be achieved only by applying proper and prudent management controls to the development of these systems. They certainly do matter, and senior management cannot afford to be unconscious of them," they say.