Analysts estimate that businesses end up spending billions for software that doesn't do what it's supposed to. Some CIOs are tired of playing the sap and are beginning to take action.

Greg Seyk, newly appointed CIO of VisionQuest, had only months to rid his organisation of its Y2K bugs. (Y2K, after all, was not a top priority for the Arizona-based national youth services organisation, whose mission is to keep kids out of jail, and Seyk had just come on board the previous spring.) The first thing Seyk did was purchase a Y2K-compliant ERP system - Lawson Insight version 7.1.5 - from Minnesota-based Lawson Software. But just as he and his staff of five got around to testing the software, Lawson released a new version (Lawson Insight version 7.1.6) that included a function for prioritising bill payments. That function had been promised but never delivered in any of the previous versions, Seyk says.

With just two months to go before the clock struck midnight on December 31, 1999, Seyk didn't have time to deploy the upgrade, even though the payment prioritisation function had been a critical selling point for the 53-year-old CIO. "We had to implement accounts payable, the general ledger, payroll and human resources to make sure they were Y2K compliant. It was no small feat," says Seyk, who is also a vice president of the private company.

Between November 1999 and July 2001, Lawson released seven new versions of its software to fix bugs or add functionality that had been promised but absent in each previous version. Seyk was outraged. He documented his problems in a series of letters to Lawson executives, met with them on two occasions, and sank a total of $US594,974 into software and maintenance to correct the flaws in their products.

And then it dawned on Seyk why the software and support were so bad: that's the way vendors make money. They push products on the market before they've been adequately tested, demand payment up front and then are often not available to deal with the sequelae of poorly performing products. (Lawson officials in the US declined to comment specifically on VisionQuest's problems with its software. All a Lawson spokesperson would say is that the company is working with VisionQuest in an effort to resolve its concerns. "We are committed to 100 per cent customer satisfaction," says Bev Bergstrom, vice president of communications for Lawson.) How Bad Software Pays Dividends CIOs have been complaining about poorly designed and buggy software forever. In a recent survey on CIO's US Web site, almost half of the 88 IT professionals questioned said they were unsatisfied with both the quality of their business software and the support. The problem is a big one: faulty software costs businesses $US78 billion per year, according to Jim Johnson, chairman of The Standish Group, a Massachusetts-based research company.

But now many CIOs are beginning to realise that the origin of the problem may lie in the economics of the industry. Vendors generate most of their revenues through perpetual licensing agreements, which force CIOs to pay up front for an application. In return, CIOs own the software and the right to use it "in perpetuity". The problem with this model is that in reality, CIOs are lucky if they can get three years out of a product before vendors release entirely new versions of their software. Vendors further pressure CIOs to buy those new releases by threatening to stop supporting previous releases - a tactic they often take both to cut their tech support costs and to get CIOs to pay again and again for what is essentially the same product.

Another problem with the perpetual model is that CIOs have to fork over an additional 15 per cent to 20 per cent of what they paid for the software in annual maintenance fees to cover product updates and tech support, according to Chuck Phillips, managing director and software industry analyst at Morgan Stanley Dean Witter. If CIOs want to receive upgrades, patches and access to tech support - as inadequate as it can sometimes be - they have to pay the yearly maintenance fee. Software companies earn a significant amount of cash from these fees. So it's in the manufacturer's best interest, at least financially, to make products that need maintenance and that have to be continually improved with successive updates, patches and versions that CIOs pay for up front. In sum, bad software works for the vendors.

There are, of course, other reasons for all the bugs. IT professionals point to a whole litany of causes: bloatware, with all its useless bells and whistles; programmers working in isolation, blissfully ignorant of how people will ultimately be using their software on a daily basis; reusable components that may already contain bugs; an absence of agreed upon professional standards; and developers who take shortcuts to meet deadlines during development.

But a large part of the story may indeed be the way vendors sell software. CIOs are finally waking up to this, and a growing number are demanding that vendors change their business models. In the US a council of IT leaders from a dozen heavy-hitter enterprises convened in August under the auspices of Boston-based analyst company AMR Research, intent on pushing for software industry reform. The group issued a peaceful statement of its desire to "work with" software companies for improvements in quality, delivery reliability and versioning. However, with big names like Becton Dickinson, Boeing, Cabot, General Dynamics and Kraft on the roster, the council has enough weight to change "work with" to "lean on".

Some other IT users have resorted to more extreme measures - such as withholding payments to put pressure on vendors - but in the US new legislation may soon make it harder for CIOs to employ such brute financial tactics. The uniform computer information transactions act (UCITA) makes it harder for customers to sue vendors and allows vendors to more easily change contract terms. The UCITA has already been passed in Virginia and Maryland and is under consideration in seven other states and the District of Columbia.

Fortunately, there are a host of alternative solutions on the horizon, and a growing number of CIOs are determined to make them a reality. They include renewable licensing agreements, in which CIOs purchase the right to use software for two to three years at about 85 per cent of the cost of what they'd pay under a perpetual licence. CIOs then have the option to renew the licence at the end of the term if they're happy with the quality of the product and the support. Subscription licensing agreements are similar to renewable licences, except the term is shorter, lasting about a year, and CIOs rent the software, as opposed to owning it.

Finally, some CIOs are opting to circumnavigate packaged software wherever possible. They're turning to open-source technologies such as the GNU and Linux operating systems, the Apache Web server and Sendmail e-mail. "People are not involved with [the open-source movement] for profit; they're involved with it because they want to write good product," says Bill Lessard, co-author of NetSlaves: True Tales of Working the Web and a former developer for Prodigy and AOL Time Warner. "If software makers see they are losing money to people going the open-source route, then they will change. Until then, it will be business as usual despite appearances."

As much as eight years ago, Patricia Wallington, president of CIO Associates and former CIO of Xerox, was envisioning a new method of buying software. "I wanted it to be like a lending library where you could find modules on the Web, buy the ones you were interested in, cobble them together and create your own software," she says. "We need to rethink the way we deliver software because it is so intransigent."

Withholding Payment by Brute Force

The economics underlying the software industry - its emphasis on quarterly earnings to impress investors - leads to the pursuit of short-term profits, often at the expense of long-term gains. And this tendency has only been exacerbated by the current market downturn. The revenue of software vendors is predicated on acquiring new customers. That initial sale provides software vendors with their biggest profit. So there is a built-in incentive for vendors to rush a new release of software out the door before it is completely tested and debugged.

Bill Crowell, CIO of Meredith Corporation, the $US1.1 billion publisher of Better Homes and Gardens, believes it is just this profit motivation that has caused many of the troubles he and other CIOs have had while implementing Oracle 11i. He had purchased Oracle 10.7SC, a client/server-based financial system that handled accounts payable, the general ledger and purchasing functions from the vendor in the US spring of 1999, with the assumption that it would be good for at least four years. But in the US autumn of 2000, Oracle released a new Web-based version, Oracle 11i, and told its customers, including Crowell, that it would be dropping support for all previous releases. Crowell had no choice but to upgrade. (Under pressure from customers, Oracle has repeatedly rolled back the end-support date of these earlier versions. Recently Oracle announced support for Oracle 10.7 applications until June 2003, a six-month extension from the previously declared deadline of December 2002.) Oracle also promised that 11i would include a feature that would automatically enter electronic records of all the purchases Meredith employees had made using their corporate credit cards into the accounts payable or general ledger system either monthly, weekly or daily depending on how the company configured it. Crowell says that when he purchased 11i, the promised functionality was absent. "It wasn't until about a year later when 11i actually had that capability," he says. It was an inauspicious beginning.

As soon as he began the upgrade, Crowell found bugs running rampant in the software, like ants scuttling over a piece of fruit. Files were corrupted. Data was lost. Processes didn't work. Screens froze. "It was just a nightmare," says Crowell. "We were getting literally dozens of developer patches to this software. Then we were getting patches for patches. The quality was just atrocious."

One of the biggest bugs bit the interfaces between application components in the financial system. The system didn't transfer data between accounts payable and the general ledger, between purchasing and accounts payable and between purchasing and fixed assets, Crowell says. The failures were bad enough that had Crowell and his team not been running 11i in a test environment, Meredith would have had to shut down its financial system. It would not have been able to do its accounting or pay its bills until the problems were solved. (Oracle officials declined to comment on either the bugs in 11i or on Meredith's specific problems with the software.) "It was clear [Oracle] never tested the interfaces because they flat-out failed the first time. We felt that what was [supposed to be] their general release software was effectively beta," says Crowell. "There's no question that they were under pressure from management to be first with a Web-enabled version of their software."

The 11i implementation was supposed to go live by April 15, but the bugs delayed the implementation by just one month, but only because Crowell's staff worked 24/7 for four months. The CIO estimates that the bugs cost his company more than $US100,000; he had to pay for contractors to help with the nine-month implementation, and he wasn't able to put staff on other pressing projects.

So how did he muddle through this debacle with Oracle? "We didn't pay them, for one. We owed maintenance of $US300,000 to $400,000, and we just didn't pay it. We said: ‘We're holding on to it until you get this thing up and running'," Crowell says.

But he thinks it wasn't so much the money that got Oracle to fix the bugs in 11i as it was the brute force he and his project leaders applied in dealing with the vendor. They called Oracle daily to see if the company was making headway resolving their problems. They also forced Oracle to give them contacts in the development group so that they could ask developers directly for help rather than going through the support team.

Crowell blames economic forces for the problems with Oracle 11i. "They're trying to move so fast to get the product into the marketplace that they're not adequately testing and debugging their software," he says. If Oracle had waited six months before releasing 11i and taken that time to test the application, he says, the upgrade would have gone off without a hitch. "Overall we're very pleased with the new application, but if Oracle thinks they're the Lexus of the software industry, [after] what they've done to their customers, you feel you've bought a Dodge De Soto," Crowell concludes.

Renewable Subscriptions-Use Now, Pay Later Crowell believes that the new renewable and subscription arrangements that are becoming more prevalent in the software world would have ameliorated the problems he ran into while deploying 11i. Under a subscription model, in which he would have paid less up front, Crowell would have had more leverage. Also, it would have given Oracle a greater financial incentive to please Crowell. In fact, Crowell plans to start buying software from Microsoft on a subscription basis in two years, once he finishes receiving all the upgrades he paid for two years ago. To him, the renewable model makes sense. It's the way his own publishing industry works. "It's a subscription. We know when the revenue is coming in. We can plan our business around it," he says. "And we deliver a quality product every month. The [vendors] need to think about delivering quality every month and a business model that allows them to do that."

Microsoft, in fact, announced that it would begin offering a brand-new subscription licence last month for its operating systems and software, including Microsoft Windows Professional and Microsoft Office Professional. The Enterprise Agreement Subscription, as the new licence is called, is a major departure from the perpetual model. CIOs will now lease the software under subscription licences. While CIOs see the potential benefits of the subscription model, many are uncomfortable with the specific terms Microsoft is offering. For instance, Microsoft is requiring that customers pay a hefty annual fee even before new upgrades are released.

Microsoft has also introduced a new and more controversial twist to its perpetual Open and Select licensing agreements. These programs pressure CIOs to upgrade to new versions such as Windows XP by February 1, 2002, at a discounted rate. If they don't, they will have to pay twice as much to upgrade after that.

Mark Grove, CIO of AmericasDoctor, a pharmaceutical services company based in Chicago, says software vendors that require perpetual licences with constant upgrades are not serving their customers. "Any vendor who's doing that is trying to force the customer to follow the vendor's business model," Grove says. "When they tell a customer that he has to upgrade at a certain time, they're forgetting that their customers have their own business cycles and busy seasons that they have to work within."

After his experience with Lawson, VisionQuest's Seyk is also considering buying software from vendors on a renewable or subscription basis when it is offered. He likes the idea of not paying for the software entirely up front. "Once you give [vendors] the cash and the software doesn't perform, you have no leverage," he says. "If [CIOs] could say: ‘Sure, you'll get 10 per cent now and 10 per cent after each quarter, and a year from now you'll get it all if [the software] works', that would be a way to hold the vendor accountable. It gives [the vendor] a financial incentive to make sure the product works."