Y2K and the e-commerce gold rush made ROI analysis little more than an empty formality at most companies. Now ROI is back.

We give you seven tips to see if your method measures up.

Steve Morelli hates most ROI analysis he sees. "Almost invariably, it's simply made up numbers to justify a project,"says the CFO of Star-Kist Foods. Morelli's blood boiled a few years ago when Star-Kist's procurement group wanted to automate its purchasing methods with a new software package. Consultants had sold the package on a research report that said companies could save $US80 on each purchasing transaction by automating it with software. Considering that Star-Kist has hundreds of different procurement transactions, that sounded like a pretty good ROI. So with the help of the consultants, the procurement people documented all those different purchasing transactions and multiplied each one by $80 to get the ROI for the project. That was their first mistake. Then they sent the report to Morelli. That was their second - and last - mistake. Morelli still gets pissed off when he thinks about it. "There's a lot of false precision created by doing lots of work and gathering lots of detail when the fundamental assumptions are flawed,"he says. "Who knows where the consultants got that $80 from. All I know is that those weren't our numbers. No one had figured out what it cost us to do procurement."

Garbage In, Garbage Out

Great-looking ROI reports based on bogus benefit predictions often slip through the approval process at big companies. The numbers are so wonderfully impressive, or so difficult to understand, that no one bothers to question them. Companies launch big software projects such as enterprise resource planning (ERP) and customer relationship management (CRM) - which can easily cost $50 million apiece at a large company - with a completely false sense of whether the project will pay off.

In the boom times of the late 90s, while Y2K troubles and the Internet added a frenzied shove behind every software project proposal, the flaws in traditional ROI analysis didn't show. But now that tough times are here, ROI is back. And it's clear the process is broken.

The break shows most clearly at the beginning - the benefits investigation phase. Predicted benefits - such as computer hardware and maintenance savings, productivity increases and head count reductions - are everything in ROI. Plug bad benefit numbers into any respected ROI maths formulas such as Net Present Value, Internal Rate of Return and Payback Period , and your proposal will look impressive, but the outcome will be garbage.

And as Morelli's procurement people discovered, predicting benefits for new software projects has got more difficult. Software projects cover larger and larger pieces of the business, making benefits harder to quantify and track. Installation complexities, user rebellions, maintenance costs and consulting fees swallow up the "easy"benefits you might expect from a new system, such as retiring an old mainframe or laying off those Cobol programmers who kept the old system running.

To justify software projects these days, software vendors and consultants are selling the productivity benefits of their software. Big software packages such as ERP and CRM are intended to change the ways people work, to make them more efficient and productive. That poses a big challenge to those charged with figuring ROI benefits on these projects. For Morelli's procurement people to figure out the true ROI of an automated procurement process, they would have had to spend many hours interviewing people about how they do their work, develop a new automated process for doing it, and subtract the difference in cost. For there to be a real ROI number to sell, they'd have to make some hard decisions along the way, such as head count reductions, department reorganisations and personnel reassignments. That's a lot of work to do before you even start a project.

As if the work wasn't difficult enough, most finance people still dismiss these productivity and process benefits as "soft"and inconsequential to the bottom line. But companies are going to have to start believing in - and doing - this hard work of determining productivity benefits, because productivity is becoming the most important piece of ROI analysis. In a recent, exclusive CXO (US) survey of 75 CEOs from a broad range of companies, increased productivity was cited as by far the most important factor (87 per cent, far ahead of the next highest factor, customer satisfaction) they use to determine if IT is delivering full value for the money spent.

Yet more focus on productivity analysis - how you do the work now and how you will do it in the future - during ROI analysis will help prevent software projects from becoming delay-ridden installation nightmares. The big failure of ERP and CRM projects these days is that companies begin the project before figuring out how they want the company and its work methods to change. That leads to costly delays. Better to have the delays up front during ROI analysis than during the course of the project, when you've already committed a boatload of money to its success and it's too late to turn back.

To get a handle on this complex ROI benefits work and keep costs for doing it from spinning out of control, companies need to develop a methodology that they use consistently across the company. Today most companies - as high as 85 per cent in a recent survey by Meta Group - claim they have a process for performing ROI analysis. But only 12 per cent of those surveyed said they actually calculate ROI for all their software projects across the company. Even among those that calculate ROI for every project, few have developed a consistent method, making it virtually impossible for them to compare proposals or track benefits during and after the projects, to really see if they got the ROI they predicted, according to Howard Rubin, executive vice president and research fellow at Meta Group (US).

But better ROI analysis doesn't just involve process, it requires more involvement and accountability from businesspeople. Right now, the process is ghettoised within IT, which is least qualified to predict how changes in the ways people work will translate to the bottom line. Instead of divorcing themselves from any responsibility once the project gets under way, businesspeople need to be involved up front during ROI benefits analysis and be willingly accountable along with IT for achieving the predicted benefits. That teamwork could save you millions down the road when the software is installed and it comes time to make the hard business decisions about cutting head count and reassigning people.

Here are some tips on how to create a new, repeatable, accurate ROI process from the perspectives of those who have done it.

1Create a list of strategic business metrics that can be applied to the ROI process for all software projects.

What are the most important strategic benefits that the business likes to see for its investment dollars? If you come up with a short list, such as this one from chip maker Intel, it will give crucial guidance to ROI analysis efforts around the company, according to Gary Anderson, Intel's director of sales and marketing applications.

Improved operational efficiencies Will the application reduce head count, increase productivity, or reduce the number of applications, hardware and support staff in IT?

Increased customer satisfaction Software that reduces order cycle time for Intel's customers scores high in ROI analysis, because Intel believes quick turnaround time makes happier customers.

Improved market competitiveness Does this application help customers design Intel chips into their products more efficiently, for example, thereby making them more likely to use Intel chips?

2 Bring IT and finance together to jointly develop a single ROI methodology for the company and then keep finance involved.

For years, Sean Magee's discussions with his finance superiors about IT projects were more like confrontations. Finance didn't just challenge his numbers, it battled every facet of his methodology for arriving at those numbers. It occurred to Magee, vice president of information services for copy machine manufacturer Lanier International, that if he could fight the methodology battle only once, it would make more productive use of everyone's time. So he paired one of his best IT project managers, the director of project management and quality, with the director of financial planning to create a standard method for calculating ROI. "If you develop the methodology in finance's terms, you'll have them pulling IT projects through the ROI keyhole rather than you having to push them through all the time,"says Magee.

Owens-Corning, a global building materials and advanced glass composite manufacturer, goes a step further. Each IT project has a finance person assigned to track the progress of benefits during and after the project. "IT has to disown - or at least jointly own - the ROI numbers with the business,"says David Johns, Owens-Corning's senior vice president and chief supply chain and information technology officer. "It gives IT more credibility when the controller organisation says we're going to save $US30 million on this project rather than me saying it."

3 Make sure you have ROI benefits that are auditable.

If you are going to plug a benefit into your ROI analysis that says you can reduce head count by 20, make sure you can audit the benefit for results. That means understanding how long it takes people to do their jobs now - before you begin the project. It also means building a 20-person head count reduction into the operating budget of the e-procurement group and making the successful cuts a part of the bonus rewards for managers and employees in the unit.

4 "Soft" benefits matter, but discount them heavily.

"I've yet to meet a CFO who will write down soft benefits - improved customer satisfaction, increased worker productivity and improved market competitiveness, for example - and use them in an ROI calculation,"says Jay Pieper, vice president for corporate development and treasury affairs at Partners HealthCare System in Boston. "They're just too hard to account for in financial terms."But that doesn't mean they aren't there.

"It's foolish to ignore [soft benefits],"cautions Ian Campbell, vice president of research for Nucleus Research, a Massachusetts-based research company. "If everyone discounted productivity gains, we wouldn't have PCs on our desks. They don't have positive ROI without the productivity gains. And they are too costly to manage relative to the hard savings they provide."

Michael Head, executive vice president of human resources for Alabama-based Regions Bank, thinks he's found a good compromise. He includes soft benefits in his ROI analyses, but he separates them from the hard benefits he expects the project to achieve. In addition to that, he discounts the soft benefits by at least 50 per cent.

5 Factor in the productivity discount.

If you save a manager all kinds of time with new software but he uses that extra time to cruise the Web, what have you gained? Not much. Employee productivity ROI depends on how structured the employee's time is. For example, if an assembly-line worker saves 10 minutes with new software, you can increase the target output of the assembly line to reflect the time savings - a hard benefit that goes right to the bottom line. Similarly, if new software reduces the time salespeople will spend doing paperwork, you can increase their sales quotas to reflect the time saved.

But if your productivity ROI depends on white-collar types, watch out, says Campbell of Nucleus Research. Nucleus discounts their productivity savings from software by up to 80 per cent. For factory workers and salespeople, the discount may only be 10 per cent or 20 per cent.

6 Separate software proposals into those that have the potential for ROI and those that are simply the cost of doing business.

Some IT projects are like fixing holes in the factory roof - you have to do them regardless of payback, says Star-Kist's Morelli.

"Sometimes you have to make a business decision to do a software project because you believe it's the right thing to do for the company,"he says. But all too often, CEOs and CFOs won't approve any IT projects that can't show positive ROI. That pushes IT to trump up the numbers on borderline projects or, worse, avoid proposing low-ROI-but-necessary IT investments such as upgrading the corporate network.

7 Make sure the ROI benefits are staggered.

One last rule of thumb: if all your proposed ROI benefits don't hit until the third year of the project, it's probably not worth doing. "Lots of things can happen in three years,"says Pieper. "If you don't have some benefits kicking in sooner, the ROI could turn to mud."Staggering the benefits also gives you some convenient check-in points to evaluate progress of the project. "Even if the benefit is a little lightweight, have something happen in 90 days,"says Pieper. "It doesn't even need to be hard dollar returns. If it's only that you want a certain number of people using the system - however incomplete it may be - within 90 days, that's OK. If you can't get anyone using it, that becomes a quantifiable milestone that may cause you to reconsider what you're doing."

In these tough times, responsible business and IT people are all repeating a responsible mantra: ROI. Companies are more careful about buying and installing new software than they have been for 10 years. During that time, software projects have changed completely - becoming big enough to matter to quarterly earnings and be splashed all over the business pages when they go sour. Yet the ROI analysis process has not changed at all during that time. Indeed, it has withered into something more appropriate for buying new trucks or leasing new factory space than for making complex decisions about technology.

One thing is clear: if you don't consider blowing up the process you have for technology ROI today to make it more comprehensive and accurate, you risk having to explain why you didn't in a quarterly earnings report somewhere down the line.

ROI Classic

What is the Achilles' heel of all the tried-and-true formulas for determining whether a software project is worth doing? All are nearly meaningless if you've done a bad job of predicting the benefit numbers that you plug into them+ Internal Rate of Return (IRR) The equivalent of the interest rate you could expect to receive if you took the software project money and invested it in something else, like a Treasury Bill, for example. Some companies like to figure their IRR minimum based on their own cost of capital and the minimum percentage return they'd like to see from the investments they make. If you're going to take on the risk and trouble of doing a technology project, for example, you may want to set your IRR relatively high - at 15 per cent, maybe. This becomes the discount rate that you plug into your NPV calculations.

+ Net Present Value (NPV) The most-favoured formula among finance people, NPV is valuable for ranking one project against another. The main problem with NPV is that it doesn't tell you when you will see benefits. For that you need to calculate payback time. NPV's main components are the discount rate (DR), which is the interest rate you decide you could get if you invested the project budget elsewhere (since you really want IT to prove its mettle you decide to peg it at 15 per cent - see "Internal Rate of Return", above); benefits for each year (B1, B2 and so on); and project costs (PC). Let's say your software project will cost $15,000 and that you expect to resell it to another company at the end of five years for $5000. You expect the software to net $3500 in benefits per year. The NPV of the project is $1351. Generally, anything above zero is worth doing. Do the same calculation for other potential projects and prioritise them according to their NPV. Here's the maths:

NPV = PC + B1 / (1 + DR) + B2 / (1 + DR)2 + B3 / (1 + DR)3 + B4 / (1 + DR)4 + B5 / (1 + DR)5 + Payback Period A critical calculation in these times of fast-changing technologies, payback period is simply the point at which you expect the yearly benefits of the project to cover the costs (see "NPV" above).

+ Total Cost of Ownership (TCO) The cost component of the NPV equation, the total cost over time of buying, installing and maintaining software. TCO is good to know, but when used in isolation it can drive you to seek the lowest cost solution.

SOURCE: KNOWLEDGE EXCHANGE BUSINESS ENCYCLOPEDIA.

Risks to ROI

Back in the days when software projects were small and limited to specific areas of the company, the effect of a bad ROI process would have been negligible. But now that software projects can cost hundreds of millions of dollars and involve every facet of a company's operations, the consequences of faulty ROI calculations can be catastrophic.

New technology The lean, mean Internet has made application obsolescence occur faster than ever. Beware of technology projects that don't offer ROI for a long time.

Mergers and acquisitions If you tend to grow through M&A, don't approve projects with long payback periods. You'll have to start over before you finish.

Underestimating training costs Nothing ruins ROI faster than employees who don't use the application as it was intended because they don't understand it. Be sure your implementation team doesn't take all your application knowledge with it when it leaves at the end of a project. You need "superusers"who helped install the application and can train others to use it.

Unfinished business processes Leaving business process redesign for the implementation phase instead of hashing it out during the ROI phase will hurt expected productivity returns.

Bad cultural fit If you have a tightly controlled bureaucratic organisation that doesn't share information willingly, don't expect your employees to give you ROI on collaboration software like Lotus Notes. Similarly, a global organisation whose applications speak only in English and US dollars will have a lower ROI.

Employee benefit What about me? Software usage rises in direct proportion to its ability to give the employee an advantage in his or her career. If there is no personal benefit component to the new software, employees won't want to use it, reducing ROI.

Valuing the Soft Stuff

When calculating ROI for a PeopleSoft ERP project for his HR department, for example, Michael Head, executive vice president of human resources for Alabama-based Regions Bank, worked with the people in payroll to determine how many hours they spent manually inputting information into Regions Bank's legacy payroll system - information that would live in databases and be automatically processed with the new PeopleSoft system. He calculated that the time savings equalled 15 full-time equivalents. But he didn't put the savings in the hard benefit column because those 15 jobs were only slices of people - the real people would not be eliminated, but they would have more time to do other things because of the new system. Head took the dollar savings that he estimated would result from 15 fewer people - at least $US7 million over five years - and halved it to $US3.5 million. Meanwhile, he put the 20 people that he was certain could be reassigned as a result of the new system under the hard benefits at $US7 million.

"You can't ignore the value of freeing up people's time to do other things,"says Head. "You just can't value it as much as you would by cutting staff, and you can't be as definitive about how much you'll actually save."That's why Head tries to make sure that he has enough hard cost benefits on a project to justify doing it without stirring in the soft benefits - they simply become a kind of soft icing on the ROI cake.