Contrary to what vendors tell you, computer systems alone are incapable of producing accurate forecasts

It's been more than two years since Nike Chairman Phil Knight owned up to the sports shoe giant's disastrous $US400 million experiment with demand forecasting software. The headlines are well known: Nike went live with its much-vaunted i2 system in June 2000, and nine months later, its executives acknowledged that they would be taking a major inventory write-off because the forecasts from the automated system had been so inaccurate. With that announcement in February 2001, Nike's stock value plummeted, along with its reputation as an innovative user of technology.

But what has since trickled out in court documents from shareholder lawsuits may be even more disturbing because it shines a harsh light on the inherent limitations of demand forecasting software. According to the documents, i2's supposedly state-of-the-art forecasting system couldn't communicate with Nike's existing systems, which impaired its ability to analyse large amounts of product information. At some point, the data even had to be entered in by hand, greatly increasing the chance for mistakes. And the forecasts themselves were way off. Relying exclusively on the automated projections, Nike ended up ordering $US90 million worth of shoes, such as the Air Garnett II, that turned out to be very poor sellers. The company also came up with an $US80 million to $US100 million shortfall on popular models, such as the Air Force One.

Nike isn't the only company with a forecasting horror story. Corporate America is littered with companies that invested heavily in demand software but have little or nothing to show for it. Goodyear, for example, implemented a demand forecasting system in mid-2000 but hasn't shown significant improvement in managing its inventory, and last year the tyre company lost more money than the year before.

Yet vendors and academics are still pushing forecasting software. In 2002 alone, companies spent $US19 billion on demand forecasting software and other supply chain solutions, according to IDC (a sister company to CIO's publisher). And in a speech in February, Stanford University supply chain guru Hau Lee extolled the virtues of harnessing software to extract customer knowledge in order to forecast demand.

Many CIOs, however, remain sceptical. Privately, members of Lee's audience complained to a reporter present that the ability to accurately forecast could hardly be taken for granted. And according to a recent Booz Allen Hamilton survey of 196 senior executives, 45 per cent said that supply chain technology in general had failed to meet their expectations. More than half - 56 per cent - blamed the shortcoming squarely on demand forecasting software. From hard experience, a growing number of CIOs now realise that computer systems alone are incapable of producing accurate forecasts.

There are a number of reasons why. To begin with, forecasting systems are only as good as the data put in them and, due to the complexity of modern supply chains - where a company wants to collect information about multiple products from multiple customers and suppliers - more often than not the data isn't accurate enough. Furthermore, software can't predict the future, particularly sudden, unexpected shifts in economic or market conditions. Nor can it exercise the kind of rational analysis or judgement that human beings excel at. Hence, demand forecasting technology is inherently limited, and companies such as Nike and Cisco that rely on it without an institutionalised set of human checks and balances will invariably end up in trouble.

"Demand forecasting sounds scientific," says Sumantra Sengupta, CIO for the Scotts Company, the world's largest supplier of consumer lawn and garden care products. "But I would say that if you looked at the split between people, science and process, people are half the equation. Algorithms are algorithms. That is not what will win the game for me."

Good demand forecasting requires a combination of accurate data and smart people. Up-to-date sales data and point-of-sale (POS) information will almost always improve a forecast. So will having the processes and people in place to make sense of anomalous results or simply to check computer-generated predictions against the pulse on the street.

"Anyone who thinks you can do it with just mathematics and statistics is only partly right," says Doug Richardson, CIO of electronic products maker Vicor. "Human intelligence is also required."