Integrating measurement of the complexity of software and performance of developers into the development lifecycle can have significant benefits in reducing risk, according to development metrics expert Ewa Wasylkowski.

A senior consultant with Australian company Total Metrics, she says systematic measurement helps realistic estimates of time, cost and effort at the beginning, and prevents unjustified scope creep.

Addressing a meeting of developers convened by local software company Equinox last month, she said metrics infused into the project will often facilitate "early observations about the likelihood of success," and enable abandonment or significant change of projects likely to fail, before too much money and effort are spent on them.

She cited several cautionary tales, including the New Zealand Police Incis project, which she says suffered badly from variations to the original agreement. She also talked about also three Australian projects, in road-tolling, ticketing and student management at the Royal Melbourne Institute of Technology, where insufficient attention was paid to risk management.

The Victorian government has introduced a strict metrics-based methodology with a role for a "scope manage." This person stays independent of the commissioning organization or the vendor, advising the project board and IT project director on the likely size and complexity of the project and potential difficulties.

Function-point analysis is at the core of the methodology, but once an initial size for the project in those terms is arrived at, she recommends giving it a "reality check" against the actual effort expended on similar projects. There are databases of such measurements, such as that of the International Software Benchmarking Standards Group (ISBSG), known as "icebag".

Realistic metrics for the project should be in place by the time an RFP is issued. Then, when proposals, and later tenders, come back, their structure and budget and time estimates can be measured against the ideal. The unduly expensive and the impossible optimists should be able to be rapidly weeded out.

Additional techniques are needed to match proposals involving packages and evaluate the effort required for customization. If all suppliers' figures are markedly higher than the customer's measurement, then the project should be re-checked against industry norms and perhaps an alternative strategy worked out.

Once into the build phase, every change request is vetted and sized by the scope manager. "You may decide you don't need those extra reports once you know they involve 30 function points and will cost NZ$2,000 (US$1,305) more,"says Wasylkowski. Perhaps their business benefit will be worth the cost but at least the cost is known.

If sizing has been carefully done from the beginning, there should be little argument in judging the wisdom of such scope creep. Proper metrics will guard against misguided measurements of completeness, she says. They will prevent facile measurements such as "80 percent of the modules have been delivered", when those modules may only incorporate 45 percent of the functionality.

Wasylkowski claims the technique has not been found substantially wrong on any project where it has been used to date but acknowledges experience with large projects is lacking. "It's currently being tried on three big projects, only one of which has reached the build stage."

But to extend the "build" metaphor, metrics are still only doing what a quantity surveyor does in a literal building project, she says.

"They don't replace workflow organization."