Improving Practice and Product

Managers want to know the return on investment to be derived from any software process improvement actions they undertake. Technical practitioners are interested in achieving perfection in software practice and the products they create. The software inspection process gathers some of the data needed to determine each, O'Neill says. Managers are informed on the costs incurred and savings achieved - information that justifies the practice. Technical practitioners are informed on defect detection rates and types of defects detected - information that they can use to improve both practice and the product.

"Interestingly, organizations employing immature software product engineering - that is, ad hoc programming - and that use software inspections, generate a very large return on investment because they insert a large number of defects," O'Neill says.

"The large ROI, which may reach 20:1, is pleasing to managers; but the large number of defects is discouraging to technical practitioners. As software product engineering maturity improves to more structured software engineering, less defects are inserted to be detected and the software inspections' ROI falls to about 4:1. At this point management may become somewhat discouraged with the falling ROI and may consider quitting software inspections, but the technical practitioners are becoming somewhat encouraged as they see the defect problem responding to their improvement efforts."

What occurs next is even more interesting, O'Neill says. The organization that sticks with the inspection process and further improves its software product engineering to a true disciplined software engineering will see its software inspections' ROI fall even further to perhaps 1:1, a break-even situation. However, the technical practitioners now have a laser-beam like focus on product quality and are experiencing and tolerating very low defect rates. At this point every technical practitioner is riveted on achieving perfection.

Managers deal with conflicting stresses. Often the drive towards perfection clashes with time to market. The successful enterprise makes the strategic selection and accepts collateral damage, and the CIO may be required to discipline the situation for the benefit of the enterprise, he says.

"CIOs very much in partnership with their business have to decide what are going to be the drivers or the specific driver that they are going to focus on in terms of their IT delivery, and the impacts of these things are always going to be your cost, time and quality," IBM's Bowey says. "One of the problems is the perception that in putting in, if you like, such a rigorous and formal inspection process, that will provide you a quality aspect but detract both cost and time."

Convincing such people of the virtues of the inspection process can be difficult, he says, because you are moving on their perceptions - always a difficult thing to do. To address their concerns you need to explain to them at length that the earlier in the development process that defects are found and fixed, the greater the cost saving to the organization.

"If you think of this as a production line, the earlier you find a fault in the production line and fix it the less rework that you have to engineer in order to be able to produce the final product that you're after," Bowey says.

"We do these different kinds of reviews for system structure overviews, external interface reviews, inter-component interface reviews, design reviews; so you can see way before you get down to actually cutting the code we're doing inspections on work product way up the development life cycle. And the earlier you do these things the more money you potentially save yourself and your client.

"I guess one of the things that you need to ensure that you are doing when you do these work product reviews is to understand them as work product reviews and not use them as education sessions for the team. The process is all about identifying and removal of defects. The minute you start to split this into a tack at team meetings or an education session for the people involved, they start to lose the focus of why they're going to the meeting. And in terms of that, you may find yourself in a position where you're not identifying the defects as such," Bowey says.