Software development at ANZ's International Systems is win-win-win for everyone: higher productivity, better morale and more pride in the product.

There are times of late when the atmosphere in the war room in ANZ's International Systems area rivals that at an Academy Awards ceremony. The ambience is tense but friendly. The excitement is palpable. There's a buzz in the air as people gather in small groups, anticipating the latest results. But at the ANZ, unlike the Academy Awards, everyone tends to be a winner.

That's because over the last 18 months the area has achieved a massive improvement in both software quality and productivity. It's also down to something like one-tenth of the average rate of defects for the industry, and running at around about 10 times better than the industry.

"It's really exciting when a new project is delivered," says head of software solutions Richard Tait. "Everyone is there waiting for the scorecard. It goes in and then there's a period where bugs become evident and defects come out. [The bugs] are all individually trapped and counted and you watch the count go up and people are saying: ‘We had to have less than four for this release if we were going to beat the last release. How did we go?' It's a bit like the Oscars."

How they go is usually very well indeed. Since embarking on the path to Capability Maturity Model (CMM) accreditation the number of errors per 1000 lines of tested code within International Systems has improved by 93 per cent. The average cost per tested line of code has improved by 35 per cent and the number of days taken to produce 1000 lines of tested code has improved by 26 per cent. All improvements show up directly in the cost and time to deliver for customers.

That higher rate of success translates directly into a capability to undertake more and larger projects. When Tait joined the bank in 1999 the group considered working on two or three bank-wide projects at a time pretty demanding. Today the group has 20 ongoing $20 million-plus projects. That's the sort of stuff that tends to keep people like Tait awake at night unless the projects are being really done well.

Fortunately for Tait, he's sleeping fine; and International Systems is confident it is well on the way to even further improvement after being accredited CMM Level 2 in April and recognised as being well on the way to achieving Level 3. The achievement places the organisation in an elite group of companies worldwide who have attained this particular software capability rating. In Australia, ANZ is the first bank, and one of perhaps two organisations nationwide, to achieve a Level 2 ranking.

Tait says the fact that so few Australian organisations have achieved accreditation is a reflection not only of how difficult it is to achieve, but also that so many organisations undervalue the benefits.

Moving on Up.

At ANZ the rewards have been tangible, and they have been achieved far faster than conventional wisdom or experience would allow. It is commonly considered to take between three and five years to achieve Level 2 certification. But that puts the emphasis of the exercise entirely in the wrong place, Tait says. By concentrating on the process, rather than the certification, International Systems moved in just months from low levels of staff satisfaction to high staff satisfaction, unhappy customers to happy customers, all driven by the same outcomes.

CMM operates on the premise that focusing on a limited set of activities and working aggressively to achieve them helps an organisation steadily improve its company-wide software processes to enable continuous and lasting gains in software process capability. It presents a set of recommended practices in key process areas that have been shown to enhance that capability. Following such practices has undeniably borne fruit at the ANZ. Tait says the turnaround in quality and productivity was achieved within six months of setting out to achieve a CMM rating, proving it is the process and not the certification that counts.

"The certification itself, of course, is relatively unimportant, [that is] the actual label. What matters is the improvement in quality, the improvement in productivity, the improvement in time to market. They're the things we're trying to get out of it," Tait says. "Those sorts of improvements both in terms of time to produce a system and the cost to produce a system are obviously important for us and our customers."

CMM also gives organisations guidance on how to master their processes for developing and maintaining software and how to evolve towards a culture of software engineering and management excellence. That guidance too has borne fruit. After a rocky start and considerable early scepticism from staff, particularly in view of the initial early workload involved, the team now finds itself highly motivated. In fact, team members now do less work because they're spending less time fighting fires and more time focused on new development.

Standard Model.

CMM was developed in the late 1980s at the Software Engineering Institute (SEI) at Carnegie Mellon University in Pittsburgh. The model has become an industry standard to evaluate software development capabilities.

Devised as a common-sense application of process management and quality improvement concepts to software development and maintenance, the process model identifies 18 key process areas representing those processes that are most important to the effective management and delivery of software. It also catalogues five Maturity Levels at which an organisation can be ranked, each indicating a level of process capability (see "Maturation Procedures", page 92). At Level 2, the software process capability of an organisation has been elevated from ad hoc to disciplined, through the establishment of sound project management controls. Level 5 represents the highest Maturity Level an organisation can achieve, where processes are continuously improving. Few organisations have been rated at Level 5.

The SEI says the CMM is designed to help companies select process improvement strategies by first determining their current process maturity and then identifying the few issues most critical to software quality and process improvement.

Immature software organisations routinely exceed their schedules and budgets, based as these are on unrealistic estimates. Hard deadlines are only met at the cost of functionality and quality, and there's no objective basis for judging product quality or solving product or process problems. Review and testing are frequently curtailed or eliminated when projects fall behind schedule.

Contrast this with the profile of the mature software organisation, characterised by its company-wide ability to manage software development and maintenance processes. Mature organisations accurately communicate the software process to staff and ensure work activities are carried out to plan. They guarantee mandated processes are usable and consistent with the way the work actually gets done, updated when necessary, and refined through controlled pilot-tests and/or cost-benefit analyses. Project roles and responsibilities are clearly spelt out.

A CMM rating is so significant because the approach is based on achieving measurable improvements to an organisation, rather than simply passing a single assessment at some time. Organisations working with the CMM focus on implementing capabilities that improve productivity, quality and cost-efficiency, not simply documenting processes. The CMM assessments also follow a special approach that ensures the required capabilities are not only defined, but also actively used. Therefore, confidence in the level of quality and productivity of CMM rated organisations is high.

CMM has been used by organisations such as NASA, Boeing, Hewlett-Packard, United Airlines and IBM. Since 1996, 900 organisations have undergone a CMM assessment. Statistics show around 40 per cent of these organisations have achieved a Level 2 rating.

A number of case studies have testified to reductions in cycle time and impressive increases in productivity, quality and predictability with improvements in maturity.

Now ANZ understands just how those improvements are achieved. International Systems runs ANZ's core banking system in 16 countries throughout the Pacific, Asia, India and the Arab nations. Release 2.5, implemented in about the middle of 2000, was a turning point for the group.

The process of striving towards CMM certification began four months after Tait came on board in July 1999 to manage all technology for customer-facing business units.

"In October 1999 we started to look at how we stood against best practice and it would be fair to say we weren't very impressed with the result," he says. "We were having significant quality problems. We'd put in releases of software and find they would continually cause problems for our customers; they had a lot of defects. We were having trouble meeting dates. So it was a classic situation with out-of-control development, poor quality, dates slipped and customers unhappy.

"But I would have to say, while we were pretty bad, there would have been voices in the organisation that would have said: ‘That's right but that's pretty much the way it is - that's what the industry is like'."

Tait wasn't one of them, instead using the benchmark to give the team targets to aim for as it embarked upon the CMM exercise. As a result, quality and productivity improved fast. Culture, on the other hand, was much slower to look up, with turnaround in that area taking about a year to achieve.

Going Public.

Tait says the process for achieving CMM accreditation is very public, and involves demonstrating a high level of capability in numbers of key process areas against which the group is then assessed.

"The way the process works is [that] you decide you want to do this," he says. "Typically you would engage someone to come in and give you a baseline of where you stand. We used Accenture for that, in November 1999. Then once you're into the process you bring in a certified examiner who comes in, certified by Carnegie Mellon, to assess your processes and do what's called a pre-check in effect."

That pre-check proceeded as planned. In April this year the group went for its formal assessment and scored brilliantly, with the assessor commenting that the group had achieved the highest CMM Level 2 he'd seen and describing the group as being close to Level 3. Such rapid progress is, to say the least, highly unusual, Tait says.

The reasons for the rapid progress are no secret. Tait has been involved in certification in other companies where the exercise has typically been approached as a process improvement. Such an approach just doesn't work, he says.

"If you tell everyone: ‘Here's the process and this is how it's going to work', you're really saying that the certification is the outcome that you're after. That makes it very hard to motivate people. It is a lot more work for them, and in those early months you're working with a group of people who are working their bits off trying to get something delivered. At the same time you're saying: ‘Oh and now I want you to do all this additional process work on top of that'. It's very stressful to teams over that period," he says.

By concentrating on the process rather than the certification it was much easier to motivate people because they were aiming towards clear goals and could see immediate and tangible results, Tait says.

Constructive Criticism.

At the start of the process, people are inevitably told that the way they do things is not very good and they need to improve. Human nature makes it likely most will initially reject such criticisms. Only after they see the improvements for themselves will they start to engage. That is exactly how things happened in International Systems.

"The culture over the period of that year really changed from one of: ‘The way we do it is terrific and that's just the way it is.'', to one where today they're continually looking to improve it even further," Tait says.

He says the improvement could not have been achieved without a highly commited management team and the assistance of Accenture in change management. "[Culture] is typically the thing that stops people from being successful," Tait says. "There are very few companies that are certified. There's a lot more companies that try, and the failure is not because the people aren't good or they can't follow process. It's typically the cultural issues that defeat them."

Indeed, manager International Systems Marcello Martino concedes staff entered a "valley of despair at some points" because they had to master the new processes on top of their regular jobs. The key, he says, proved to be showing what they were achieving along the way towards meeting the targets.

"If you can show people exactly where they're improving and exactly what it is that's making that improvement, then they are able to see that and see the light at the end of the tunnel," he says.

"One large part of understanding where people are in terms of the culture and the changes was a regular survey which we called a Change Readiness Assessment. It tells us on a regular basis how well we communicate the change or how people are receiving those changes and how they react to some of those things. And if you do that on a regular basis you can see the change over a period of time," Martino says.

"A good way of showing that is what we called a spider-web chart. It showed the first assessment, which looked like a spider web. Then we superimposed another assessment over that, and a third and then a fourth. You could see very quickly where things were improving or where they were not improving."

Today Martino says the team is really pumped. They benchmark themselves against external companies on a fairly regular basis and therefore know how they stand against commercial providers. There is a public objective that the group should be better than any external provider.

"Basically, IT runs as an open market and we need to do a terrific job in order to gain the credibility of our customers," he says.

Once cultural barriers are overcome, Tait says, the benefits are many and varied. For instance, because CMM certification is a team-driven process, achieving certification makes you less dependent on your staff. All IT shops tend to have to or three "stars" that they rely on to pull them out of difficult situations. ANZ today relies on such people much less because of the strength of the team as a whole.

Lean and Mean.

ANZ has a commitment in the market to keeping its costs flat. Tait says the technology organisation is one of the few he knows that either keeps spending flat or reduces it every year while doing more each year. He says it is thanks to the bank's financial position of being a lean, mean bank that the technology area is able to do much more, more aggressively, with lower risk of failure, lower cost and faster delivery.

Now, having applied the CMM process to International Systems, ANZ has similar exercises running in its shop in Bangalore, which has 350 developers, and across the rest of the International Systems organisation. Tait says achieving certification in those other groups will be made easier thanks to the availability of templates and mentors who will act as champions for other parts of the organisation.

"We're applying it across India and across other parts of the Australian and New Zealand organisation. They will be quicker because there's templates, there are mentors, we take the staff out of the team that's been successful and disseminate them around other areas so they act as champions. So the next steps are really for the other parts of the organisation to come up to a similar level to our lead-time in International."

Maturation Procedures.

The SEI Capability Maturity Model for Software describes the principles and practices underlying software process maturity. Its raison d'être is to help software organisations improve the maturity of their software process through an evolutionary path from ad hoc, chaotic process to mature, disciplined software process.

There are five Maturity Levels categorised under the CMM:

Initial. The software process is characterised as ad hoc and occasionally even chaotic. Few processes are defined and success depends on individual effort and heroics.

Repeatable. Basic project management process are established to track cost, schedule and functionality. The necessary process discipline is in place to repeat earlier successes on projects with similar applications.

Defined. The software process for both management and engineering activities is documented, standardised and integrated into a standard software process for the organisation. All projects use an approved, tailored version of the organisation's standard software process for developing and maintaining software.

Managed. Detailed measures of the software process and product quality are collected. Both the software process and products are quantitatively understood and controlled.

Optimising. Continuous process improvement is enabled by quantitative feedback from the process and from piloting innovative ideas and technologies.

The SEI claims predictability, effectiveness, and control of an organisation's software process will improve as the organisation progresses upwards through the five Maturity Levels and claims that, while not rigorous, the empirical evidence to date supports this belief.

Except for Level 1, each Maturity Level is decomposed into several key process areas that indicate the areas an organisation should focus on to improve its software process.

The key process areas at Level 2 focus on the software project's concerns related to establishing basic project management controls. They are requirements management, software project planning, software project tracking and oversight, software subcontract management, software quality assurance, and software configuration management.

The key process areas at Level 3 address both project and organisational issues, as the organisation establishes an infrastructure that institutionalises an effective software engineering and management process across all projects. They are organisation process focus, organisation process definition, training program, integrated software management, software product engineering, intergroup coordination and peer reviews.

The key process areas at Level 4 focus on establishing a quantitative understanding of both the software process and the software work products being built. They are quantitative process management and software quality management.

The key process areas at Level 5 cover the issues that both the organisation and the projects must address to implement continual, measurable software process improvement. They are defect prevention, technology change management and process change management.

Each key process area is described in terms of the key practices that contribute to satisfying its goals. The key practices describe the infrastructure and activities that contribute most to the effective implementation and institutionalisation of the key process area.

Source: The Capability Maturity Model for Software, by Mark C Paulk, Bill Curtis, Mary Beth Chrissis, all of the SEI; and Charles V Weber, IBM Federal Systems Company