Critical.
Authoritative.
Strategic.
Subscribe to CIO Magazine »

Bursting the CMM Hype

As companies stampede offshore to find companies to do their development work, they first need to understand what CMM ratings really mean. Yet few CIOs bother to ask crucial questions, say IT industry analysts and the service providers themselves.
Page:

CIOs want to do business with offshore companies with high CMM ratings. But some outsourcers exaggerate and even lie about their Capability Maturity Model scores.

As soon as she walked into the meeting, Jane Smith knew that the executive on the other side of the desk wanted to buy something that Smith wasn't supposed to sell: a trumped up rating for the executive's software development division so that his company could qualify to bid on contracts from the United States Department of Defence.

Smith (not her real name) is one of a select group of experienced IT pros, called lead appraisers, who go into companies and assess the effectiveness of their software development processes on a scale from 1 (utter chaos) to 5 (continuously improving) under a system known as the Capability Maturity Model, or CMM. The company she was visiting wanted to move up to Level 2, but based on some initial discussions, Smith knew that the company was a 1. Level 1 describes most of the software development organizations in the world: no standard methods for writing software, and little ability to predict costs or delivery times. Project management consists mostly of ordering more pizza after midnight.

After a few initial niceties, the executive leaned across the table to Smith and another lead appraiser who had accompanied her to the meeting and asked: "How much for a Level 2?"

"That's when I got up and left the room," Smith recalls. "The other appraiser stayed. And the company got its rating."

The stakes for a good CMM assessment have got only higher since Smith's close encounter with corruption some 10 years ago. Today, many US government agencies in addition to the DoD insist that companies that bid for their business obtain at least a CMM Level 3 assessment - meaning the development organization has a codified, repeatable process for an entire division or company. CIOs increasingly use CMM assessments to whittle down the lists of dozens of unfamiliar offshore service providers - especially in India - wanting their business. For CIOs, the magic number is 5, and software development and services companies that don't have it risk losing billions of dollars worth of business from American, European - and increasingly - Australian corporations.

"Level 5 was once a differentiator, but now it is a condition of getting into the game," says Dennis Callahan, senior vice president and CIO of Guardian Life Insurance. "Having said that, there are some Level 3 or 4 start-ups that we might consider, but they have a lot more convincing to do before I would do business with them. They would be at a disadvantage."

With CIOs increasingly dependent on outside service providers to help with software projects, some have come to view CMM (and its new, more comprehensive successor, CMM Integration, or CMMI) as the ultimate seal of approval for software providers. Yet CIOs who buy the services of a provider claiming that seal without doing their own due diligence could be making a multimillion-dollar, career-threatening mistake.

That's because software providers routinely exaggerate their assessments, leading CIOs to believe that the entire company has been assessed at a certain level when only a small slice of the company was examined. And once providers have been assessed at a certain level, there is no requirement that they test themselves ever again - even if they change dramatically or grow much bigger than they were when they were first assessed. They can continue to claim their CMM level forever.

Worse, some simply lie and say they have a CMM assessment when they don't. And appraisers say they occasionally hear about colleagues who have had their licences revoked because of poor performance or outright cheating in making assessments.

Page:

Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.

More about: Bluechip Infotech, Carnegie Mellon University, Department of Defence, Forrester Research, Hayes, HIS Limited, IBM, ICICI Infotech, IMP, Infotech, ISO, Level One, Mellon, Northrop Grumman, OnStar

Comments

Post new comment

The content of this field is kept private and will not be shown publicly.
Users posting comments agree to the CIO comments policy.
Login or register to link comments to your user profile, or you may also post a comment without being logged in.
Related Whitepapers
Latest Stories
Community Comments
Latest Blog Posts
Whitepapers
  • Developing an Information Strategy - Strategize, Align, Govern, Execute, and Optimize
    An information strategy defines how a company will use the data it collects to achieve a competitive advantage. It is a comprehensive, constantly evolving plan that encompasses five distinct actions. In this white paper we explore how these five vital actions, as well as the technologies that enable and support them, can help organizations develop an effective and broad-reaching information strategy that drives positive change.
    Learn more »
  • Lower Your IT Costs When You Standardize on Oracle Database 11g
    As business operations become more complex, the demand for change in IT increases, along with the associated risks that must be mitigated. Today’s IT professionals are asked to manage more information and deliver it to their users in a timely manner with ever-increasing quality of service. And in today’s economic climate, IT must also reduce budgets and derive greater value out of existing investments.
    Learn more »
  • Eight threats your antivirus won’t stop - Why you need endpoint security
    News headlines are a constant reminder that malware attacks and data loss are on the rise. High-profile incidents that make big news might seem out of the ordinary. Yet businesses of every size face similar risks in the everyday acts of using digital technology and the Internet for legitimate purposes. This paper outlines eight common threats that traditional antivirus alone won’t stop, and explains how to protect your organisation using endpoint security.
    Learn more »
All whitepapers
rhs_login_lockGet exclusive access to Invitation only events CIO, reports & analysis.
Recent comments