We all know the arguments for benchmarking: It lets you compare your costs and service levels against those of similar organizations. It helps you focus your resources on improving any processes that have higher costs or lower service levels than industry norms. It can provide a solid foundation for comprehensive improvement plans.
Here's an argument against benchmarking: It can waste money.
True apples-to-apples comparisons don't come easily. You have to dig around to find out, for example, which accounting rules are used. Does a programmer's hourly cost include bonus and benefits? How about facility, technology and administrative support costs? You also have to make sure you understand how the various organizations in your benchmarking cohort calculate things like application response time, network availability and the percentage of incidents resolved on the first call. Accounting for all of those differences takes a lot of effort and many hours of work -- in other words, a lot of money.
Is benchmarking likely to be a waste of money? The answer could very well be yes if these factors come into play:
• Weak IT management processes. It takes several years to fully implement ITIL or to achieve significant improvement from a PMP program, and any benchmarking data gathered will be unfavorable (and likely inaccurate) until the new processes are fully adopted. Premature benchmarking merely indicates the size of the improvement opportunity. While this may motivate some organizations, the same information can be gathered more quickly by consulting someone who's an expert in ITIL, PMP, Cobit or other control process.
• Poor IT accounting and weak metrics. When internal processes are weak, data collection becomes quite complex. The benchmarking team needs to make assumptions about data allocation and must combine data from multiple sources. While this may be sufficient for planning purposes (which only need to be directionally correct), true benchmarking implies a high degree of precision. Even minor allocation errors make the analysis less precise and allow skeptics and critics to challenge the data and associated conclusions.
• Executives who aren't yet committed to change. Benchmarks help organizations determine how to change, but not why. IT leaders frequently base decisions on logic and rigorous analysis and will understand the need for process changes merely from benchmark results. But many executives want compelling arguments in addition to data; they want passion, energy and inspiration. With such executives, benchmarking is insufficient, and perhaps even irrelevant.
• A need for radical restructuring. Benchmarking only identifies opportunities to make incremental improvements to existing processes. When you need to rethink your mission, redesign the organizational structure, increase customer engagement or alter the way you deliver products and services, benchmarking is a total waste. It will have value only after you resolve the bigger issues.
Benchmarking requires a stable context, a significant investment of resources and periodic refreshes. Use them wisely. It's a diagnostic tool, not a replacement for sound business decisions. And certainly not a magic potion. Don't even think about benchmarking if you don't know how to use the results.
Read more about management in Computerworld's Management Topic Center.
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.