It's Hard to Measure Success
But Schlumberger's results aren't typical. For survey respondents, improving project success rates is a top goal, yet getting metrics that prove that PMOs are working takes time. In the CIO/PMI survey, 42 per cent of companies with PMOs less than 1 year old didn't know or do not track success rates. Only 22 per cent of companies with PMOs older than five years said the same. It's inherently difficult to pinpoint project success rates for PMOs less than three years old simply because there's no track record of completed projects. Even if CIOs can determine cost savings or success rates, benchmarking results against other organisations isn't a reliable gauge of progress because so many variables factor into the success of a PMO. "To justify the existence of a PMO, companies can build a business case with relative ease," says Robert Handler, vice president of Meta Group's enterprise planning and architecture strategy service. "Yet people want a good quantitative number, and it's difficult to have that silver-bullet ROI that's applicable in all cases." For Schlumberger's de Montmollin, the biggest benefit of the PMO - giving the CIO the status and financial details of all the company's IT projects - isn't something he can quantify.
One relatively quick metric to come by is customer satisfaction among internal end users. Burlington Northern Santa Fe Railway (BNSF) scores customer satisfaction numbers on completed projects and tracks ongoing activities quarterly. Since the PMO was instituted, these customer satisfaction scores have been consistently improving. Jeff McIntyre, BNSF's assistant vice president of technology services, says the company is struggling with other metrics that could peg project improvements directly to the PMO. "No two projects are alike, so it's difficult to do comparisons," he says. In addition, BNSF sent about 40 per cent of its development work offshore, so it's hard to attribute specific results solely to the PMO, says McIntyre. Yet BNSF is pursuing harder metrics; technology services is working on a Balanced Scorecard that will try to nail down measurements during the next year that paint an accurate picture of the PMO's effect on the bottom line as well as on processes and learning.
To create a PMO that is a good cultural fit, Handler and others recommend starting out with well-defined pilot projects that rely heavily on input from project managers in the business units. At OHSU, project management officer John Kocon concurs. "You have to really understand the culture, look at industry standards and best practices, and tailor them to the organisation," he says. "There's some give and take with project stakeholders who may resist doing things in a prescribed way."
To overcome such resistance, Kocon enlists support among senior managers. Others involved with PMOs say that senior management must be involved - either in terms of sponsorship or a direct reporting relationship - if PMOs are to be effective.
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.