Menu
Menu
Measuring the Piece of String

Measuring the Piece of String

Outcomes measurement is steadily gaining traction within Australian government organisations as a more accurate way of extracting value from those precious funding dollars.

As one of the largest local governments in the world, keeping the city of Brisbane running presents a very real logistical and administrative challenge. But that challenge has been easier in recent years when it comes to apportioning the $1.7 billion budget, thanks to a fundamental shift in thinking about the way the city creates its budgets and long-term plans.

The change came about several years ago, as the Brisbane City Council (BCC) began doing some navel-gazing as part of its millennium bug remediation plans. Much of that introspection focused on the issue of resource allocation. How, strategic planners asked themselves, could they think about expenditures in a way that would be more reflective of the value those expenditures provided to the city?

It was a difficult problem, particularly in a culture that - like most - saw funding in terms of input budgeting: allocating money at the beginning of the year based on educated guesses about how much would be needed. Strategy planners talked about broad issues such as water, traffic and air quality, and staffing levels. However, when it came to improving them those planners could only work in financial terms - for example, allocating more or less money to roads maintenance based on what was spent the previous year.

Early on, BCC had identified this as a far from ideal situation. “We recognised that looking at budgeting in terms of how much we had spent on contractors and overtime was no way to manage an organisation,” says Greg Ponych, principal budget finance officer with the BCC. “There’s only so much money to go around, and we need to be certain of the efficiency of the things that we’re doing and the effectiveness of those things.”

In other words, it is hardly useful to know only how much the council is spending on road improvements; what is important is whether traffic flows have improved in the long term. Similarly, it is not so important to have a number representing the quality of the city’s air; such figures only become meaningful when compared with those from other cities, and over time.

BCC searched the market for a budgeting tool that would allow it to allocate funding to particular areas and measure the value of that funding by the outcomes it produced. After initial searches to find budgeting software that would facilitate outcomes-based measurement, BCC was told no such system existed. It took the next-best approach, buying a budgeting application from Comshare and allocating developers to build the other functionality it needed. By late 2000, a strategic planning front-end application was in place, providing nearly 400 BCC users with access to the underlying Oracle database.

Three years down the track, the system has enabled a sea change in the way the council views its assets and budgeting. Data is captured from a broad range of environmental monitoring systems and fed into a centralised database, which contains both operational and evaluative information that is used intensively during budget deliberations. Because the data is coming from a single source, it can be reconciled easily and is used to create a variety of reports used in strategic planning.

As a result, BCC’s decision-making process has changed “from a numbers game to a strategy game”, says Ponych. “You still need the numbers, but you’re not having nearly as much time consumed with getting the numbers together. In a government situation, talking about outcomes is often hard; it’s much easier to talk about outputs, such as how many water mains we put in. But now we’re bringing together the financial and non-financial views of performance into a single point, and grouping specific services and products that we deliver under [broad] agendas.”

The New Face of Budgeting

BCC’s budgeting awakening was an early instance of what has become a much broader recognition that the magnitude of financial investments has little bearing on the benefits those investments deliver. In government organisations, outcomes measurement has steadily gained traction as a more accurate way of extracting the most value from precious funding dollars that have become increasingly difficult to come by.

In the corporate world, outcomes measurement has been rolled up under the banner of corporate performance management (CPM), a new business intelligence paradigm in which quantitative reports on business performance are paired with more qualitative outcomes assessments to allow performance-based costing.

Just as BCC’s largely in-house-developed system enables, modern CPM applications provide a way of gathering and acting upon business performance and financial information quickly, rather than waiting until period-end reports have been generated and analysed. In addition, qualitative outcomes such as customer satisfaction are continually measured from the various figures available to the system, and relevant stakeholders are immediately notified of change that exceeds allowable thresholds.

Over time, the collection of performance data helps organisations develop a clearer picture of which financial investments have produced the most valuable returns, producing a feedback loop that ultimately ends up in more detailed budgeting.

Research suggests this cyclical approach is becoming important to companies that used to be content with numbers alone. Conducted by CFO Research Services late last year, a study of 245 US companies found widespread frustration amongst CFOs who had become disenchanted with the limitations of conventional money-based budgeting.

Some 61 per cent of those surveyed reported they anticipated making changes to planning, budgeting, forecasting and reporting capabilities within the next 18 months. Approximately 71 per cent cited the reason for change as the need for better visibility into current results, while 70 per cent wanted a better understanding of future performance trends.

One of the biggest problems identified in the survey was the ability for budgeting mechanisms to handle change during the year: just 46 per cent of respondents were happy with the speed and accuracy of plan re-forecasts. Only 31 per cent were happy with their ability to model and test the impact of proposed changes. Perhaps most revealing, around 31 per cent also said that their current performance management processes had actually impeded their ability to provide decision support to the business.

Clearly, the problems that BCC identified four years ago have remained problematic. The issue is even larger within government departments, where strict policy controls and reporting requirements mean it may be impossible to implement such broad-based outcomes measurement policies.

Even when the will for change is in place, getting there can be difficult: proper outcomes measurement requires a wealth of data from different sources, but getting long-term access to this data requires navigation of a mire of privacy and other legislation, as well as the usual power struggles over ownership of data.

Complex departmental structures can be equally problematic, as the Sydney Catchment Authority (SCA) found when it recently began a program to improve budgeting and outcomes measurement across its structure.

A Catch-all Solution

Created in 1999 to manage the health of Sydney’s drinking water catchment and to deliver bulk raw water to Sydney Water, the SCA administers nine internal support and operational divisions located throughout the 16,000 square kilometres of land it manages.

In building its corporate structure, the SCA found that one of its biggest challenges was to coordinate communication and data sharing across its internal divisions. This effort was hampered by existing work practices based around Excel spreadsheets created to serve specific needs. These spreadsheets were often managed on a cash accounting basis rather than accrual accounting basis, which created a mismatch between the Excel spreadsheets and the data in the Sun Financials system.

The SCA bought Cognos business intelligence software and set about the process of reworking its reporting mechanisms. Aiming to improve integration across the board, it worked with Deloitte Consulting to replace the old system with automated tools linking the Sun Financials and Cognos systems to provide accurate information to internal and external users.

“Being a government body, the authorisation and delegation to spend money is very tightly controlled,” says Rhonda Wheatley, CFO with the SCA. “Because we were only two years old, we were a greenfields site and our focus was on implementing consistent, best of breed software to manage the organisation. But there had been a disconnect between the way non-financial managers did their costs and the way the financial systems used it. It was necessary to tighten internal controls as we developed our own culture and own corporatisation. Making everybody accountants wasn’t the answer, so we designed systems that operational groups could work with. It wasn’t a finance, IT or HR driven process; it was organisationally driven, which is key to why it works.”

The SCA’s business plan has set the context for a far more consistent financial management system. The Cognos system is based on a zero-budget approach that helps managers build budgets in line with corporate objectives. Use of consistent activity codes has enabled far more detailed comparisons of budget items, which will allow for better comparisons of costs versus benefits over time.

A further project, which recently went live, expanded the reporting system with forecasting and budgeting capabilities that are providing better visibility of performance across the organisation. And as the SCA’s fragmented culture continues to change, it will have both the technological base and the corporate impetus to make better performance management a reality.

The Quest for Better Outcomes

Because reporting is so important in government, the shift to outcomes-based reporting will have significant implications for the efficiency of resource allocation and utilisation. While it makes good business sense, reaching these goals is somewhat harder.

Widely discussed anatomies of CPM illustrate the technological components that go into an outcomes-based measurement system. At the core is a solid business intelligence system, which handles the heavy lifting of data analysis and report generation. Complementing that is, necessarily, a method for monitoring organisational performance and comparing it with business goals - for example, a Balanced Scorecard or corporate dashboard application. Also critical is a formalised method of measuring and organising the non-transactional elements related to outcomes monitoring.

“The practical implication of this is that you actually have to step back and define your processes much more explicitly than in the past,” says Simon Hayward, vice-president and research fellow with Gartner, which has actively documented the emergence of CPM philosophy. “Most organisations don’t necessarily have process models; things happen but they don’t necessarily know why. [Once you change this] you find yourself able to manage things in a different way: you can manage the whole activity rather than pieces of it.”

Getting there, however, takes a lot of work. At the highest level, the challenge of improved reporting comes down to the perennially difficult issue of integration ? both with measurable data and less readily measurable techniques. Combining those types of information requires a coherent technology framework, as well as the training to teach users how to make the most of the business tracking and reporting regime.

Perhaps no part of government is more advanced when it comes to outcomes measurement than the health-care sector, where the sheer breadth and depth of services delivered has made outcomes an extremely important part of regular operations. This imperative has become even more pressing in recent years, as ever-tighter budgets put spending under the microscope and performance comparisons drive efforts to consolidate redundant services.

For Queensland Health (QH), the push to improve outcomes reporting has led the IT and business teams straight into the maelstrom of integration that accompanies budgeting in such a large and complex organisation.

QH recently implemented a Crystal Decisions business intelligence platform to provide data analysis against a 100GB AS/400 database that contains historical and current performance data for Queensland’s top 25 hospitals. Previously, the department was running a Seagate Info analysis suite that forced each hospital to submit its own data - and department analysts to manually combine that data into an Excel spreadsheet.

Crystal Decisions has allowed the department to take a higher-level view, combining data from the many hospitals into a single database that has made relative performance comparisons easier than ever. The department pays considerable importance to issues of data quality, doing weekly evaluations to ensure that the information flowing from each hospital is of good enough quality to support comparative reviews.

The analysis possible using the new system has substantially improved visibility of organisational performance data, combining sheer transactional information with qualitative information such as explanatory comments that can be entered by divisional managers. This has allowed unprecedented analysis of spending in relation to more qualitative and subjective outcome measures, such as patient satisfaction and quality-adjusted years of life.

Being able to compare both qualitative and quantitative information across the whole of Queensland Health has been a significant coup for managers, who can now answer the burning “compared with what?” question that lies at the core of outcomes-based reporting. Yet even with the progress the organisation has made, changing the analysis and reporting culture is an ongoing process, says Bill Stomfay, team leader with the pricing strategy team inside Queensland Health’s Procurement Strategy Unit.

“We have reached the stage of comparative reporting, and are now getting into exception reporting and flagging variances,” he says. “It’s very hard to get the proper benchmarking before you’ve completed the technology. In terms of where we need to go with IT, it’s about getting an integrated, patient-centric view of health services provided to individuals - as opposed to what we do now, which is keeping a tally of how many people go through our facilities. Once you can hone in on the significant variations, you can set the agenda to go through and identify specific areas to focus on in the future.”

Ultimately, the application of business intelligence to very real business issues is the goal of outcomes-based reporting. As government departments steadily improve their operational knowledge through implementation of better business intelligence systems, the natural progression into outcomes-based reporting will produce better information, and more efficient operation, in the long term.

Performance Management Peeves

As a recent survey proves, CFOs are growing increasingly frustrated with the limits of money-based accounting

A study of 245 US companies conducted by CFO Research Services late last year found widespread frustration amongst CFOs who had become disenchanted with the limitations of conventional money-based budgeting.

Performance management processes are broken. While 63 per cent of finance executives polled say they are satisfied with the speed and accuracy of their monthly financial reports to business units, only 48 per cent are satisfied with the speed and accuracy of plan re-forecasts during the fiscal year, and only 40 per cent are satisfied with the amount of time it takes them to create an annual performance plan. Just 31 per cent are satisfied with their ability to model and test the impact of proposed changes. A similar percentage say their current performance management processes actually stand in the way of their ability to provide decision support to the business.

Most CFOs plan changes to their performance management systems and capabilities. Some 61 per cent of survey respondents plan changes to their performance management systems in the next 18 months. One area widely targeted for change is forecasting - specifically, the ability to produce rolling rather than static forecasts. Only 36 per cent of the respondents have that capability today, 84 per cent expect to have it within three years. Many CFOs also plan to produce budgets with a sharper focus on key business drivers: 27 per cent have the tools and procedures in place to do that today, 76 per cent expect to have them in three years

Major reasons for seeking change are to improve visibility into current results and better understand future performance trends. Of those surveyed, 71 per cent cited the need for better visibility into current results as a factor in their decision to overhaul their performance management systems, and 70 per cent cited the need to better understand future performance trends. Another 42 per cent cited poor economic conditions and the need for tighter cash flow controls, 25 per cent cited regulatory changes such as the US Sarbanes-Oxley Act. New financial regulations are especially worrisome to large companies (those with $5 billion or more in annual revenues) - 50 per cent of these companies cited regulatory changes as a reason for improving performance management.

There are obstacles to improving performance management. The two main barriers are a lack of integrated IT systems and competing priorities. Other obstacles cited included change management fatigue, lack of funds for process and/or systems change, concern about the difficulty of implementing a new system, inadequate senior management support and resistance from the business units.

There is pressure to abandon spreadsheet-based planning and budgeting. Only 11 per cent of survey respondents are very confident that a spreadsheet-based process ensures the accuracy required for CFOs to feel comfortable signing off on financial statements, as required by the US Sarbanes-Oxley Act. Another 42 per cent say they are somewhat confident, leaving 47 per cent who say they are not confident.

Companies are migrating towards single-solution software. About half of companies use a non-integrated mix of performance management software across their organisations today. In three years, most plan to standardise - either using a single solution across the company (50 per cent), or by using an integrated set of best-of-breed software (39 per cent).

Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.

Join the newsletter!

Error: Please check your email address.

More about Brisbane City CouncilCognosComshareCrystal DecisionsDeloitte ConsultingDeloitte ConsultingDeloitte ConsultingGartnerOracleQueensland HealthSydney WaterThe City of Brisbane

Show Comments
Computerworld
ARN
Techworld
CMO