With 81 major projects behind it - and some mistakes inevitably made along the way - the Defence Department (DoD) knows a thing or two about market testing and contracting out. Since 1991, DoD has let a not inconsiderable $1553 million worth of contracts and achieved projected savings of $155 million, in a broad-based program which will eventually see just about every aspect of its operations go to market, save those at the "sharp end" of Defence.
We last spoke to DoD Inspector General Mike McNamara for CIO's special Government Edition in July last year, when he explained the process adopted during CSP-IT, the department's Commercial Support Program - Information Technology market testing project.
But McNamara's market testing and contracting experience goes way beyond that single exercise, which resulted in a big win for the in-house team. So CIO went back to him for a broader perspective on things. It was a richly rewarding visit that helped us compile this McNamara's Guide to Market Testing.
1. Garner Resources. Effective market testing demands enormous resources and doesn't come cheap. DoD has been lucky. It has had the support of Ministers, Defence Secretaries and Chiefs of Defence Forces who've been prepared to put their money where their mouth is.
"The sorts of activities we've had in our market testing exercise are not simple," McNamara says. "We are buying services, which I would argue are more complex than buying products or goods, and we need to have staff expertise either in-house or augmented by consultants who are adequately trained in the process of advanced and strategic-level procurement. We're talking about sophisticated services that are very complex - especially in running a complex mainframe data centre, for example. There's really a degree of high-quality expertise that needs to be trained and developed," he says.
Not all that expertise need be available in-house. When DoD wants commercial expertise it involves a range of consultants who provide a decision-making framework that assists in evaluations. "The teams that are involved are not one-man or one-woman bands," McNamara says. "You need a group with skills in government procurement requirements, financial evaluation, legal and contractual matters. In that regard we have relied on private sector and the Australian Government Solicitor to great advantage." But McNamara says having people with strong human resources (HR) and industrial relations skills is equally critical. "This is not just to manage the transition but also the uncertainty people are facing. We need to continue what my colleagues call the 'Day Job', while these market testing processes are on.
People are uncertain about their future. The informal information systems are much more powerful than any Pentium I've ever seen, and we need to make sure that informal information system is appropriately informed, and correctly informed. If you don't tell people what's happening, and don't make the facts available, they'll imagine the worst."Those are the words of experience. In the early days DoD didn't always manage personnel relations issues well. The results were unfortunate. These days there is a formal mechanism under which CSP meets with the unions on a regular basis and has extensive consultations with affected staff.
2. Tell 'em Clear, Tell 'em Often. As important as dialogue with staff is communication with industry. Without it, McNamara says, you're bound to miss some important opportunities. CSP consults extensively with industry, and uses the Internet to ensure suppliers are well aware of upcoming opportunities in the competitive environment.
There's also a formal consultative mechanism, the Defence CSP Consultative Forum. McNamara co-chairs the forum with Lend Lease Corporation's Rosemary Kirkby, who is particularly strong in HR, IT and change-management skills. "She brings some very good private sector experiences to that forum which are quite invaluable. That's really been an enormous fillip to the development of CSP. In fact a number of the practice notes that have come out have in their origins concerns raised in that forum," says McNamara.
3. Debrief the Market. You must ensure the availability of a robust market to compete for your business. According to McNamara, the only way to do that is to ensure suppliers believe your market testing process is completely fair and equitable, and then to help those suppliers learn from their mistakes.
CSP has a formalised process of debriefing unsuccessful tenderers that McNamara says can be a 'mind-expanding experience' for both sides. "My private sector colleagues tell me that quite often they'll bid for a contract and that's the last they'll hear of it. I think in terms of the quality of bidding, be it public or private sector, debriefing is a task that ought to be mandatory for all of us.
"It's not easy," says McNamara. "It's hard to do the debriefing, because you need to be quite precise about what you tell people, and it's difficult for some people to accept that they've lost. But it's also very good to debrief the winning tenderer, because you can say to them: 'Your tender was quite good except in x or y or z, and you need to lift your corporate game there.' They are so pleased to have won the tender they really do devote attention to it."4. Manage the Transition. Business continuity won't come about unless you manage the transition to a changed delivery provider, and that means making sure those in charge of the transition have adequate skills. But incentives can also help.
In times gone by all efficiency savings from the initiatives of government departments were recouped by the Department of Finance (now the Department of Finance and Administration). "That department has been very, very commercially sensitive and sensible in ensuring that we do retain the savings we recoup from our market testing initiatives," McNamara says.
For Defence that means it can retain efficiency savings for use in higher priority capability areas.
5. Focus on Outputs. "We simply have learned from experience that we need to have output performance-based contracts, not ones telling people how to undertake particular functions," McNamara says. "For example, I used to jokingly say when we were discussing the CSP-IT project with industry, that we didn't mind if they ran the data centre on an abacus, as long as the output was right.
"We didn't tell them they had to use proprietary or particular hardware in the data centre. That was their commercial decision, whereas in times gone by we might have tried to specify how tasks were to be done. That would not be utilising the commercial smarts that are available in the area."6. Explain the Process. The process being undertaken must be well understood and totally transparent. CSP has a well-documented manual and methodology which have been fine-tuned through consultations with industry and staff, and which are posted on the CSP home page on the Internet.
"The methodology is part of the process and if we have decisions that are not visible, defensible or auditable, our credibility will be irreparably impaired.
We spend a lot of time making sure we do have that process well tied down and part of the task of the CSP team within Defence is to ensure the decisions meet those criteria," McNamara says.
7. Learn from Experience. Some of the issues vital to CSP today weren't even on the radar screen four or five years ago.
"We've learned from our experience and we've learned from our interaction with staff and with industry and with unions in terms of how we need to modify the process. And the sorts of documentation that industry now contemplates is much more explicit and I believe much easier to come to grips with than [during] our first forays in this area," says McNamara.
8. Establish the Mechanisms. CSP has indeed been a learning process, and one thing the team has learned is that contracts do not manage themselves.
"I think in the early days of our market testing we thought if we contracted a function out, the end result was that we devolved the responsibility to the contractor and it was no longer a concern of ours. That is certainly not the case," says McNamara. "We need to put in place appropriate contract management mechanisms and the skill set there is a rather different one from that needed for undertaking the contract internally.
"Those staff need to be business oriented, flexible in the sense of allowing for the reality that they are looking for a service to be delivered and not something that's always easily definable in fine detail, and accountable for the management of their activities."9. Get the Criteria Right. In response to lingering concerns from failed tenderers back in the early days, CSP has developed a practice note on value for money and the decision tree approach it takes in assessing value for money.
"The thing is that in an environment where the Defence budget is constrained, it would be all too easy for us to accept the lowest price conforming tender.
We've resisted that temptation completely and have focused on the value for money issue, which is quite separate from the price," McNamara says.
"The Government's general procurement guidelines talk about value for money but they talk about it in terms we found while helpful, really didn't give you the cookbook you needed for practical applications. We've spent some time developing our own."10. Small Ain't Beautiful. "The other issue we've moved away from in market testing is penny-packet activities. Small is not beautiful in this context - and we do reap the economies of scale, no doubt about that - but that's not to say we don't facilitate the involvement of small-to-medium enterprises in the process," says McNamara. "What we are doing is ensuring that they, as well as the larger enterprises, are aware of contracts that are coming up.
"I note with delight that some of our very best contracting performers are not household names but small enterprises, some of whom are former Defence or public service personnel who've seen a market opportunity and have approached the large primes and are certainly performing extremely well."11. Resist the Rolls. One of the hardest parts of writing service contracts is defining the service levels required, many of them never before defined.
"For example in the IT context, when we talked to the program managers who were responsible for receiving service about their requirements, their typical response was: 'We need about what we get now.'"The next stage in the process required us to define some possible service levels, and we had to discourage some of our program managers from demanding Rolls-Royce solutions when a Holden Commodore would suffice. Defining an appropriate service level is, I would suggest, a major challenge to all of us.
And all of us are bombarded with the latest literature that talks about incredibly fast response times, and certainly we could acquire those, but they are cost drivers. So we need to be very careful in terms of value for money," says McNamara.
12. Integrity is Everything. Probity is absolutely essential to any market testing process. The Inspector General's office devotes considerable resources to developing fraud control plans and lecturing on probity and ethics. McNamara says that while allegations of impropriety have arisen from time to time, in the market testing area at least, they have proved to be pretty baseless.
"At times in the early stages some people required some guidance on how they should behave in relation to would-be contractors, because some of them saw their future careers moving towards the contractor. So we needed to get some guidance - and again, that is in the manual. I would expect in the next month or so we would have some new material out that documents that in fine detail."McNamara points out that in the private sector, the requirements of good corporate governance make probity every bit as essential as in the government sector. "Indeed in further development of our guidance to our staff in this we've relied a good deal on some private sector experience, because some sections of the private sector have paid particular attention to this," he says.
13. Conflict Ain't Clever. In recent times DoD has devoted considerably more attention to the issue of partnering with contractors than it initially believed would be necessary.
"It's quite clear that adversarial arrangements, particularly in the Defence context, are not smart," says McNamara. "We don't want to have a situation where a private sector contractor undertakes the task and we spend all our time in disputes with them looking at the fine print of contracts.
"That sentiment is shared by our contractors and we've had the interesting experience now where some of the contracts have come up for renewal, and our staff members are saying: 'Do we really have to recompete [for] these contracts? Can't we just keep with these people who are delivering a wonderful service that's much superior to what we used to have when we had it in-house.'"That's an intriguing experience, and it says something about the changed mindset, especially if you recall that these people - the internal Defence people - were competing against the contractors in earlier days," McNamara says.
"So we've got some very good and robust partnering arrangements. It not a cosy collusive sort of thing, it's an arm's-length and proper thing." DoD's guidelines talk extensively about partnering, and it has also drawn extensively from NSW Department of Public Works material, which McNamara says proved very useful. "We do seek a partnership-oriented, cooperative long-term relationship and this is particularly important in IT given the rapidly changing nature of the technology. If we tried to lock our contract in some sort of time-warp frozen situation we'd be in desperate straits.
"So we're looking to replace the traditional arrangements by a cooperative, team-based approach, but not a cosy collusive one. It's not about looking over their shoulder every minute of the day, or we might as well do the work ourselves."14. Redefine Your Core. Conventional wisdom says only core activities should be quarantined from market testing, but McNamara says he's noticed a trend in some organisations to develop innovative criteria defining "core" which is designed to prevent them from going to the market.
To circumvent this problem, McNamara suggests both the public and private sector should adopt this definition of core activities, developed by the Defence Efficiency Review: "Activity should be determined as core where it is not practical to establish business rules that would allow it to be contracted out."15. Remove the Brakes. So far, the CSP process has produced some impressive outcomes but the pace of change has been criticised as being too slow. Now DoD is looking at ways to accelerate the process. The idea is to engage in business process reengineering of activities before they are market tested.
"We'll run a fairly heavy roller over the activities and make sure we've got as realistic a proposal as you can get in terms of cost and performance," says McNamara. "You recall our savings under CSP have been about 33 per cent of the original baseline cost of the activity. What we would like to do is recoup some of those costs early in the process instead of waiting until the end. So we're using some BPR tasks to undertake that and my colleagues in the Defence Acquisition Organisation have quite a substantial project under way."McNamara concedes this could potentially slow down market testing, and points out that Eileen Schapiro, in the book Fad Surfing in the Boardroom defines BPR as "the consultants Full Employment Act".
"We need to be aware of that and we'll certainly put some fairly hard time frames in terms of the further implementation of market testing," he says.
The Commercial Support Program manual is online at http://www.csp.gov.au/MANUAL/toc.htm
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.