Another Trip to Hell

Another Trip to Hell

The timing for a project blowup could not have been worse. In the summer of 1999, Hershey Foods suffered a glitch in a $112 million new enterprise system built to automate and track every step of the company's candy-selling business. Just days before Halloween the problem was still unresolved, and business took a scary turn for the $4.4 billion candy-maker. The lost orders, missed shipments and disgruntled customers that resulted from the company's systems woes were well publicized in both the trade and business press. While Hershey would not reveal its losses from the glitch, third-quarter revenues were down $151 million from 1998.

What happened to Hershey can happen to any company, any time of the year. Unexpected changes in management or business direction, or the loss of key staff members, can send high-profile projects into a tailspin. Many problems are preventable, and yet the same ones seem to crop up time and again. The horror stories below tell of out-of-control consultants, project managers who've never managed an IT project before and sponsors who disappear until the product is finished and then report that it's absolutely nothing like what they had in mind.

CIOs can minimize the risk by clearly defining leadership roles, getting commitment of sponsors from the business and securing executive support and participation. Other steps are less obvious. Not all of the CIOs who agreed to share their stories felt comfortable disclosing their companies' names or their identities. However, the lessons they've painfully learned have helped them become better leaders today.

If you suspect IT is working in a vacuum with no help from the business sponsor, pull the plug-lest you care to witness 11th-hour scope creep.

The Project

In 1995, Omni Hotels, a small, struggling hotel chain, decided a snazzy guest recognition system would boost customer loyalty and drive repeat sales. Omni's executive team, desperate to improve the company's lackluster performance, pinned the company's financial future on the project. Hotel employees would reward repeat guests with gifts or room upgrades as they checked in and note guest preferences for future visits and promotions. The marketing department happily promoted the loyalty program to customers and hotel managers, but, when the system went live a month behind schedule, the hotels hated it: It had little business value, was difficult to use and ran slow as molasses. "It was a joke," says Thomas Murphy, CIO at Royal Caribbean Cruises in Miami, who was Omni's vice president of IT at the time. The system was yanked just six months later.

The Problem

Early on, Murphy tried in vain to get the company's vice presidents in one room to define the system requirements. The executives couldn't be bothered with the details, however, and sent their underlings instead. When IT presented a demo to the business units, all hell broke loose. Managers from marketing, sales and operations began fighting over functionality. Lacking input from the business units, the system had been designed without any rules for granting the rewards. No one could agree on when and how guests would receive gifts. The hotel managers-who had never been consulted about the project-were hardly thrilled about making room in their budgets for bottles of wine and other pricey gifts. Worse, the marketing department had kept the CEO in the dark from day one. "When we sat down, his expectations were wildly different than what was being developed," Murphy recalls. "That was a gut-churning moment."

Murphy admits that the lack of a business sponsor was the ultimate oversight. The business units couldn't decide among themselves what they wanted, the CEO had a grandiose vision of a dynamic system that would never happen in the allotted time frame, and the hotel managers who had to implement the program wanted nothing to do with the whole mess. Politically, it was impossible to pull the plug-yet deep down, Murphy knew the system was going to be a sham.

Murphy and his team raced back to the drawing board and spent a month frantically reengineering the system in an attempt to placate the various warring parties and meet the CEO's desire to get the application up and running ASAP. The laundry list of requirements included ease of use, speed, improved reporting capability and interfaces to other legacy systems-among other surprise requests. Oh, and make it sexy, they added. On top of that, there were drastic technical problems to address. The software had been developed on a Microsoft Access database rather than on a relational database, which could have scaled to the requirements of Omni's 40 hotels. Further, users had to access the software through sluggish dial-up networking since Omni did not yet have a wide area network installed. Updating the database each morning tied up a PC at the front desk for an hour. Finally, no formal training or documentation was provided to the hotels. First-time CIO Murphy candidly admits, "My relative inexperience certainly did not help the situation."

The Recovery

There was none, and the company lost $250,000 in the process. Murphy says the failed project succeeded only in further exacerbating internal conflicts between business units that had been brewing for some time. Years of turnover in senior management had created a rift between long-term Omni middle managers and the autocratic executive team. Not long after the system went live, TRT Holdings of Irving, Texas, acquired Omni, and Murphy decided not to join the new company.

Heavenly Hindsight

1. Sponsor, sponsor, sponsor! And it's not enough just to assign the role-CIOs have to be sure of commitment. Eventually, Omni's VP of Rooms Operations became the official business sponsor, but Murphy says his travel schedule kept him out of the office most of the time.

2. Insist on a well-defined scope. Require participation from business sponsors and users-even for the mundane technical details. "The business did not want to hear about technical challenges," Murphy says, remembering that the reigning attitude was that IT ought to be able to make anything work.

3. Do your infrastructure homework. Will your network and databases support current and future demand and usage patterns for the system you're building? Do whatever you can to assess the situation, even if it means bringing in a consultant from the outside.

4. Choose technology, not skills. If you don't have the right skills for the project, hire them. Murphy admits that he used Microsoft technology simply because those were the skills he had on staff.

Pity the company that lets a star consultant take over. Like good government, every project needs a system of checks and balances.

The Project

Big egos are no big deal in Hollywood, yet one studio didn't foresee the consequences of allowing an egomaniacal consultant to control a royalty project. The bread-and-butter accounting package in music studios, a royalty system calculates and tracks payments to performers and helps studios avoid lawsuits over contracts. While these systems often cost upward of $10 million to develop, this studio had determined it could develop one for $7 million in 18 months, says the former IT director of the studio's music division (still working in Hollywood, he requested anonymity). Its secret weapon, he says, was a savvy designer who had created a stellar royalty system for a competitor.

Before long, the project began to hit sour notes. The studio's lawyers protested how much the system would automate the contract process. Studio vice presidents stepped into the ugly fray and delayed the process further. Finally, the designer ignored everyone and began creating the system he had envisioned from the beginning. When the IT director left the company in 1997 (out of general frustration with the studio's divisive culture), the project was still under way, having racked up $10 million in costs with no completion date in sight.

The Problem

There were too many stakeholders who wanted a say in the outcome, and no strong sponsor who could mitigate the factions and get things back on track. The executive in charge of the project was a senior user with no experience leading such a project, which only contributed to the politics. "He was far more concerned about spinning the right message to the executive committee than he was about applying good oversight," the IT director says.

This left the designer-in whom the vice presidents had placed the utmost confidence-with the power to take the project where he pleased. "From the beginning, he became almost unmanageable," the IT director remembers. "He refused to share design and analysis concepts with the rest of the team. He refused to be beholden to budgets and schedules. He basically said, 'That's not the way my mind works.'" For the IT staff, the studio's closed corporate culture made frank discussions nearly impossible without screaming matches. "There was never an atmosphere of low-penalty information flow," he recalls. "No one was allowed to take the authority to make decisions, and yet no one was comfortable making decisions as a committee."

The Recovery

Again, there was none. A consultant who also worked on the project confirms that the studio is still developing the system. The executive user in charge of the project was replaced with another senior vice president, who applied even less budgetary and scheduling discipline to the project.

Heavenly Hindsight

1. Emphasise team, not individuals. Not only is playing favourites bad for team morale, but it gives too much control to one person.

2. Appoint an experienced it project leader. "If you have a $10 million project, don't put it in the hands of someone who has never managed a $50,000 project," the studio IT director says.

3. Don't kill the messenger. Project leaders and other participants need to feel comfortable waving a red flag without jeopardizing their career.

Few CIOs neglect infrastructure planning today, yet for urgent projects it's easy to minimise network needs and focus on the application. The oversight can be costly in more ways than one.

The Project

In 1992, a California pharmaceutical company yearned to reach new levels of productivity in an industry where it usually takes $500 million and between eight and 12 years to bring new drugs to market. The company hoped a new global knowledge sharing application would cut its eight-year development cycle in half-and presumably, the associated costs. The multimillion dollar project was a combination of custom-built and off-the-shelf application software. To further complicate an already complex system, the development was split between IT and a "major" consulting firm. Nine months into the project, the network lab tested the system and found it suffered from sickly response time. "It was so poor it was unusable at first," says Jeff Lucchesi, formerly a network manager at the pharmaceutical company, now CIO at DHL Airways in California. The system and its software components had, inexplicably, not been designed to run over a wide area network.

When the application was finally released, it was eight months late, $1 million over budget and lacked the functionality promised to its users. "The only thing on time," Lucchesi asserts, "were the consultants when it came to getting their checks."

The Problem

The first mistake IT made was leaving its infrastructure group out of the project until the testing stage. "IT didn't have time to scope it out well," Lucchesi remarks. "We were designing and building it at the same time." Second, no single person was accountable for the project. The consulting firm had its own project manager, as did the pharmaceutical company. To add to the confusion, each division in the company had its own IT director, and not all of them reported directly to the CIO. Consequently, when it was apparent the design had fundamental flaws, the finger-pointing and sidetracking began. "That was when the CYA ["cover your ass"] really kicked in," Lucchesi smirks. The consultants wedged themselves between IT and the business, taking advantage of the company's decentralised IT structure to win the trust of the business.

There were other problems with the consulting relationship. The contract was designed as a time and materials (T&M) agreement, which means the consultants billed as they worked rather than agreeing to a fixed cost based on deliverables, upfront. This contributed to scope creep, worsening the network problems, Lucchesi says. Today Lucchesi won't even entertain a T&M contract.

The Recovery

In the end, R&D users did not get the system they wanted because pieces of the application had to be stripped out to improve performance. To no surprise, IT/business relationships suffered measurably as a result. Lucchesi applies the lessons he learned from this story to his current job at DHL, where he's proud to report he's never had an infrastructure problem. He recently completed an 18-month enterprise system project under budget and on time.

Heavenly Hindsight

1. Commit time upfront for project planning. If that's not possible, IT needs to inform business sponsors of the risks (software bugs, deadline extensions). Require the involvement of those responsible for infrastructure, development and project management during initial meetings with the sponsors.

2. Test the network end to end. Do this testing before you start development and before purchasing additional bandwidth or making network upgrades. If you don't, Lucchesi warns, you'll wind up spending too little, or too much, on bandwidth.

3. Hire the best project manager possible. Lucchesi uses professionally trained project managers rather than shoehorning a technical person into the role.

4. Avoid T&M contracts. Since the experience, Lucchesi insists on a fixed-cost relationship with consulting firms. However, he adds, "It takes a lot of upfront work to really scope the project and put in contingency plans."

The time to learn that your company is adverse to big change is not midway through an enterprise system project. It's wise to assess business goals and cultural readiness before purchasing any technology.

The Project

In 1995, mass confusion reigned at an Atlanta-based manufacturer of forestry products. Its 12 divisions were running separate back-office systems, and financial reporting was a nightmare. For instance, $7 billion in sales invoicing was actually internal billing between divisions that exchanged goods and supplies. To simplify, the CFO and CIO jumped headfirst into the zany world of SAP. At first, the various business VPs and stakeholders were all for centralization. On paper, it sounded like the right thing to do.

But 11 months into the project, trees began to fall left and right. Once the vice presidents realised they would soon lose their respective empires for the sake of centralisation, they revolted. The project was stopped dead in its tracks before the pilots had begun, according to Dan Sheehan, a former senior IT manager on the SAP project, now CIO at Atlanta-based staffing firm Acycs. As a result of the project, the company had to write off $120 million in software development losses in 1997.

The Problem

In Sheehan's view, the biggest gaffe was the classic IT mistake of choosing the tool before understanding the business problem. There were no frank discussions with the executive team about the company's long-term strategy and goals until after the contracts were signed and the project team was well under way. This leads to another issue: cultural readiness. The CIO and CFO, who were champions of the project, did not consider whether the company could handle such sweeping change all at once. The plan was a big-bang approach of launching pilot projects in two divisions on five SAP modules simultaneously, followed by the rest of the company.

And to pick up a common theme, consultants got in the way. The company used four consulting firms at various phases of the project, all of whom bickered among themselves over how to approach the project. "SAP didn't have their own methodology at the time, so we were taking all the advice from third parties-who were probably making it up as they went," Sheehan observes. Furthermore, many of the consultants were more interested in controlling the project (and overstaffing it, he adds) than partnering with the company. The project was eventually shelved.

The Recovery

Once the project was killed, senior management gave division vice presidents the choice to implement SAP or not, and not surprisingly, only one did. Sheehan says the company's systems are still not integrated, and little has changed. Sheehan left soon after to take a senior IT position with another major manufacturing company in Atlanta, where he experienced yet another trying ERP experience resulting from a change-adverse culture.

Heavenly Hindsight

1. Help people feel comfortable with technology. With his current employer, Sheehan takes "road shows" to the business to demo products and discuss new technology concepts like ERP.

2. Make business needs a priority. Sheehan says he strives to be a resource to his current CEO rather than touting his own agenda.

3. Proceed with caution. At the forestry company, Sheehan says they should have opted for a slower, phased-in approach for SAP, launching the product in just one division initially.

4. Don't let consultants rob you blind. Some consulting firms employ a "pyramid" approach, where they involve many people in order to rack up fees. During the SAP project, Sheehan recalls how discussions between consultants and their managers were invoiced as billable time.

Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.

Join the newsletter!


Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.

More about MessengerMicrosoftSAP Australia

Show Comments