Attacking the millennium bug calls for a prudent battle planCIOs start talking like Patton when asked about the transition from 1999 to 2000. Talk of plans for the switchover is peppered with references to "shock troops", "war rooms", "readiness drills", "contingency plans" and the ever-popular "triage". Give 'em a cigar, and they're ready for battle.
The objective, of course, is a seamless segue to the 21st century, a business-as-usual "non-event", according to several Y2K program managers. To ensure reality matches those rosy scenarios, CIOs and their staffs are constructing intricate battle plans for the final months of 1999. They're guarding against recontamination by tracking the continued compliance of applications that have been fixed and tested, crafting contingency plans, putting together SWAT teams to handle unanticipated problems and conducting drills to gauge their Y2K readiness.
"Most people still believe that they're going to finish their applications on time and the transition will be fine," says Phil Carrai, president and COO of McCabe & Associates, a Maryland company that develops visualisation and Y2K testing software. "But you need to start developing your plans now for how you're going to handle it. You can't circulate a memo about your plans for the transition on December 15 of next year and expect everyone in your organisation to absorb it."Like any big event, the Y2K roll-over requires preplanning, with preparations completed at various intervals during a 12-month period or more (kind of like planning a wedding, only less joyous). For better or worse, the first two days of the year 2000 are a Saturday and Sunday. A well-conceived battle plan will mean the difference between a celebration ringing in the new year or an unprecedented weekend from hell.
12 Months: Contingency Planning
Few CIOs and Y2K managers are willing to acknowledge the prospect that all their vital systems won't be ready by December 31 of next year. As a result, they're likely to leave contingency planning until the last minute.
That's a mistake, says Carol Dow, Y2K program manager for the Vanguard Group in Pennsylvania. "You need to have your contingency plans hashed out by the end of this year," she says. Why so early? Date-related failures are likely to occur long before 2000 arrives. And contingency plans, such as identifying an alternate vendor for an important product or service, can take months to implement.
Contingency planning should be a team activity, involving not only IT staffers but also representatives of each business unit, business continuity experts and attorneys. "You need all those people to help you understand what kind of failures can hurt you the most, what areas have the highest probability of failure and what your options are in dealing with those failures," says Chas Snyder, director of the Y2K project for Levi Strauss' US operations in San Francisco.
The contingency planning team needs to think about both kinds of Y2K failures: those caused by internal systems and those caused by external systems. If an internal system goes down, is there an older backup system? Snyder says that if Levi Strauss' EDI transactions falter, orders will be transmitted via fax, just as they were in the days before EDI. At another level, can the process somehow be handled manually? Can it be outsourced temporarily to a reliable provider?Unfortunately, it will be harder to plan for external system failures. "We're most concerned about the things that aren't under our control," says Dennis Grummer, director of the Century Compliance Project Office at Sears Roebuck in Illinois. "What if we don't have power or phone service?" Grummer says that his company is working with US trade associations like the National Retail Federation to get information from utilities about the status of their Y2K projects -- and apply pressure if necessary. "There's strength in numbers," he says. "You can stress the issue more when you get together with other members of your industry."Good contingency plans can't have only one level of "what ifs", though. Barbra Reagor is the senior director for risk management solutions at Bellcore, a New Jersey software and service company that has helped telecommunications companies recover from bombings, earthquakes and floods. "You need to think deeply about your contingency plan," she says. "How solid is your plan B? Go to your hotsite providers and backup service providers and ask, 'What happens if all of your customers show up at your door on January 1?'"9 Months: Pollution AlertCompanies talk about being "done testing" by this quarter and fourth quarter as though the early completion date were a goal in and of itself. But testing is never finished. Y2K managers are more concerned with what they call "continued recertification" and "compliance tracking" than ever. The danger is that once systems are tested and returned to production later this year and in 1999, they'll be recontaminated. That possibility is frightening.
Programmers adding new functionality or making additional fixes can introduce non compliant routines, or external entities can send you bad data. Users can corrupt spreadsheet-based applications by using two-digit dates. Howard Rubin, president of the consulting firm Rubin Systems and chairman of the computer science department at Hunter College in New York City, says that half of all Y2K failure events will be caused by what he calls "systems pollution-injecting bad genetic material into an application".
To avert such a situation, you must institute -- or beef up -- change management processes. Old habits die hard, and it's easy for programmers to forget about date standards (or windowing procedures) when working on newly compliant systems. "Look at all changes that get made and do a walk-through with the programmer who made them," advises Ian Hayes, president of Clarity Consulting in Massachusetts, and co-author with William Ulrich of The Year 2000 Software Crisis: The Continuing Challenge. Hayes knows of a financial services company that has discovered 14 new date-related bugs in the 18 months since its compliant and supposedly remediated systems were put back into production.
"There's a high likelihood that programming changes will reintroduce Y2K errors, and if you don't have a really good change management environment, you're vulnerable," warns Hayes.
It's also important to watch for bad data coming in from outside sources. "You need to sit down with the subject matter experts and walk through every process," says Jerry Hermes, consulting manager of Y2K solutions at application-development software vendor Micro Focus in California. "What happens if you get bad data at this point? How would you detect that, and what impact would it have?"Companies should put controls in place to prevent employees from purchasing new products that aren't compliant. And they should be wary of buying products marketed as "Y2K ready" that don't live up to that billing, according to consultant Jerry Trognitz of International Consulting Services in Dallas. He also advises companies to try to limit the scope of new development in the months leading up to the transition. "You need management support to do that," he says, "but it works wonders to limit the amount of recontamination you'll see."Employees also need to undergo a shift in mind-set. Communicate the need to handle dates properly, with a "19" before the last two digits. "No one has really talked about the 'thinking' problem," says Jon Church, senior director of operations and Y2K program manager at Software AG Americas, headquartered in Reston, Virginia. "Recontamination is still very real until people start putting 19s before their 98s."6 Months: Assemble the SWAT TeamAs the potential for unseen snafus increases, Y2K managers say that a reliable, rapid-response team should be ready.
"You need a SWAT team that's on call to handle any unanticipated problems during the transition," says Irene Dec, vice president of IS and corporate information technology and year 2000 program manager at The Prudential Insurance of America in New Jersey. "The team should include technical people, operations people and business people. You need the right blend of knowledge and skills so that the team can handle any kind of issue."The SWAT team needs to have access to comprehensive documentation about the entire Y2K project. "What's my checklist of all the work that was done, and how much was each system tested?" asks Carrai. "How are these systems tied together, and who's responsible for what? What are the downstream effects if this system stays broken for a day or a week?"To answer those sorts of questions, Prudential has established a database of all Y2K activity that includes project plans, contact names and technical alerts. "That entire package of information would be available on a second's notice to the SWAT team," says Dec.
SWAT team members also need to clearly understand the process for alerting senior executives about problems. "Decisions about contingency plans need to be made by executives in the business areas who can best determine the risks of damage to the business," says Hayes. "If you have a supply chain failure and you start running low on parts, how should you react? No IT person can make that decision alone."Before time runs short, it's crucial to make sure that all of the most important SWAT team members are committed to working over the transition weekend. Hayes reports incredulously that the programmers' union at a major European airline has already decreed that none of its members will work overtime that weekend. It's the biggest party of the century, union officials reportedly said, so "plan around us -- we're not working".
As the six-month mark rolls around, Y2K reporting needs to shift into high gear as well. "With 180 days left, you need to start managing your project team like a war room," says Rubin. "You're managing in real time. You need day-by-day status tracking and bite-sized tasks with instant accountability."3 Months: Readiness DrillsEven the best SWAT team can't be expected to execute a Y2K transition flawlessly without any practice. For that reason, many experts advise running "readiness drills" over the course of a weekend to simulate how the team would respond to a major system failure. What steps do they take to diagnose the problem? To minimise downstream effects? Is the team comfortable with using the Y2K documentation? Do members know who to call for decisions about contingency plans? Are they familiar with the plans themselves?"You want to get your SWAT team in shape, although you hope that you won't need them," says Earl Todd, Y2K project manager at Boeing's Rocketdyne division in California. "You want a team that knows exactly where to look [for the causes of problems] and is familiar with the procedures for fixing them."And the simulations can't begin too early, says Ulrich, president of the Tactical Strategy Group in California. "Some companies have systems that will blow up next year," he claims. "You don't want to be taken by surprise."Come next year, the organisation's awareness about Y2K should be at a peak.
Employees at all levels should be on the lookout for any indications of date-related problems and should know who to contact. Help desks should be equipped to handle increased call volumes. Companies that operate on just-in-time principles might want to stock up on extra inventory. The SWAT team should be ready to swing into action, all IT staffers should be on call and contingency plans should be ready to go.
Happy New Year
Your heightened level of awareness shouldn't dissipate once 2000 arrives.
Expect problems to crop up throughout the year, especially around Feb 29 (which appears in years ending in 00 only every 400 years), and month-end, quarter-end and year-end reports. "Your SWAT team needs to stay on call, your hotline and support numbers should stay open and everyone should know exactly what to do if there's an issue," says Prudential's Dec.
"A loss of data integrity could take days or weeks or months to recognise," says Snyder. "That's the scary thing. We're deploying all the resources we can to look for problems like that."Some experts say that organisations will need to practise Y2K vigilance not just in 2000 but for several years afterward. "You need to be on guard the whole time you're undoing creative fixes and patches, dismantling contingency plans and taking down data bridges that you don't need anymore," Ulrich cautions. "That could last until 2005."By then, CIOs and Y2K project managers will know whether it was their best-case scenarios or worst nightmares that played out. With luck, the battle fatigue you suffer can be easily cured by a nice vacation.
Scott Kirsner can be reached at firstname.lastname@example.org.
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.