Nightmare on Config Street

What does it take to bring down a Web server? Try a misplaced comma in a configuration file. That tiny typo once took three servers offline for a major player in the hospitality industry, says Jim Hickey, vice president of marketing at mValent, a producer of configuration management products. A routine check of configuration files using mValent's Integrity app uncovered the error, which might otherwise have gone undetected.

In fact, three-quarters of enterprises surveyed by mValent said they'd suffered application downtime during the prior month due to a configuration glitch.

"One of the dirty little secrets of the software business is that there are hundreds of configuration files with tens of thousands of individual parameters that need to be tuned to make the infrastructure work and keep apps running," Hickey says. "What keeps IT pros up at night is worrying about who has access to these files, what changes are being made, and if they're happening in a controlled fashion."

State Street, a Boston-based custody bank, uses mValent Integrity to check for errors in in-house application development for its Wealth Management Division. Joe Kennedy, vice president of technology architecture and R&D, estimates 30 percent to 40 percent of the problems his organization encounters are due to configuration errors, not bad code. Avoiding such errors is critical to keeping the business running.

"When there's a configuration error, nine times out of 10 you have an outage," Kennedy says. "That's just not acceptable in finance. When you're dealing with people's money, you can't be down."

Configuration management is really part of the bigger challenge of managing in a constantly changing environment, says Charles Ramsey, executive vice president at Service-now.com, an on-demand IT service management company.

"What's keeping IT execs awake is trying to understand what the heck is going on in their environments," Ramsey says. "I recently met with the CIO at a major wireless carrier, which has a change management app so complicated no one uses it. They probably have 85 systems of record in the IT org stored in Access databases, Excel spreadsheets, and on the mainframe. There's no point of integration between them."

Ramsey says enterprises can get a handle on such problems by combining asset, change, configuration, and problem-management tools into a single system of record -- which, not surprisingly, is what Service-now offers.

"The service desk is a critical component," Ramsey says. "If there is true integration and all applications behave in a similar manner, processes like change, problem, asset, and release management all will contribute to having a more effective service desk."

Help! My Network's Overrun by Rogues

Pop Quiz: How many enterprises have software installed on their desktops that their IT departments don't know about and wouldn't approve of if they did know about them?

Answer: All of them, says Peter Evans, vice president of marketing and business development at Internet Security Services.

"Probably 100 percent of enterprises have a problem with rogue software," Evans says. He also says employees typically download software that makes their jobs easier or favorite programs they've used in the past. Many times, though, they're installing IM clients or peer-to-peer apps, which can cause serious problems.

"Any software installed without appropriate oversight can introduce security risks," says Ed Moyle, manager of CTG Consulting, an IT staffing and consulting firm. "We're seeing a lot of interest in extrusion prevention software that scans outgoing network activity for confidential or proprietary data, to make sure it doesn't leak out of the firm."

And if rogue software troubles your sleep, imagine a bigger nightmare: rogue networks. People often think nothing of bringing in insecure devices and logging on to the corporate LAN, says Evans. He cites one instance at a major financial institution in New York where an employee brought in a Wi-Fi enabled laptop. He then began broadcasting an unencrypted, ad hoc wireless network with the name "Apartment" across parts of lower Manhattan, inadvertently connecting to another network and opening an unsecured bridge into the financial institution.

"You cannot predict where wireless is going to be," he says, which is why ISS recommends performing periodic vulnerability scans of clients' offices for unauthorized hardware, including Wi-Fi devices.

He says enterprises may move to a security-on-demand model, where the network automatically scans your device and, if it determines that it's insecure, takes appropriate corrective actions, such as downloading an agent to secure the device for however long you need to log on.

"At a high level you have policy and technology measures that govern what people can and can't do with their machines," says CTG's Moyle. "But in any organization of any size, there will always be nooks and crannies where it's hard to find out what's really going on."

The Compliance Secret Hiding in the Closet

When hackers attack your VOIP system, when employees take sensitive data home on thumb drives, when configuration errors or rogue software takes down your network, it's not just an IT disaster, it's increasingly a compliance problem. And when organizations ignore this reality, it can easily put them in Dutch with state and federal laws.

"Under (California law) SB 1386, people know if a laptop with personally identifiable information on it gets stolen they must disclose that," Moyle says. "But they don't understand that if you put the same data on a thumb drive and bring it home with you, and your home machine has been compromised by spyware, you're still required to disclose that the data has been compromised. They don't know they're out of compliance. It's a huge problem."

But keeping up with the reporting requirements of laws such as SB 1386, HIPAA, Sarbanes Oxley, the Gramm-Leach-Bliley Act, and all the rest too often becomes a primary responsibility of IT pros who already have full-time jobs. Combine that with poorly understood requirements and poorly defined IT controls, and you have a recipe for regulatory disaster.

Little wonder then that IT firms are struggling more with their SOX audits this year, says Wynn White, Oracle's senior director of security and identity management.

"One dirty little secret of compliance is that the bar keeps getting raised, and what met the requirements a year ago isn't working this year," White notes. "I've spoken with a number of customers who failed this year's audits even though they passed the year before."

Ed Hill, managing director of IT audits at Protiviti, a risk management consultancy, says the most likely reason is IT orgs didn't correct problems noted on last year's audits.

"If you have a deficiency one year that's not deemed 'significant,' and you don't do anything to alleviate it, the next year it almost always becomes significant because it's a repeat finding," Hill notes.

White says there's no simple solution, but he has hope. "It's been ugly for the last couple of years, but our customers understand they need to take a number of steps to become compliant, and that no single solution will do it for them."

In organizations that lack a formal compliance team, dealing with compliance issues saps IT resources that could be used to build the business, says CTG's Moyle. "They still need to build that customer tracking application they promised, but now they have fewer resources to do it."