1.Get VM Sprawl Under Control

CIOs such as Michael Abbene, who runs IT for Arch Coal, understand the problem of VM sprawl full well: VMs take minutes to create. They're great for isolating certain computing jobs. But the more VMs you have, the more security risk you have. And you'd better be able to keep track of all those VMs.

"We started by virtualizing very low-profile test and development boxes," Abbene says. "Then we moved some low-profile application servers. We've been moving up as we've been successful. We understand we're increasing our risk profile as we do that." The company currently has about 45 production VMs, he notes, including Active Directory servers, and some application and Web servers.

How do you control server sprawl? One approach: Make creating virtualized servers and VMs as disciplined as creating physical ones. At Arch Coal, the IT team is rigorous about allowing new VMs: "People have to go through the same process to get a server, whether it's physical or virtual," says Tom Carter, Arch Coal's Microsoft Systems Administrator, who works for Abbene.

For this purpose, Arch Coal IT uses a change control board (made up of a cross-section of IT staffers from disciplines like servers and storage, serving on a rotating basis) to say yes or no to new virtualized server requests. This means, for example, that people in the applications group can't just build a VMware server and start creating VMs, Abbene says - though he's had developers ask to do just that.

VMware's VirtualCenter management tools as well as tools from Vizioncore can also help manage VM sprawl.

Ignore VM sprawl at your own peril, says IDC's Elliott. "VM sprawl is a huge problem, causing lag times in the ability to manage, maintain performance and provision," he says. Also, unexpected management costs will arise if your number of VMs gets out of hand, he adds.

2.Apply Your Existing Processes to the Virtual Machines

Perhaps the sexiest aspect of virtualization is its speed: You can create VMs in minutes, move them around easily, and deliver new computing power to the business side in a day instead of weeks. It's fun to drive fast. But slow down long enough to think about making virtualization part of your existing IT processes, and you will prevent security problems in the first place, says IDC's Elliott. You will also save some management headaches later.

"Process is important," he says. "Think about virtualization not just from a technology standpoint but from a process one." If you're using ITIL to guide your IT processes, for example, think about how virtualization fits into that process framework, Elliott advises. If you're using other IT best practices, look at how virtualization fits into those processes.

One example: "If you have a server-hardening document (prescribing a standard set of security and setup rules for a new server)," Hoff says, "you should do the same set of things to a virtual server as to a physical one."

At Arch Coal, Abbene's IT team does just that. "We take our best practices for securing a physical server and apply those to every VM on the box," Abbene says. Steps like hardening the OS, running antivirus on every VM and ensuring patch management keep those virtual boxes in tune with the same procedures used on physical ones, he says.

3.Start with Your Existing Security Tools, but Be Critical

Do you need a whole new suite of security and management tools for your virtualized environment? No. Starting with your existing set of security tools for the physical server and network world and applying them to the virtual environment makes sense, says Hoff. But do press your vendors to tell you how they're keeping up with virtualization risks, and how they'll integrate with other products going forward.

"There's a false sense of security in relation to adopting physical tools for the virtual environment," IDC's Elliott says. At the same time, he adds, "it's very early in the market", for new security tools designed with virtualization in mind. That means you must press your legacy and potential start-up vendors a little harder than usual.

"Don't assume the platform-level tools (such as VMware's tools) are good enough for you," Elliott says. "Look at the start-ups and the legacy management vendors. Press those legacy vendors to do more, and provide guidance for them."

Jim DiMarzio, CIO at Mazda North America, follows this strategy in his enterprise. Like Arch Coal, Mazda NA runs VMware's ESX Server 3 software at the core of its virtualized servers and has been ramping up its number of VMs recently. DiMarzio says he expects to have about 150 production VMs running by March 2008. He's using the virtualized servers for Active Directory servers, print servers, CRM application servers and Web servers - the last being a mission-critical app since Mazda uses these Web apps to serve information to all its dealers, DiMarzio says.

To secure these VMs, DiMarzio decided to continue with his existing firewall and security products, including IBM'sTivoli Access Manager, Cisco firewall tools, and Symantec's IDS monitoring tools.

At Arch Coal, Abbene and his team are sticking with the security tools they're already using, while also investigating tools from start-ups BlueLane and Reflex Security. "The [legacy] security and change vendors are trying to work hard to catch up and they're behind," Abbene says.

BlueLane's VirtualShield product for VMware, for instance, claims that it can protect virtual machines even in cases where certain patches are out of date, as well as automatically scanning for possible problems, updating problem areas, and protecting against some remote threats.

Reflex Security's Virtual Security Appliance (VSA), which Hoff describes along with BlueLane's software as one of the few emerging products worth attention right now, essentially serves a virtual intrusion detection system (IDS), adding a layer of security policies inside the physical boxes where the VMs live. It could help block a hypervisor attack, among other possible future troubles, Abbene's team figures.

Abbene says his IT group has also discussed adding a second internal firewall to further isolate the VMs, but he's concerned there might be a performance impact on the virtualized applications.

IDC's Elliott cites a few other virtualization security tools worth examining: PlateSpin, known for physical-to-virtual workload conversion tools and workload management tools; Vizioncore, known for file-level backup tools; Akorri, known for performance management and workload balancing tools; and storage firm EqualLogic, recently acquired by Dell and known for iSCSI storage-area network (SAN) products optimized for virtualization.