At MarkMail, a free service for searching public mailing list archives, "Our team is using virtualization in a number of ways," reports John D. Mitchell, developer (his official title is "Mad Scientist") at MarkMail. "The key is that virtualizing each and every service in the system gives us a lot of flexibility. In development, we aren't stressing the physical resources much so we have lots of virtual instances running on just a few hardware nodes, including the database."
MarkMail is using OpenVz (http://wiki.openvz.org) so the images in the developer sandboxes are identical to the deployment environment. "Virtualization makes it very to test different setups, versions, etc.--we can always just blow away an instance and reinstantiate it from a clean checkpoint," Mitchell says. Using virtualization reduces the amount of hardware they need to buy/lease and manage, according to Mitchell. "In total, we're running about a 3 to 1 ratio of virtual instances to hardware nodes."
The Limits of Virtualization
There are, developers acknowledge, part of the process where virtualization can't help.
"One area where virtualization doesn't help is in stress testing our product, which is a network-based backup appliance," notes 3X's Floehr. "While we could test 10 simultaneous clients virtually, it isn't a true stress test because the 10 clients are only going through, say, four physical CPUs and two NICs, which are constraints that would not exist if they were 10 physical clients in the real world."
"Virtualization does add some complexity," adds MarkMail's Mitchell. "It always adds some latency. But, even worse, you can't rely on VMs to give a clear view of the performance characteristics as you change the underlying execution environment, such as going from development and testing into production."
Finally, Mitchell notes that "The current generation of desktop virtualizers still have too many annoying quirks, bugs and missing features," at least with regard to their use by developers for software development.
"For load testing or scaling, you have to run on the raw hardware, the physical machine," says Microsoft's Mark Friedman. "Performance characteristics, unless you expect to deploy on a VM, are very different. And the same applies to the state of the art for performance measurements; you can't trust the [Physical Machines] you get on a VM currently." Although, Friedman notes, over time he expects this to be fixed.
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.