Testing and debugging services

Another underappreciated aspect to SOA deployments involves testing and debugging. "In a lot of ways, SOA done properly gives you faster time to market," ADP's Bongiorno says. "But you give some of that back in the testing."

Although the use of stringently defined service interfaces can ease integration testing across pairs of services, the many-to-many nature of service interaction and the variety of hardware and software systems that provision them make testing difficult. "You can't get your whole enterprise into a QA lab," says eBay's Barrese, so you have to scale your test platform as much as the business case warrants.

eBay has built some of its own QA tools for automated regression testing to help test the many execution scenarios inherent to SOA but uses off-the-shelf tools such as the those produced by Mercury Interactive. (ADP also uses automated regression tools from Mercury.) In addition, eBay is evaluating the open source Apache Axis service-testing tools with its BEA and IBM platforms.

Financial transaction processor SunGard requires that test cases for all services be built up front, to ensure that all interactions and requirements are thought-out, says Nils Winkler, technical architect at SunGard. Unit tests are built from these test cases, so they're available for use in automated regression testing by all service developers whose processes might use the specific service. "You can do the testing with the data you already have," he says.

A related challenge is version control. As you have more and more services, you have to expect that you'll have to support multiple versions because you can't update all your services at once. A registry or repository can maintain version information as part of your services' standard attributes. This safeguard is important, so other services can adjust their expectations accordingly, Bongiorno says.

The best way to manage version control is to design for it in the first place, The Hartford's Moreland says. "Go in with the expectation that you will retire it," he says, such as by setting -- and enforcing -- end-of-life periods for all services. And while you have multiple versions of a service running during the necessary transition time, use translation services wherever possible to map calls to the old service instead of the new one, filling in any new data that can be derived or assumed in the translation.

No amount of testing will catch every error, of course. Transactional errors can be caught as part of the monitoring process. But to identify logical errors in the services themselves at run time, the calling services must look at the returned message to see whether there is a mismatch in format, policy, or other expected attribute, based on the business process specifications, Moreland says.