Old concepts and new technologies have recently converged to generate a new strategy to improve IT responsiveness while reducing integration costs. It's the integration layer, and it may put an end to all those complaints about IT's slowness and inflexibility.

Reader ROI

• How messaging and services work together
• How to get developers to write service components
• How a SOA can create IT-business alignment

For years, whenever Wisconsin state CIO Matt Miszewski tried to discuss systems integration with agency heads, he sensed a fog settling over the room. "You could see it in their eyes," he says. "They tuned out."

Worse, when the fog cleared, it was replaced with anger. Difficulty with integration, Miszewski could tell they were thinking, was just another IT excuse for slowness, inflexibility and inability to give them what they wanted.

And Miszewski couldn't blame them. Technical limitations and time pressures often made integration a haphazard affair. Rushing to meet deadlines, developers cobbled together direct links (point-to-point integration) to share data and business logic among applications.

While that kind of integration is quick and relatively simple, over time it has crippled the health and flexibility of most IT architectures by creating a cobweb of hundreds, even thousands, of brittle linkages that have to be torn apart and reassembled every time one of the applications changes.

All those links, built up over decades, have created a crisis that goes far beyond IT. The rise of the Internet has made businesses completely dependent on IT to add new capabilities. The foundations of those new capabilities often lie buried inside old systems that were never designed to communicate with one another, let alone over a global network. Adapting those systems to communicate - that is, systems integration - can take so long that entire generations of business opportunities (new products, alliances, mergers and acquisitions) can grow old while IT fiddles with the wiring. Unable to explain the complexity of their problem, CIOs inevitably wind up on the defensive, with only the most patient, tech-savvy CEOs able to commiserate.

But markets don't commiserate with anyone. Markets see only one speed: fast. Aesop didn't spend any time on Wall Street. On the Street, the tortoise almost always loses.

The urgency and intractability of this problem is underlined in CIO's "State of the CIO" survey: Integration has been the number-one technology concern of CIOs since we began doing the survey in 2002. (CEOs also put it at the top of their list when we began surveying them in 2004.) CEOs' three primary complaints about IT, according to survey after survey - that it is too expensive, too slow and too inflexible - all lead back to integration.

In short, the business is tired of waiting for IT to catch up with its demands.

And so are CIOs.

How the Integration Layer Can Untangle the Enterprise

All this is hardly news. CIOs have been painfully aware of integration's costs for years. What they've lacked is a unified strategy for dealing with those costs. But recently some old concepts and some new technologies have converged to generate a new, winning strategy that promises to blow away the cobwebs and radically improve IT's responsiveness (while also reducing integration costs). It is the integration layer, a virtual stratum in the architecture that is composed of two major pieces: messaging and services.

The foundational piece - known as the messaging infrastructure - is like a good executive assistant, translating, routing and monitoring information from different systems without these systems needing to connect directly. Adding, changing or removing a system becomes a matter of modifying a single link, rather than ripping apart connections to all the different systems it may need to communicate with.

But while the messaging infrastructure makes connecting systems easier, it doesn't free business processes from their mainframe prisons, or eliminate redundancies in applications, or provide any impetus to create a useful architecture. Indeed, a good messaging infrastructure can perpetuate the chaos by making it easier to deal with.

Of course, not every company has redundant systems and processes. For them, messaging may be enough. But for most companies - especially big ones with dozens of different ERP systems all doing more or less the same things - messaging alone won't create an effective integration layer.

Messaging is an IT solution, not a business solution. It has long lacked a higher purpose, a strategy. Service objects (or just plain "services") is that strategy, and it is the second core piece of the integration layer. The idea behind services is simple: Technology should be expressed as a unit of business work - like "get credit", or "find customer record" - rather than as an arcane application such as ERP or CRM. This is an old concept, based on object-oriented programming from the 80s. Services extract pieces of data and business logic from systems and databases around the company and bundle them together into chunks that are expressed in business terms.