The CIO's Essential Role in Experimentation

As the CIO, you play a key role in making business experimentation a core competency in your company. The discipline of experimentation, the collection and analysis of data, the presentation of that analysis to the business in a way that encourages learning, the storage of and access to past experimentation data in a way that's useful for future experiments, and the systems that present an experiment's results for optimal learning - all these are tasks performed by information systems. You should review your IT architecture to ensure that your standards and policies enable rapid and cost-effective experiments.

But supporting business experimentation is only part of your role. CIOs are well-positioned to champion experimentation within the enterprise. You can do this by making experimentation a part of IT operations strategy.

Train your staff in the process and discipline of business experimentation. Develop skills in your staff that will enable them to identify, build, run and analyze experiments. Organize for rapid experimentation by examining and redesigning routines, organizational boundaries and incentives. Train small groups of key people to iterate rapidly and learn from each iteration. Above all, make clear that IT employees need to focus on organizational learning.

Experimental Trials

To some executives, business experimentation may feel like they're giving up control of a key part of strategy formulation. After all, if they let any part of the enterprise try out new ideas at any time, it might be difficult to rein it back in when the time comes to formulate a single business strategy. In the current business climate, however, this way of thinking is wrong. Organic growth has become an important criterion for market valuation, and the rate of innovation is a key input for the rate of organic growth in large companies. Innovation from anywhere in the company is a good thing.

From the research we have conducted with our partners at The Concours Group consultancy, we have found that the greatest benefit from business experimentation occurs when an organization is able to integrate the approach into its everyday processes. The success of experimentation depends on an organizational culture that accepts learning as a goal.

It's not an easy feat to create an environment that walks the line between so-called failed experiments - where the discipline of data collection, analysis and iteration results in learning even if the experiment itself doesn't produce a desired result - and the frivolous waste of resources, where ideas are tested in an undisciplined manner. Using an experiment as a justification for a one-off project doesn't create a culture of experimentation. That kind of approach to experiments signals that managers have to "get it right the first time", and that failure is not acceptable.

The challenge for senior managers, then, is to establish an environment in which experimentation can flourish, while at the same time building processes and controls to ensure that resources are not squandered. This is certainly a complex challenge - which is why we believe it will partially distinguish between business's winners and losers in the future. Business experimentation might be the only process that is designed to continually produce the organic growth and operational efficiency your company needs to remain competitive.

James Cash is the emeritus James E Robison Professor of Business Administration at Harvard Business School. Keri Pearlson is a research director with The Concours Group and co-author of Managing and Using Information Systems