Real-Time Data: Too Much of a Good Thing?

by Barbara Depompa Reimers

The need to speed up business decision-making to keep from falling behind the competition is driving companies to use real-time analytical tools. With them, they hope to more quickly exploit key corporate transaction data housed in databases, enterprise data warehouses and other data stores. But the move to instant analytic insights comes with big trade-offs and incremental costs, say analysts and users.

Glib executive orders to provide real-time data feeds everywhere in the company can be counterproductive if the data quality is low or the company doesn't have the processes in place to actually analyse and act on the data. Besides, real-time data analysis may be a high-payoff pursuit in only a few mission-critical areas of the company.

Those negatives don't appear to be quelling interest, however. In a December 2002 survey of 700 IT executives by US-based Evans Data Corporation, 48 per cent of respondents said they were already analysing data in or near real time, and another 25 per cent reported plans to add real-time analytics this year.

A key issue is how the term real-time analytics is defined, because there's confusion about what constitutes real time versus near-real time versus not real time. Joe McKendrick, an analyst at Evans Data, says a strict definition of real-time analytics is dynamic analysis and reporting based on data entered into an operational system less than one minute earlier. At most businesses, however, analytics is considered real time if it's conducted on data collected within the past hour, and near-real time is analytics conducted on data collected within the past 24 hours, McKendrick says.

Absorbing the Data McKendrick says the growing interest in real-time analytics can be attributed to the pressures on businesses to make faster decisions, keep smaller inventories, operate more nimbly and track performance more carefully.

Trouble is, there may be no reason to use real-time analytics if a company can absorb its transaction information only on an hourly, daily or weekly basis. For example, at New York-based MBIA Insurance Corporation, overnight updates work best to feed the OLAP engine used to help the company decipher risk related to mortgages, insurance policies and other services.

"What we've found is that overnight updates are plenty, as long as users can gain immediate access to the information they need to make business decisions quickly during the day," says Lynn Jacobs, associate vice president for IT applications at MBIA. Using the Essbase OLAP tool from Hyperion Solutions, business users at the insurer can quickly analyse financial information in order to assess risks and make fast business decisions. The number of financial analysis applications supported by the Essbase tool will be expanded this year from the current 10.

Too Many Iterations In many traditional business environments, real-time analytics can be overkill. Barton Goldenberg, president of CRM consultancy ISM Incorporated, offers the following example: Three managers meet to discuss business development issues, each armed with sales, market or target information they captured from the system at different times of the day. Each manager has created a snapshot to present an accurate picture, but the numbers won't match. "This situation will hardly help speed decision-making," Goldenberg says.

In other cases, real-time analytics may make it easier to make bad decisions fast, because of faulty data.

Managers at Tennessee-based Forward Air Corporation now can get their hands on real-time information via the company's intranet. Using specialised reporting and analytical tools from Appfluent Technology, they can track things like the amount of freight shipped by various customers. The next goal, says Glenn Adelaar, vice president and CIO at the trucking company that serves the airfreight industry, is to enable department managers to analyse more of the transaction data that's accessible via the real-time analytics tool.

But a manager who wants to track his salespeople's daily contribution will find that timing is a big issue. Right now, about 90 per cent of sales transactions are entered within 20 minutes of completion, Adelaar says. "But that 10 per cent will get some poor salesperson clobbered," he says. Adelaar says that although it's critical to get accurate data to business managers quickly, the business processes that support those data feeds must be made bulletproof to attain high-quality insights.