The CIO's Reliance on IT Industry Analysts

Executive Summary

US CIO and Darwin Magazines conducted a survey to gage CIOs' use of IT analyst firms' services to make IT purchase decisions. More than half of the executives we spoke with said that IT market research is important to overall IT strategy. Research is used most commonly to educate on a technology or an IT issue. Likewise, education was most frequently cited as the most valuable use of IT market research.

The vast majority (84%) of respondents feel the information they receive from IT industry analysts is reliable. Additionally, the majority of respondents feel that the information is objective. However, they expressed concerns that the same companies that they report on are also clients, which leads CIO to believe that IT professionals may be unaware that most analyst firms have a mix of both vendor and users in their client base.


CIO Magazine and Darwin Magazine conducted a telephone survey of 200 CIOs on their reliance on IT analyst market research in making IT decisions. We defined IT analyst firms companies that are not allied with a financial services company and that exclusively track information technology related products and services and offer written reports and consultative services on a subscription basis. Respondents were randomly selected from CIO's circulation file. Surveys were conducted between November 15 and November 28, 2000.

In terms of company size, 24% of respondents worked in companies with less than 1,000 employees, 30% had between 1,000 and 4,999 employees. 9% said their company employed between 5,000 and 9,999 employees while 38% worked in organizations with 10,000 employees or more. (3% did not answer or did not know.) When asked to estimate the company's gross sales or revenue for 2000, 18% of respondents estimated their company's revenue at less than US$50 million. 28% said their company's revenue would be between US$50 and US$100 million. 27% estimated their annual revenue to be greater than US$1 billion. 7% of the companies surveyed were non-profit. 23% either did not know or did not answer.

Key Findings

Use and Value of IT Analyst Market Research53% of our survey base believe that IT analyst market research is somewhat important to their overall IT strategy. 33% indicated that it was very important or extremely important while 14% felt it was not at all important to their IT strategy. (1% said don't know or refused to answer.) The most common use of IT market research among the IT executives we surveyed was to educate themselves or their staff on a technology or topic (67%). 55% of respondents indicated that they used market research to make a technology purchase decision and 42% used IT market research to compare their company's metrics to industry averages. 40% relied on IT market research to make a vendor decision while 29% used it to make a technology integrator decision. 27% said they relied on IT market research to determine the strategic direction of their company's IT. On average, companies included in our survey spent US$552,000 on IT market research in 2000.

When asked which areas market research firms provided the most value, IT executives listed educating themselves or their staff on a topic (41%) most frequently, followed by making a technology purchase decision (22%), determining strategic direction of IT (21%), comparing company metrics to the industry average (16%) and making a vendor decision (14%).

Reliance for decisions/specific decisions/level of satisfaction35%, or 69 out of 200, respondents said they had relied on IT analyst market research for a major IT purchase decision. The following chart is based on these 69 responses:

Decision % That said they were Satisfied Number of Respondents Software selection/purchase 93% 56 IT Services 92% 53 RFP Preparation and discovery 95% 35 Strategic consulting 94% 50 Contract negotiation 98% 22 IT consulting firm selection 95% 28 Accountability/objectivity/concern about conflicts of interestWhen asked whether or not analyst firms have adequate accountability to their organisation if their recommendations are incorrect, 56% of our respondents said no. 33% felt there was adequate accountability. (11% said don't know or did not answer.) Recommendations to improve accountability most frequently included tying compensation to accuracy and results and writing contingencies into contracts. (See verbatim comment for question 4.) Our survey asked IT executives to rate the reliability of the information they receive from IT analysts, using a scale of 1 to 7. (7indicated completely reliable and 1 indicated not at all reliable.) 56% of respondents indicated that the information was reliable or fairly reliable (answered 4 or 5). 28% said the information was very or completely reliable (answered 6 or 7) while 11% indicated they were not reliable.

Our survey asked respondents to rate the objectivity of information they receive from IT industry analysts on a scale of 1 to 7, where 1 meant not at all objective and 7 meant completely objective. In response, 28% of respondents felt the information they receive from IT analysts was very or completely objective (answered 6 or 7) while 15% said the information was not objective (answered 1,2 or 3). 53% said the information was objective or fairly objective (answered 4 or 5).

When asked how concerned they were that the vendors covered by analysts were also the analysts' clients, 40% said they were very to extremely concerned (answered 6 or 7) and an additional 40% were concerned or somewhat concerned (answered 4 or 5). 16% (answered 1, 2,or 3) said they weren't very concerned or not at all concerned.