CIO: It's obvious that a vendor trying to sell you a software product might take advantage of the input bias. But are employees really that manipulative, or is it simply an ingrained part of corporate culture?

Schweitzer: Sure, organizational culture is part of it. But people really are that manipulative. Think about the concept of people overstating expense reports. Does that really happen? Yes. And it's even more subtle than that; it's about creating impressions. I have students who come up to me and say they spent five hours in the library working on a project, or they studied for a week for my exam. Those measures shouldn't matter. I'm interested in how well you did on the test, not how many hours you spent staring at the book. But people will convey that information nonetheless. And even if you know it doesn't matter, it still has an influence on your judgment of the outcome.

CIO: What are some instances in which CIOs would likely rely on irrelevant input information?

Schweitzer: In one of our experiments, we submitted two presentations on emerging technologies to university students. They saw two videotapes of someone describing a new technology they knew nothing about. With each presentation, we told half of the group that the presenter had spent a long time preparing the presentation, and the other half that he had spent a shorter time preparing. After viewing the tape, they judged the quality of the presentations along several different dimensions, and with every one, they gave higher marks when we told them the presenter had put in more time. What was surprising was that even when they indicated that they knew the preparation time didn't really matter, they still were influenced by it.

People's judgments often depend on how easy it is to evaluate something. If it's easy to measure the outcome, we may not rely on input measures. If someone is performing in the Olympics, their time is a clear measure of how they did. But in other cases, such as judging how innovative a pharmaceutical company is, it's harder to reach a decision. So you might rely on whatever objective measures are available, such as how many patents the company has. But in reality, that's not a good measure because more patents are filed for small modifications of existing compounds, and there aren't really that many big blockbuster drugs involved.

CIO: OK, so how are CIOs influenced by input bias?

Schweitzer: A CIO may need to judge the innovativeness of a technology company. They might rely on quantitative input information such as R&D expenditures. But that may not be the best indication of innovation.

In judging the quality of a software package, they might look at the number of lines of code in a program. But that's not really what they should care about - it's whether the program is effective. If they could measure speed at which a program could complete a task, that would be a better measure for them. But if they are comparing packages that perform different kinds of tasks, that's hard to do, so they rely on things they can actually measure.

Similarly, when judging employee performance or interviewing job candidates, CIOs may rely on whatever objective measures are available, relevant or not: how much face time employees put in, how many hours they spent on a project. But that's not really what CIOs should care about.

CIO: Are you saying that even when faced with a really rotten employee or a project that's tanking, CIOs can be fooled into thinking the outcome is better than it actually is based on input like face time or the number of people working on a project?

Schweitzer: No. If something is really bad, input bias isn't going to work. In one experiment, we offered people two samples of iced tea. In one instance, we asked people to compare normal raspberry and lemon iced teas. We told them one was made with an expensive machine and one was made with an inexpensive machine. We got the same results we did with other experiments: Input bias mattered, and most participants rated the tea made with the expensive machine a better tea. We conducted the same experiment with the same teas, but we added lime juice and salt to them. They were really terrible. For the bad teas, they didn't care about the expense of the equipment used to make them. They were just bad.

People tend to think more critically when they experience a low-quality outcome. It shocks them into being much more careful and calculative. When a CIO encounters a bad outcome, he will go off automatic pilot and make judgments based on quality. If you have an employee who's just a disaster - even if there's high input - it's not going to help. If there's a software product that keeps crashing, you can't sell it based on input measures.

CIO: You say the more accountable people are for their decisions, the more likely they are to rely on irrelevant input. Please elaborate.

Schweitzer: It comes down to the issue of justifiability. If you have to justify your decisions to someone else, tell someone why they were passed up for a promotion or explain to your board why it should invest in a certain system, you're more likely to rely on input measures because they give you some justification for your decisions.