Back in the 1960s and 1970s the business requirements were done by "analysts" who were from the business. There was no "IT industry" so everyone in IT came from the business. These ex-business staff were trained in "systems analysis techniques" and then expected to define the business's systems needs.

Then, in the 1980s, the first group of "IT professionals" emerged from university having been trained in systems analysis techniques. They said: "We're the (IT) industry experts, we'll take over now." And so they applied their analysis techniques to the real world to define the systems needs.

Partly because they lacked the business/industry knowledge, the results were not great. Various alternative ways of defining requirements were progressively developed — JAD, RAD, Prototyping, etc. — and, more recently, the "vanilla" adoption of the packaged software's design.

What all of these approaches seem to have in common is a high level of dissatisfaction with the eventual business outcomes.

The inability to define good business needs/requirements has also led to other project delivery problems — such as the desire to narrow the scope of the project so as to minimize the requirements workload and risk, the desire to adopt "vanilla" systems regardless of whether they are suitable or not — both of which reduce the project's value and business's satisfaction with the results.

Another change from the 1960/70s is the additional complexity now being attempted with systems. This is not an excuse for failure but a reason why it is so critical to get this process right.

Most business requirements approaches fail on two dimensions.

The first dimension is a lack of a true process orientation. When you're building a system you may need to know the features and functions required. But the business does not work through features and functions, but through processes. Business requirements need to be defined in business process terms — how the organization wants to do business, compete and make money in the future.

This needs to include the process and information flows (so you can see where the system can cost-effectively automate the process) as well as the organizational change requirements (so you can prepare the organization to adopt and use the new system) and the desired business outcomes (the business end states to be achieved).

So your requirements generation process needs to deliver the systems and organizational change requirements in process terms and the business measures of success (in measurable end states terms). Most requirements processes fail on at least two of these dimensions. (While there is a lot of focus on processes these days, most is misfocused. Approaches focus on, for example, getting the symbols of the flow maps right or the data for Six Sigma outputs. Process requirements need to focus on the value of each and every step in the process, the information needs of these steps and the resultant business changes required to make this new way of working happen.)

The second dimension missing in requirements definition is an understanding of the neuroscience involved in requirements definition.

Being asked "What do you want from a system?" also only taps into people's conscious brain (often called "top of mind"). This is only one-sixth of the brain and even less of people's total knowledge. All real knowledge is in their non-conscious.

As one Chairperson said to me, "Being asked what you want out of a system is like being asked what you want out of life! Where do you start? What are the parameters?"

If you rely on people's conscious "requirements" you'll fail. You need to tap into people's non-conscious to define their real requirements. People are conscious of the immediate problems and issues with their processes and so will seek to have these resolved. However, few are conscious of the strategic intent and rationale of their processes — what they are really trying to achieve and the reason (or lack of reason) for each step in the process. This knowledge is held in their non-conscious and has to be tapped into to identify what they really want the process to do, and why.

Tapping into their non-conscious requires their intimate involvement in the requirements definition process — documenting the existing and new processes down to the operational problem level. It requires running workshops where people write down their thoughts before any brainstorming so as to tap into their inner knowledge before their thoughts are channeled by the loudest workshop participants.

(If you look at the current 'agile' iterative techniques they are really trying to get to the non-conscious requirements through repetition. In the 1960/70s, some of this non-conscious business knowledge was already in the analysts' head as they had come from the business.)

Project manager take away

If you don't get the business requirements right, however well you deliver the project, the client/business will be dissatisfied. (And, its no good blaming them for giving you the wrong requirements, you may be right, but you'll never win the argument.)

If you're leading the requirements step, look at the requirements definition process to be used to see how it gets behind the obvious requirements and delves into the strategic intent, rationale and value for each process step.

If you're inheriting the requirements, look to see if the future processes are about half the complexity of the current processes, that the current state has not just been automated and that what makes this organization different has not been lost in the process.

And remember you want requirements for both systems and organizational change — the same requirements step should produce both simultaneously. This has the advantage that you can often make some of the organizational changes early to deliver immediate benefits, long before any system changes are implemented.

To read part one of Jed's first column, Why Projects Fail: Part One, Wrong Scope, click here

Jed Simms is CIO magazine's weekly project management columnist. Simms, founder of projects and benefits delivery research firm Capability Management, is also the developer of specialized project management and project governance Web site www.project-sponsor.com