E-Commerce Missing Link: Software Requirement Specifications

Why do almost 80 percent of e-commerce and other software development projects undertaken in the U.S. fail to be completed on time, or within budget, or to function as anticipated? What is the biggest cause of these failures?

Most problems stem from an unwillingness or inability to develop adequate software specifications at the beginning of an e-commerce or other software development project. Instead, requirements are often determined informally and added to a project in process in an uncontrolled and ad hoc manner. Poor understanding of and minimal attention to software requirement specifications constitutes the missing link in the e-commerce and software world.

Attention to software requirements is analogous to the situation that all of us face in responding to tooth decay. With dental care you can either choose to have a little pain now — or wait and encounter a lot more pain later. The longer people ignore the importance of software requirements, the more expensive requirements deficiencies become.

Software requirements are detailed descriptions of how an e-commerce site or other software system is supposed to function. In the requirements development phase of a project, user needs are explored and determinations made regarding exactly what software should be created.

Mistakes in software requirement specifications can be a hundred times more costly to correct once a site goes live or the shrinkwrap comes off than if defects are corrected in the beginning of a project. Fixing or rebuilding expenses can easily consume 30 to 50 percent of development costs in poorly planned projects.

Setting the Wrong Benchmark

Inexperienced managers who eschew normal software development processes are a big part of the problem. Too many managers define success for themselves in terms of how soon they start developers writing code.

There is a management tendency to use whatever development personnel are available at the time a project is set to begin, rather than paying more money to attract better developers or waiting until better personnel can be freed from other projects.

Managers with little or no experience in supervising successful e-commerce or other software development projects often object to suggestions that their first supervisory effort be directed towards a small and simple project, rather than a complex one with little or no chance of success under their leadership. Another common error is for inexperienced, untrained managers to resist delegating authority or adopting a team approach.

Dot-com Boom Encouraged Bad Practices

The dot-com boom provided huge amounts of cash for e-commerce projects that had not been fully thought out, that promised too much in terms of features, and that were being overseen by people whose understanding of software development processes was clouded by ignorance, self-delusion and hubris.

In Seattle, the logic of many venture capitalists (VCs) was that they only needed one out of every 10 of their projects to succeed as planned. If one project exceeded expectations, then it could cover the losses from the other projects that failed.

The venture capital community in Seattle had (and continues to have) expectations that fuel self-destructive e-commerce project development efforts. VCs’ expectations for high profit rates force e-commerce project planners into lying about the profit potential for new ventures being considered for funding. Numbers are pulled out of the air in order to satisfy VCs’ prerequisites. Expectations for immediate revenue generation following VC funding force new enterprises to skip over time-intensive activities that are central to successfully developing software requirements specifications.

Less greed and more realism on behalf of VCs would allow new ventures to properly complete their initial planning and software requirements specification activities, thereby raising the chances that such ventures would eventually succeed. But VCs’ clarion calls for revenues in the second quarter and profitability after the first year have led to disaster after disaster. Few VCs or old-industry managers seem to be able to learn from their mistakes and to change their approaches accordingly.

VCs Rebound, Mistakes Continue

The VC environment is rebounding from the doldrums of 2001. Now the emphasis is on funding ventures that have received modest funding amounts from a small circle of angel investors. Anything more than eight angel investors makes a new enterprise less attractive to VCs, who will expect angels to step aside.

The credibility of the VC outfits is undergoing a makeover, as many of them seek to sweep previous mistakes under the rug. Otherwise there would be questions such as: “You funded an online party-planning portal and now you want me to take direction from you?”

Today, bad software development practices are most commonly seen at internally funded projects or projects supported by the venture arms of established IT firms. It is not uncommon to find new e-commerce efforts being launched without formal specifications having been adequately developed first. Instead, many projects function with requirements being added orally after substantial development efforts have already occurred.

Adding requirements to a software project at any time increases the costs of that project. Without undergoing a rational requirements-development process at the beginning of a project, managers can quickly lose control of a project’s costs. Designing an optimal architecture also becomes impossible. Ironically, many managers’ attempts to control project costs by initially skipping over most of the requirements-development process usually has the opposite effect by increasing overall project costs and the time needed for completion.

The concept of data dictionaries is an idea foreign to many project managers and clients. Data dictionaries define data elements, data attributes, and how data is structured. Data dictionaries should be developed during the requirements specification phase of a project.

Business logic should also be defined during the requirements specification phase. Under the ad hoc or minimal-effort approach to software requirements development, business logic is often mixed in with the logic for how data is stored or presented. In poorly managed projects, business logic might also be buried within other parts of the larger code base.

In poorly managed projects, changes in business logic might be difficult to achieve without major work arounds. Such difficulties are also encountered in many attempts to integrate or update old legacy systems, where the original developers and project managers are gone, there is little or no documentation available, and work arounds have been heaped onto older work arounds until the whole system is a mess.

Except in the tiniest projects, the ad hoc or minimal-effort approach to software requirements development often creates poorly documented, unsupportable software that displays the worst aspects of legacy systems the minute it is released.

The uncontrolled addition of major new requirements late in a software development process encourages workarounds and other high-risk approaches to software development. Poorly documented workarounds are a major challenge in attempts to update old legacy systems and to integrate old systems into modern e-commerce applications.

The ad hoc approach to setting software requirements makes those requirements difficult to prioritize and monitor. Accountability suffers and the finished products are often hard to maintain and support.

Change Control

The standard best-practice process for software development is to “baseline” a set of software requirements once they have been agreed upon and before any code is written. A group of people is usually given responsibility for deciding whether proposed changes to those baseline requirements (and resulting changes in the project’s budget, workplan, and timeline) should be allowed or rejected. The group of people who approve and deny changes to the baseline might be called the change control board.

Projects often use change control plans to formalize the process of considering changes to baseline specifications. Change control is commonly extended to include system architecture, product design and other work products.

One person in a project team is often designated to take the lead in identifying, developing and validating draft specifications. This person might be called a requirements analyst, systems analyst or business analyst. Once a baseline has been set, the requirements analyst might then be assigned to track the status of each requirement, manage changes in requirements, and coordinate interactions between those requirements and other projects and work products.

Reform Efforts

Many U.S. business schools are now doing a good job at teaching students about the importance of IT strategy and about integrating information technology throughout the enterprise. What is lacking is instruction in the practical aspects of implementing IT projects, especially in terms of developing specifications, workplans, and managing change control.

Without properly establishing requirements, how can anyone hope to write an IT development contract, workplan, scope of work, implementation schedule, monitoring program, quality control plan, budget, or service agreement that can be effectively implemented? IT contracts are often so vague that they could be used to perform any type of project. At InternationalStaff.net, vague contracts are called “moon shot contracts” because they could be used to go to the moon.

Problems with managers who cannot or will not use requirements and change control processes properly are not restricted to e-commerce projects or even to the private sector. Many government agencies have long track records of failure in this regard, despite their heavy reliance on outside expertise.

In subsequent articles, the structure and sequence of e-commerce and other software development projects will be outlined, along with practical tips on developing software requirements specifications.

I invite you to use the Talkback section below to share lessons learned from software projects that you have been involved in. Please identify factors that contributed to successes and failures.