Software development projects are notorious for having a high failure rate. In the context of this paper, “failure” is defined as, “not meeting the project sponsor’s expectation and/or stated requirements”. This would include such things as failure to function in the intended way as defined in a requirements document, not obtaining the required performance standards, going so far over budget that the project is canceled, or incurring so many bugs that the end-users view the system as unusable.
I began programming business applications twenty-nine years ago. In that time I’ve worked as a systems support engineer, developer, solution architect, director of development, consultant, trainer, and CEO of a software company. What I’ve learned from these years of experience is that projects fail repeatedly for a very short list of reasons. This paper will identify those key points of failure and offer simple guidance on how to avoid them – I say simple because to adequately cover all of the ways to solve software development problems takes volumes of books.
1 – Requirements
Many, if not most, companies have a natural history in the migration of their data storage, workflow, and reporting processes. The typical path of transformation is to go from paper, to spreadsheet, to database, to sophisticated business application. During this transformation, which often occurs over many years, the terminology and workflow process that were used when the germany company database business operated on paper often gets carried over to the spreadsheet. Business jargon and processes are established around how the business needs to operate under a paper-based system and continues after the company migrates to a spreadsheet-based system. This repeats itself again when adopting the database-based system, and so on.
The problem with this is that once a company has finally matured to using a fully capable business application for streamlining workflow processes, expanding the businesses capabilities for analyzing and reporting on business data, that system’s full capability is rarely realized. This is not due to the inability of the technology or the programmers creating it, it is typically caused by the business not being properly analyzed when preparing the requirements.
All too often, the internal sponsors of the project, end-users, business analysts, and other domain experts, are often in too much of a time constraint to meet milestones imposed by a Project Manager or Business Manager. Thusly; the project misses a truly golden opportunity to realize a much higher ROI on the system, greater productivity increases, longer life of the system, and better suitability for the way the business currently operates.
Here’s how you might resolve the problem:
- Advise/enlighten the PM: Let the PM and the project’s stakeholders know of the consequences of not evaluating the workflow process and domain terminology sufficiently.
- Document the cost of needing to rewrite a system: A rewrite in only a couple of years, or worse, never getting the system launched at all, compared to the extra time to conduct a proper analysis needs to be reviewed during the initial planning of the project. Engage the business analyst and/or architect to help with this as early in the process as possible.
- Question conventional terminology. Create a dictionary of the domain’s “Ubiquitous Language”. Challenge each term and its meaning to each stakeholder, sponsor, or end-user. In other words, requirements gathering is more than just collecting nouns and verbs.
- Work with a Domain Expert: A domain expert – versus everyday end-users – can analyze business processes that need to improve and how the system can accommodate that. Don’t just assume the data set tells the whole story about how it is used. The business analyst, or domain expert, must have a solid understanding of your business, not the technology to be used to serve it. Again, this should be done in collaboration with the architect.
- Create simple to understand user stories: Good user stories are short, precise, and limited to single actions. They should clearly state who, what, and why for each action the end-user or the system needs to perform. Don’t create elaborate requirements documents that obscure the intent of the requirement – it’s the old adage of, “can’t see the forest through the trees”.
2 – Translation of Requirements to Technical Specifications
The biggest “hat trick” in developing software is taking business concepts, which are often rather abstract in nature, and then converting them into very literal, concrete technical specifications. Many times the context of the business processes are either not understood by the programmers or, not accurately translated into the technical specifications and ultimately into the code of the system.
The problem with this is that you have business people producing the requirements and technical people making that translation. Unless the technical person has a true understanding of your business and, its business concepts, then the translation will most often be wrong. Unlike translating two languages with Google translate, where a person can guess at the meaning of words not translated correctly given a specific context of the conversation, a computer application cannot. Concepts, processes, actions all have to be very specific in order for the computer to process it.
Many development companies assign the task of making this translation to programmers. This is inherently flawed as programmers are dealing with the finest details of coding rather than the higher level, abstractions found in business. Bridging this gap in concepts and level of detail is nearly impossible to do well and, often time produces catastrophic failure in the project.
This is witnessed by observing the code and comparing it to the user stories. Often time the code combines multiple unrelated user stories into the same file, making it all but impossible to understand, modify, extend, verify, or maintain.
Another observation is that the code will be missing complete concepts derived from the domain experts and will be accommodated by a lengthy bit of code that works around the concept rather than articulates it. Examples of this would be where there are well used, common terms of the business, which relates to either specific data or specific processes that are real-world things in that particular business domain. When reviewing the code, it is common to see none of these terms used, but instead, replaced with technical jargon, arbitrary abbreviations, or worse, single letters. This makes it difficult to impossible to know if the code is truly matching the requirements. Even if the end-user functionality seems to be there and working, it doesn’t mean the code was constructed properly. What this can lead to – and almost always does – is that there is a high probability that while the first iteration of the system might seem to work fine, when the business wants to extend a feature’s capability or, add new features, the foundation of the code just won’t support it. I cannot count the number of times either I or other technologists have had to advise the client, “A rewrite is required”.