There is a lot of sword-fighting about the differences between Waterfall and Agile, and I can understand why people draw lines in the sand. There are those who have tried Agile development, are genuinely excited about it, and want to tell the world about their discovery – mostly because it solves problems that they’ve encountered, including long hours of unsustainable development. On the opposite side, there are those who feel like they are being told that the way they have been developing software all along is wrong – and they feel resentful because the past is being indicted.
The problem is that many the arguments that do arise shouldn’t exist at all! This post explores the origin of Waterfall development and what the “originator” of the concept felt was the proper approach. The question is, have a good many Waterfall projects are approaching software development as recommended by the one who is credited in creating the Waterfall model?
Winston W. Royce is credited with describing what is known today as the Waterfall model in his paper, Managing the Development of Large Software Systems, published in 1970. Royce did NOT use the term Waterfall, in the paper, although he did include what is considered a classic Waterfall diagram in that paper:
As many agile enthusiasts are quick to point out, Royce – on the very next page – went on to state, “I believe in this concept, but the implementation described above is risky and invites failure.” Actually, I’ve heard some people paraphrase this into Royce saying that “You would never want to build software this way.” This is implicit rather than explicit, as Royce acknowledges only that this model is risky; the entire paper describes the challenges and how to make the process less risky, from Royce’s perspective.
First, the challenges:
According to Royce, one major challenge is where the testing occurs and that some things are difficult to get right and must be verified through testing. Royce notes that, “The testing phase which occurs at the end of the development cycle is the first event for which timing, storage, input/output transfers, etc., are experienced as distinguished from analyzed. These phenomena are not precisely analyzable.”
Royce also recognized the potential impact of these problems: “A simple octal patch or redo of some isolated code will not fix these kinds of difficulties. The required design changes are likely to be so disruptive that the software requirements upon which the design is based and which provides the rationale for everything are violated. Either the requirements must be modified, or a substantial change in the design is required. In effect the development process has returned to the origin and one can expect up to a lO0-percent overrun in schedule and/or costs.”
Royce also understood another major source of trouble with software projects: Requirements. Royce observes, “For some reason what a software design is going to do is subject to wide interpretation even after previous agreement. It is important to involve the customer in a formal way so that he has committed himself at earlier points before final delivery. To give the contractor free rein between requirement definition and operation is inviting trouble. “
The steps Royce recommends to correct these problems illustrate significant insight into what software should have been all along – but hasn’t been.
Royce felt that it was important to get everyone on the same page. (Imagine that!) “Write an overview document that is understandable, informative and current. Each and every worker must have an elemental understanding of the system. At least one person must have a deep understanding of the system which comes partially from having had to write an overview document.”
The overview document was only the beginning for Royce. He had a strong preference for documentation, and lots of it: “In order to procure a 5 million dollar hardware device, I would expect that a 30 page specification would provide adequate detail to control the procurement. In order to procure5 million dollars of software I would estimate 1500 page specification is about right in order to achieve comparable control.” (Remember, we’re talking 1970 dollars here.)
Royce also observed that customer involvement was essential to success. “It is important to involve the customer in a formal way so that he has committed himself at earlier points before final delivery. To give the contractor free rein between requirement definition and operation is inviting trouble.”
Finally, Royce recommends staying away from the single-pass Waterfall model. In fact, his diagrams point towards iterations and what he hoped for versus what he found to be reality. The first being iterations confined to successive steps:
But Royce admits that iterating is likely to occur as shown here:
To make iterative development productive, Royce put forth the notion of doing it twice: “If the computer program in question is being developed for the first time, arrange matters so that the version finally delivered to the customer for operational deployment is actually the second version insofar as critical design/operations areas are concerned.”
Royce doesn’t mean to the entire project twice, but he does advocate using a simulation of the final product: “Note that it is simply the entire process done in miniature, to a time scale that is relatively small with respect to the overall effort. The nature of this effort can vary widely depending primarily on the overall time scale and the nature of the critical problem areas to be modeled. If the effort runs 30 months then this early development of a pilot model might be scheduled for 10 months. For this schedule, fairly formal controls, documentation procedures, etc., can be utilized. If, however, the overall effort were reduced to 12 months, then the pilot effort could be compressed to three months perhaps, in order to gain sufficient leverage on the mainline development.”
In the end, Royce’s recommendations for an effective Waterfall project are really recipes for success in any project. Involve the customer, make sure that the team is on the same page, prototype and iterate. The amount of documentation and level of team collaboration are the primary differences between Waterfall (Royce’s way) and Agile development.
And how many Waterfall projects keep Royce’s advice in mind? It would be interesting to see how many projects that are struggling are heeding Royce's advice, and to what extent.