“Doctor, every time that I do this, it hurts! What can I do?”Why do smart people continue to do things that hurt when it comes to software projects? Because we’re human; and as humans who have been actively and continually working at improving our software development efforts we have come to believe certain things. Those beliefs are in turn reflected in what we value. And because we value certain things, these values are incorporated into the approaches and practices that we subscribe to as a means of achieving success.
“Well, don’t do that anymore!” – Old Joke
If we believe software projects are predictable, for example, then we value the process of planning, which leads us to implement an approach and practices that support gathering information and planning in detail up front. (Mezick, 2012)
This is also why it becomes a challenge to change. As humans we want a balance between what we believe and value and our behaviors and actions. If there are inconsistencies, we will find ways to eliminate those inconsistencies. (Keller, 2011)
As we’ll soon see, agile development will likely challenge your existing beliefs about what works in one way or another. This is also why I’m devoting this chapter to talk about how non-agile approaches are causing us pain. I want to create at least a moderate amount of dissatisfaction with the status quo in your mind so that when I describe how agile is different, the problems that agile is striving to overcome will be crystal clear to you.
We’ve improved our software development processes by formalizing certain things (and I’m not saying that this is a bad thing, we just need to keep improving), and a part of this formalization involved identifying specific phases as shown in Figure 1-6:
Yes, this is the classic waterfall model originally put forth (but not named) by Winston Royce in a 1970 paper, Managing the Development of Large Software Systems (Royce, 1970). This diagram depicts a flow that progresses from one phase to the next sequentially, assuming that work in the previous phase is complete before flowing down to the next phase.
There is an appeal to this model because everything feels well-ordered, structured and logical. We have reduced work into crisp, clearly delineated functions with well-defined boundaries that we anticipate being able to work through like a well-oiled machine. And we have some age-old evidence that indicates this is precisely how we should approach software development, as shown in Figure 1-7:
This makes sense, doesn’t it? It costs a lot less to find and fix a problem with a requirement as we’re defining our requirements than it does to correct a problem with a requirement after we’ve coded it, when we are testing the software, right? We have found support for this neat, linear approach.
In case you are wondering if agile development is asking us to abandon this economic model, the answer is no. We need to keep this in mind. But we do have a problem: we will encounter variability in most software projects. As you will see, with agile development we can address the cost of change issue that this model highlights when dealing with variability by working differently.
However, non-agile approaches drive us towards conformance to this model because we have built up a body of knowledge and practices predicated on certain beliefs. I’m sure you noticed the triangle in the last two diagrams. This makes it easy to add the constraints of the Project Management Triangle (also known as the Iron Triangle) as shown in Figure 1-8:
These constraints are intended to represent that you can’t change one side without changing the other, which is why good project managers always view projects as a series of negotiations. This approach is one mechanism associated with traditional project management that allows us to mindfully address variability (but without other changes it will keep the cost of change high).
However, if we equate success with delivering software projects on time, on budget with the expected features, we violate this basic tenet of project management. Our plans and expectations become geared towards controlling and minimizing change, which also puts us at odds with being adaptive and responsive to the business.
And it gets worse. Our expectations are in the right place, though. We want it all delivered with quality, which is another constraint that I’ve added in Figure 1-9:
Now we are boxed in, for what we consider to be valid reasons. And when we have enough bad experiences because this didn’t work out, we respond by adding more process, controls and oversight. We demand greater rigor in all phases, with more reviews, reports, approvals and signoffs at every phase. We replace managers in hopes that new individuals in charge will catalyze the development organization.
Of course, if we continue to use the same approach with the same expectations, odds are that we’ll get the same result.
Recall that Winston Royce is credited with conceiving the waterfall model in a paper he published in 1970? Well, the interesting part is that on the very next page after introducing this model Royce states, “I believe in this concept, but the implementation described above is risky and invites failure.” (Royce, 1970)
Royce went on to describe an iterative workflow that operated between the various phases as more realistic. Enough said (for now)!
Be Prepared for Change
As I previously pointed out, agile development will challenge at least some of your beliefs about what it takes to succeed. The good news is that you can start small and make a series of incremental changes that are aligned with being agile. However, you may find yourself conflicted with what you believe and what agile asks you to do.
No one is asking you to change your beliefs. But to give agile a fair shake, act as if you believe. See what results you obtain based on an honest and true effort on your part. And consider that because you are experimenting with new things, you made need coaching or training plus some practice time to get proficient. Finally, reflect on whether your beliefs should change based on your own experience. The option to change your beliefs remains yours.
Consider agile to be more like a road and not a destination, as shown in Figure 1-10. Your journey will take down a road where eventually, you will face a wall of your own beliefs (and values) that are reflected in your approach and practices. You can choose to turn back or break through this wall.
If you elect to break through this wall, you continue down the road, but approaching new challenges in new ways, using a different mindset. The remainder of this guide will cover what it means to be on the other side of that wall.
This post is a draft of content intended for an upcoming ebook: Agile Expectations: What to Expect from Agile Development, and Why. Blog posts will be organized under the “Agile Expectations” content on the left-hand side for easy reference. I welcome your comments and feedback! – Dave Moran
Boehm, B. (1981). Software Engineering Economics. Prentice-Hall.
Keller, C. P. (2011). Beyond Performance: How Great Organizations Build Ultimate Competitive Advantage. McKinsey & Company.
Mezick, D. (2012). The Culture Game: Tools for the Agile Manager.
Royce, W. (1970). Retrieved from Managing the Development of Large Software Systems