Software Development Workflow Shouldn't Mirror its Phases

April 11, 2012

(My apologies for posting a day late, but I’m in one of those cycles where I’m generating content just-in-time instead of having a good backlog ready. I paid for it this week, as I ended up very sick Monday evening and wasn’t able to make a Tuesday post.)

In the future, we won’t need to program computers. They will program themselves to perform whatever tasks we ask them to perform. Until that day arrives, we’re faced with doing things ourselves. So for now, we need to generate value by flowing through the standard development phases as quickly and easily as possible. With agile development we target delivering potentially shippable software at the end of every sprint, but NOT by working in a linear fashion:

It’s tempting to reduce work to its constituent parts and have specialists complete each phase in strict, gated phases where we sign off on the work before beginning on the next phase. We have evidence to support this approach, including studies that show how the relative cost to find and fix defects increases substantially at every phase.

Other studies confirm that there is value in getting things right in each phase, and to be productive we should be doing things like:
  • Understanding and verifying the requirements because this has been proven to reduce defects and prevent costly rework later.
  • Creating good designs and conducting design reviews because good design not only reduces the number of defects in a system, good design allows us to add new features in reasonable time frames.
  • Conducting code reviews because inspections have proven to reduce defects.
  • Creating a suite of automated unit tests with excellent code coverage that enable us to refactor the code with confidence, supporting the ability to continually and productively evolve and improve the design.
While a linear model with well-defined phases and specialists to perform that work makes sense to us, it is actually counter-productive in actual practice.

A classic example of a problem with functional specialization and functional departments working in isolation shows up when Quality Assurance testers raise a flag about a feature delivered by development that is more than just a bug, but a problem where the QA group had a very different interpretation of the feature than development did. And now development wants QA to test what they built, and QA wants development to change the feature to conform to what they expected to test – and someone else needs to come in and work through what should have been done in the first place. (It seems like a long time ago, in a galaxy far, far away… but I have seen this happen.)

Granted, teams should be communicating and sharing information enough to avoid this problem, but functional hand-offs introduce bottlenecks, creates overhead and inefficiencies that we want to avoid because we aren’t producing the greatest output for the smallest effort as Peter Drucker advised us to do many years ago.

Another reality is that when the actual work is performed, things get a little messier. For one thing, we find that we need to iterate:

Why? Because despite our careful attention to detail, there will always be questions and issues that surface as we translate the “what” into the “how” because the work is complex in nature. We need UI designers, database analysts, developers, testers, business analysts and the actual business users to collectively shape the outcome.

Successful software development is a complex, non-linear, and collaborative endeavor, and applying the principles of Lean and agile help us to obtain all of the desired benefits with the minimal amount of overhead. It’s the combination of the right mindset, approach, tools and techniques that achieve this. And it is a disciplined approach. I’ll cover more in my next post.