When it comes to software development, there is a lot that goes into this one principle that is critical to agility. Let’s take a quick look at a fictitious scenario that reflects all too many actual experiences.
You are a development manager for a company that is launching a new product, and everyone is excited to be starting fresh. However, this fresh start doesn’t mean that unlimited funds are available. It has been made clear by the executive steering committee that a marketable product must be produced within one year.
And after some initial wrangling a high-level feature set is targeted. Even though it is understood that the likelihood of everything making into the first release is small, there is a quiet expectation that most of the feature set will make into the first release. This concerns you, but you are comforted by the fact that you have a team of young, energetic, eager-to-please developers and testers who are up for the challenge.
They come out of the gate fast. New features are developed rapidly and it looks like you are in for clear sailing. And it is, for a while. Unfortunately, during the latter half of the project the team begins to struggle. It seems that every day testers are finding more and more defects. Your brief situational analysis:
- The developers can’t keep pace – defects are being reported faster than they can be fixed.
- Addressing all of these defects is taking time away getting the remaining feature set finalized.
- You don’t understand the root cause of the defects, but the following concerns flash through your mind: Have these defects been there all the time, undiscovered until now? Is the team getting tired and perhaps a little sloppy? Are there greater issues with the underlying design of the software itself, causing what should be unrelated areas to break when new functionality is added? A combination of all of these?
- While you would like to dig into these issues and take corrective action that will benefit you and your team in the long-term, you have another consideration. The business wants what they want.
You are taking a political hit because some of the desired features were dropped from the release, despite the expectation being set at the outset that this would most likely occur. Adding to your woes is that while the quality of the product is deemed good, it isn’t considered great. Everyone knows – but doesn’t talk about – the downgrading of some defects at the end of the cycle so that the deadline could be met with the quality goal of “no critical defects” in the product.
In addition, your team has told you that they really need to clean up some of the code because they brute-forced the coding of the final features in order to meet the deadline, plus they now realize that some of the early foundational work could be improved as well. Your hope is that there will be time to address to these issues later, if the product proves to be a success.
And shortly after its release, the product turns into a success – good news! The bad news is that there is a demand for new features and an ever-increasing volume of defects being reported by customers that need to be fixed. Revenue growth becomes intoxicating and being “responsive” to the customer is equated to fixing their defects and shipping new versions to meet contractual demands. (e.g., “I’ll buy this product if you add x…”)
Your dilemma is that it is taking longer and longer to add each new feature; and with each release the number of outstanding defects increases. This in turn is leading to longer release cycles, and your team is dealing with both maintenance and new product development. You respond by allocating a dedicated maintenance team with the hope of stabilizing the system and freeing up the core team to focus on value-add features.
Let’s stop right there. How agile is this? Even if the business is willing and able to adapt to change, the product itself is demonstrating to be anything but adaptable or extensible. It is carrying too much technical debt. The following diagram illustrates the problem:
Feature delivery is depicted as small bars that are increasing in size over time, meaning that it takes more time and effort to deliver each feature. In agile terms, velocity is declining. The decline in velocity relates to the decrease in product adaptability – the ability to adapt and extend the product. If this remains unaddressed, product costs will naturally rise as a consequence. In the scenario above these costs were further increased by adding a maintenance team to the mix. (This is not preferred because the team creating the problems remain removed from dealing with the very problems they are creating and are not incented to change.)
When the product adaptability and product cost lines cross, we have reached a point where it is costing more to work on the product than the value being returned. The product has ceased to be an asset and is now a liability. The only way to derive value is to keep it operational by performing small fixes without adding any new features (“maintenance mode”).
And the explosion at the brick wall? That’s there to illustrate a point that you will reach if technical debt remains unhandled; the point where the developers throw up their hands and declare that it is impossible to add any new feature to the product; if you want to add more, they tell you, than the product needs a rewrite.
The solution to this requires discipline. And I’m not talking about process discipline. High technical debt is present in software systems where there is a high degree of process discipline. Instead on leaning too far in any one direction, agile development strikes a balance between business, technical and process discipline.
The business needs to prioritize what it wants and work in collaboration with the development team(s) to produce a valuable customer outcome. It takes a measured, disciplined approach to define and produce only what is necessary to deliver value in the eyes of the customer, and to do so using a lightweight approach that leverages frequent feedback loops.
Likewise, development teams must be technically disciplined in their approach to produce valuable customer outcomes. Unless it is agreed up front that you are producing a throw-away prototype, the business wants the ability to extend and adapt a product in the future – as quickly as possible; the business does not want to be in a position of paying more and more to get less as time goes on. Despite their desire for features now, greater expense and delays in the future isn’t what they are signing up for, either.
This is what technical pillar is all about.
Practices such as pair programming and test-driven development exist to guide us in keeping the code base clean, which means well-designed, testable, reliable, and extensible. There can be challenges to these practices, some of which were covered directly in previous posts. I’ll quickly cover a couple of major points here.
The scenario above talked about how the team “got out of the gate quickly.” Implementing good technical practices from the outset will be slower than “just getting the code to work.” Any time that you hear the work just, take it as a warning! In this case, “just getting code to work” means that you will be faster early on, but that is all. You will pay for that initial speed repeatedly in terms of greater time and costs to implement additional features in the future.
Another problem with implementing technical practices is that it is impossible to measure how much you saved by taking the time now to keep the code base clean. You can’t measure how much damage you avoided (at best, you would be speculating). Conversely, it is possible to measure the amount of time being spent writing automated tests and refactoring the code. There will be those who will complain that a healthy percentage of time is being spent on those activities instead of implementing new features.
My primary advice is to take the long view – you can measure a team’s velocity, for example. Declining velocity can definitely be a consequence of the code becoming more complex, but it’s always recommended that you adopt the “go and see, and understand deeply” approach as there may be other contributing factors. Routinely running cyclomatic code complexity checks provides a useful gauge on whether the code base is becoming unwieldy. Another revealing indicator is when it is reported that “it takes a long time to get new developers up to speed because the code base is very complex.”
Ultimately, the desired outcome of using these practices it to prevent our software from becoming a liability, allowing us to quickly and easily extend and adapt its behavior to provide greater value to our customers. Technical debt is avoidable, as are the headaches and disappointments that come with that debt.
This post is a draft of content intended for an upcoming ebook: Agile Expectations: What to Expect from Agile Development, and Why. Blog posts will be organized under the “Agile Expectations” content on the left-hand side for easy reference. I welcome your comments and feedback! – Dave Moran