Agile Development and the Realities of Software Development

July 30, 2010

In my post, What is Better About Agile Development? (a reprint of the article Top 10 Reasons to Use Agile Development that I wrote for, my tenth and final reason for using Agile is that Agile addresses the realities of software development and the needs of the business. This post will expand upon this topic.

One reality of software development is that there is still too much unsustainable development in the industry as a whole. Squeezed projects with significant overtime and stress prevent us from getting smarter, let alone working smarter. Technology and business are continuing to change at ever-increasing rates; keeping pace with the business while keeping our skills current and relevant is a major challenge, yet essential if we are going to improve productivity. Through sustainable development and continuous improvement, Agile development helps teams and individuals do both.

A few tangible benefits associated with Agile development that meet the realities of both software development and the business – contrasted against traditional, plan-driven development projects – can be categorized broadly as: The early bird gets the worm.

Software project teams need feedback about features from the users. No matter what documentation was prepared in advance of development, there is nothing like having the users seeing the working software to obtain final, definitive agreement on whether the software meets their needs. Signing off on a specification and “building to the spec” might meet a project measurement, but it won’t necessarily satisfy the customer.

Why? In my experience, I’ve found that users aren’t particularly good at dealing in the abstract, or at least they aren’t as good at dealing in the abstract as software developers. They need to see things, not conceptualize them. That, coupled with the fact that communication between individuals is always subject to misinterpretation at some level, leads to the ever-present “requirements problems.”

I’ve heard the phrase, “Now that I see it…” quite often over the years, followed by a change request. Agile development’s frequent delivery of features enables users to routinely inspect the (working) software to provide immediate feedback. And because the highest-priority features are delivered first, software project teams get early feedback on the most valuable business features, allowing for course-corrections to occur early, instead of late, in the project.

Because the highest-priority features are being worked to completion early, the business can benefit from an early return on investment. With traditional, plan-driven projects, software is delivered as a complete package, but the business must wait until the end of the project to begin realizing a benefit.

Another, related benefit to using Agile development is that valuable time is not spent on the details of features that may or may not be a priority later. With traditional, plan-driven projects, you run the risk of investing time and energy into defining and designing features that may not be desirable or even needed. Time is money, and I prefer not to waste either.

Early delivery, early feedback, and an early return on investment; the early bird (the business) does get the “worm” with Agile development – or is it really a caterpillar that morphs into a butterfly in the cocoon of the Agile team?

Another reality is that business is rapidly changing. As Faisal Hoque points out in an article, The Speed of Business Today, “Constant change is the new dynamic of the global economy, and makes agility even more necessary than at any point in business history.”

Software development should align itself to the needs of the business by being able to respond change, and Agile development does just that. In the 2009 State of Agile Development Survey sponsored by VersionOne, the number one benefit obtained from implementing Agile development was the ability to manage changing priorities. 90 percent of the respondents said implementing Agile either improved or significantly improved their ability to manage changing priorities. (The survey data included 2,570 participants in 88 countries.) An older survey, the Shine Technologies 2003 Survey, was another global survey of actual experiences using Agile development. The highest ranked positive feature reported in the survey was the ability to “Respond to change over plan.”

In terms of general alignment between the business and IT, the 3rd-highest benefit obtained from using Agile development reported in the 2009 State of Agile Development Survey was an Improved Alignment Between IT and Business Objectives. This is supported by the Shine Technologies 2003 Survey, which reported that 83 percent of those surveyed stated that business satisfaction was better or significantly better.

Satisfaction can be a function of a variety of factors, but one reality is that software is being designed and built for the customer. People like to feel involved, that their needs are paramount in your mind. Traditional, plan-driven projects “respond” to the customer, but usually with change control and a “here’s how the project will be impacted” assessment. A rigid, process-oriented approach isn’t bad, but it can develop into an almost adversarial relationship where the customer feels that they are the one that must press for changes to their software.

This is because traditional, plan-driven projects have a focus on “planning the work and working the plan” where delivering to the specification is the key measurement. While there are project managers who run their projects as a series of negotiations (which helps maintain the relationship between the team and the customer), the very nature of the process causes people to lock in on what is defined – and change is becomes disruptive.

Agile development is designed to handle change; people don’t become wedded to a plan because they haven’t invested a lot of time and effort defining, estimating, and designing everything prior to actual construction. Agile teams wait until the last responsible moment to begin diving into the details of any one feature. And features that haven’t been started yet are considered negotiable at all times. This allows the business to change its mind with ease. And because Agile teams work with the customer throughout the project, the customer feels involved and valued throughout the entire process.

Agile development also stresses the need for the use of sound technical practices as a means of improving productivity. While you don’t have to be on an Agile team to make use of these practices, Agile development promotes them. Automated testing, Test-Driven Development, continuous integration are all geared towards improving throughput without sacrificing quality or sleep.

My final point, and final reality, is that software professionals are all adults, and they should be treated as such. Agile development, with its autonomous, self-directed teams allows people to define and manage their own work, which provides a motivational – and productive – boost.

Dan Pink, in his book Drive: The Surprising Truth About What Motivates Us, cited a study conducted by researchers at Cornell University that examined 320 small businesses, half of which granted the workers autonomy, the other half relying on top-down direction. The businesses that offered autonomy grew at four times the rate of the control-oriented firms and had one-third the turnover.

Top-down, control-oriented management brings its share of problems. In his book, The Way We’re Working Isn’t Working, Tony Schwartz puts this into solid perspective. Schwartz states, “The all-too common dynamic is today’s workplace is parent-child. Most employers tell employees when to come to work, when to leave, and how they’re expected to work when they’re at the office. Treated like children, many employees unconsciously adopt the role to which they’ve been consigned. Feeling disempowered and vulnerable, they lose the will and confidence to take real initiative or to think independently. Doing what they’re expected to do often becomes more important than doing what makes most sense.”

Command-and-control management creates the need to manage people more closely! Is that what we want? I know that I don't. Agile development's final contribution to the realities of software development is that it provides the opportunity for everyone to be treated as valued, professional, adults. That works for me!