Software Development is a Learning Process

August 15, 2009

Have you noticed that software projects struggle with predictability? There are a a number of reasons why predictability is elusive, but one reason is that there is a great deal of learning going in, particularly in the early stages of a software project.

Those on the business side do not understand how to design and build software, which places them in the position of being able to articulate what they want to get out of the software, but unable to provide a design. Conversely, programmers can design and build software, but lack the business knowledge required to put business value into the software.

In order to build truly valuable software, business and software professionals must collaborate to solve difficult problems in the most effective way possible. This collaboration occurs most heavily at the beginning of a software project, but it does not end when the requirements have been gathered.

Unfortunately, all too often software projects demand solid estimates immediately after the requirements have been articulated. The assumption is that since the business has stated their problem and goals to the software professionals, it is now up to those in the software side to determine just what it will take to design and build the software.

With many software projects, these estimates become the very real Date by which the team will be judged. If the team misses the Date, the project will be regarded as a failure. Is this fair?


The most effective software projects continue to learn and explore throughout the life of the project. Just because the requirements have been stated doesn’t mean that the software professionals have everything that they need. The design and development process will raise questions, and this is where the business can and should support the programming staff.

Collaboration will yield better results than having programmers go off on their own. Programmers are smart people, but they aren’t qualified to make decisions about the business. Programmers spend a good portion of their time keeping up with technology and methods for translating business needs into working software, but don't let them make guesses about the business! This is a recipe for frustration and the need for re-work; and you will likely discover that you need to re-work the software at the point when you think that something is "done."

Most projects significantly reduce learning – also known as variability – within 20-30% into the project, as reported by Steve McConnell with the Cone of Uncertainty.

The exploratory, learning nature of software projects is advantageous to the business. I’ve seen projects where programmers – once they’ve reached a solid understanding of the business problem at hand – engage in a productive dialog about how to apply technology towards the problem, yielding greater results than what could have been otherwise achieved.

The main takeaway is that software projects should begin with a rough estimate, but then seek to refine the estimates at the 30% mark to gain a more accurate picture of what the project entails and determine what the real Date is.

I believe that this is true for any project, regardless of the methodology that you are using. Teams will gain greater insight as they step deeper into a project. They will learn more about what really needs to be built, and they will have a gauge on how fast they can build out the feature set as well as having greater insight to any potential risks.

This is not unprecedented, and has been used in other technological projects outside of the software industry. In an interview this summer about the moon landing and Northrop Grumman Corporation’s role in designing the lunar module, Dick Dunne, former public affairs director was quoted as saying,We were learning as we were building. We were pushing the technology envelope. Windshields were cracking and engines weren't working."

The process used for designing the lunar module was strikingly similar to a good software project today. Gerry Sandler, who eventually became president of Grumman Data Systems, cited a strong team approach for the success of the project.

"You could talk to anybody at Grumman about anything," Sandler said. "You could see any boss, any specialist, anybody, and say I'm interested in this, let's talk about it."

Dick Dunne also noted that they held daily “stand-up” meetings – just like an agile/scrum team today. "Everyone went to that meeting and if you had a problem, you better tell it the way it is and not pass the buck," said Dunne.

Hmmm. Regular communication, confronting the facts, and learning as you are building. If we got to the moon 40 years ago using this process, and it seems reasonable that we can build software today following the same guidelines! And like a lot of things, nothing is as certain as it appears on paper.