Optimism Isn’t Just for Developers

May 18, 2010

“Developers are overly optimistic.” When it comes to software project scheduling, that’s one refrain that I’ve heard almost once too often. The reality is that developers aren’t the only ones who are optimistic about work estimates.

A recent study by social psychologist Dr. Mario Weick shows that people who are in charge – those who set policy and decide on courses of action – make time predictions that are inaccurate and overly optimistic. Why?

“The more people focus on what they want to achieve, the more they tend to neglect impediments, previous experiences and task subcomponents that are not readily apparent,” Dr. Weick explains. “Power tends to increase people's focus on intended outcomes. Although this can be beneficial, in the context of time planning we reasoned that power would lead to greater error in forecasts.”

It isn’t because people have greater faith in their abilities or that they see things through rose-colored glasses, either. “…Power affects what people focus on when they plan the future,” Dr. Weick says, “and this seems to be the root of the greater bias in powerful individuals' time predictions.'

The problem with planning and estimating doesn’t end with developers or those in charge. People in general tend to underestimate task-completion times. It’s known as the planning fallacy. A study conducted in 1994 involving demonstrated that the students did a terrible job of estimating how long it would take for them to finish their senior theses.

When it comes to software projects, estimates are regarded as poor, and industry evidence supports this. The Standish Group’s periodic Chaos Report is a widely quoted source. The statistics appear grim:
  • Average cost overrun: 45%
  • Schedule overrun: 63%
  • Actual functionality delivered: 67%
While there are plenty of skeptics regarding the Chaos Report figures, no one doubts that there are challenges with software project estimation. My quick list of planning and estimating issues with software projects:
  • Because trying to predict an end date at the beginning of the project is begging for trouble. There is too much uncertainty involved.
  • Mistaking estimates for commitments. An estimate is supposed to be an approximation and not a precise figure.
  • Software development is a people-intensive endeavor, and people introduce a high degree of variability into the equation. The complex interactions and dynamics of the personalities involved cannot be captured in an estimate.
  • Everything isn’t known by everyone up front. There is a great deal of learning going on throughout the course of the project. 
  • Developers Everyone tends to be overly optimistic.

4 comments

The initiation processes determine the nature and scope of the project. If this stage is not performed well, it is unlikely that the project will be successful in meeting the business’ needs. The key project controls needed here are an understanding of the business environment and making sure that all necessary controls are incorporated into the project.

June 9, 2010 at 3:20 AM

True. We all tend to be overly optimistic. Even our pessimistic estimates frequently appears as too optimistic.

But then, it shouldn't be an excuse when we have historical data at hand and refuse to use it. It shouldn't be an excuse when we use our optimistic attitude to try to trick the reality - this always fails as the reality tends to come to us and kick our butts every time we become overly optimistic.

So yes, optimism isn't only developers' fault but we should treat that as a good excuse to accept the situation we brought us into.

January 19, 2011 at 5:21 PM
Janice Clements said...

That's just the way they are. You can't change that fact. Thanks for your insights.

Computer Repair Services

December 6, 2011 at 9:44 AM

Post a Comment