Two Steps = Better Project Forecasts

June 28, 2011

In my prior post, I noted that there appears to be a 30 percent rule involving software projects: You’re going to spend 30 percent of your time figuring out what you’ve gotten yourself into. This places implications on the planning and estimation process that in my experience are best solved by Agile planning and estimation.

There is a tremendous amount of time and effort expended up front with traditional project planning, and one area in particular that is problematic is the tasking out of all of the work and guesstimating a duration associated with each and every task. There are two flaws with this:
  1. Software projects are the most unpredictable in the early stages of a project. A team needs to perform actual work to get their arms around the project.
  2. As I pointed out in Optimism Isn’t Just for Developers, people generally underestimate task-completion times.
While there is comfort in seeing everything laid out in a plan with risks and contingencies noted, what are the odds that the plan will play out as predicted? Virtually none; investing in significant planning up front is wasted effort. As the actual project work is conducted, people will realize that their initial estimates were flawed. And let’s not forget those inevitable change requests… The plan will need to be re-worked.

I prefer Agile development’s approach to planning and estimating because it gets a team to work quicker, and it has proven to be more reliable in the long run than any planning I’ve been a part of using traditional techniques. I particularly like Agile planning because it avoids the “optimism problem” that comes with those initial duration estimates.

Agile development uses a two-step planning and estimating process. Step One uses lightweight (comparatively speaking) planning: the User Stories are prioritized and sized – but a time component is not assigned to each User Story.

The premise is that humans are better at determining if one thing is bigger than another than they are at determining how long it will take to complete a unit of work. An Agile team will start with a small story and assign a point value to it – this number is completely arbitrary – but this User Story and point value becomes the reference point to gauge the size of the remaining work.

For each and every subsequent User Story, the team asks itself how much bigger or smaller is compared to the reference story – or any other story that the team is comfortable with, size-wise. Points are assigned to each story that reflects how much bigger or smaller each story is. Relative sizing is faster than tasking everything out and estimating duration, but it leaves off an important part of the planning equation: How long will it take to complete the work?

Answering the “How long will it take?” question is Step Two. This is also where I prefer Agile development to traditional planning. Start the work and let the team demonstrate – through delivery of actual features – what their true velocity is. This not only gets the team down to brass tacks much faster, but once you have a couple of iterations or sprints under your belt, the project end date can be forecasted with a good degree of reliability.

What if teams are off on their relative sizing? Won’t this impact the forecast?

If a team is somehow off on their sizing, the good news is that in my experience, it’s consistent across User Stories, not randomly spread. Actual work performed and velocity of the team will reveal this and allow the team to make adjustments to the story point values of the remaining work and the forecast.

Agile development isn’t about no planning. Agile development eliminates the waste associated with significant up-front planning that will be subject to change one way or another once the project is under way. It’s about spreading the planning throughout the process; with the Product Owner incorporating changes in priorities and the team tasking the work out on a just-in-time basis. Agile planning and estimating sizes the work and then measures how long it takes to actually complete the work, providing a more accurate assessment on when the project will actually be completed.

We haven’t run any numbers on this, but my sense is that the aggregate time spent on planning during Agile project is at least as great, if not greater, than the time spent on planning a traditional project. I feel that it is more effective planning because Agile planning and estimating doesn’t devote energy in developing and conforming to a plan, but instead forecasts, measures and adapts to ever-changing realities.