The Elusive Definition of Success with Software Projects

November 27, 2009

What is the state of software projects today? Are software projects more successful today than they have been in years past? Think about any software project that you know of that is just getting started. How confident are you that the project will be considered a success?

One problem with software projects is determining the definition of success. Consider the following, classic definition of a successful software project.

Delivery of software:
  • On Time
  • On Budget
  • Of High Quality
  • Containing the Expected Features
Does this meet your own definition of success? More importantly, just how is the software industry doing in terms of projects being completed and meeting this classic measure?

A 2006 Chaos Report from The Standish Group indicates that software projects are improving over time, as demonstrated by a similar study that they conducted in 1994. Let’s take a look at the numbers and how they contrast over the twelve-year span.

According to The Standish Group, 19% of the projects reported on in 2006 were outright failures, which is certainly an improvement over the 31.1% failure rate reported in 1994. A look at the rest of the figures reveals a different problem, where projects move from failure to “challenged.” A challenged software project is defined as failing to meet the classic definition of success in one or more ways, such as incurring a budget or schedule overrun, or having features trimmed from the final delivery, but the project itself is not deemed to be a failure.

The 2006 Chaos Report states that 35% of the software projects succeeded in meeting the on time, on budget, and expected features criteria during 2006, relative to only 16.2% in 1994. (While not explicitly stated, the assumption is that the quality objectives were met as well.) If we remove the 19% of outright project failures for 2006, we arrive at a significant, 46% figure for software projects that were in the challenged category during 2006.

Although these figures paint a dismal picture of the software industry, they only represent a piece of the puzzle. In fact, there is – and should be – a modified view of success. This was illustrated in a 2008 survey conducted by Scott Ambler, who had an overall goal to explore how Information Technology (IT) professionals define project success.

There were 279 respondents to Scott’s survey, and their definition of success in comparison to the classic definition is as follows:
  • On Time: 58% believe that delivering when the system is ready to be shipped is more important than delivering on schedule.

  • On Budget: 70% believe that providing the best ROI is more important than delivering under budget.

  • Of High Quality: 82% believe that delivering high quality is more important than delivering on time and on budget.

  • Delivered with the Expected Features: 83% believe that meeting actual needs of stakeholders is more important than building the system to specification.
After reading the survey and noting the responses to the classic measure of success, I reflected on past experiences and conversations that I’ve had with others about the nature of software success. In general, the measure of success to a great degree is a function of organizational culture. I’ve talked to some frustrated project managers and developers who have told me that in their company, teams that deliver on time are recognized as successful over other teams who deliver as little as a week late.

The real rub is that teams that delivered on time is still regarded as successful in spite of the fact that their software required substantial defect-fixing months after delivery, while the teams that were only marginally late delivered with higher quality, and did not need to allocate substantial time and effort to support a product of poor quality – and were able to move on to more productive ventures.

I decided perform a modest cross-check of the results of my own by posting a question on LinkedIn in the Software Development category. I cited the survey and asked: “What should the definition of success be? Are we being more successful today than yesterday because we are changing the definition of success?”

Needless to say, I received a range of responses. There were answers that qualified success as meeting the business objectives of the project, while other answers were in the “it depends” category. Overall, while there was agreement with the classic definition in principle, there was a lot of deferring back to what the stakeholders of the project defined as successful.

This leaves us in a situation where not only are software projects challenged, but the very definition of success is also being challenged. It is certainly in the best interest of everyone to have a candid conversation about the definition of success before taking on a project. Above all, acknowledge that there will be challenges, such as people changing their minds about the requirements, or details surfacing that change everyone's understanding of the original requirements.

And don't expect to have it all. The classic success criteria is about predictability, and software projects are rarely predictable. I wrote about this in my Can Software Projects be Predictable? post.

Scott Ambler’s December 2008 Software Development Project Success Survey

Standish Group Report: There’s Less Development Chaos Today by David Rubinstien, March 1, 2007

Software Magazine: Project Success Rates Improved Over 10 Years

What Football brings to Software Development

November 20, 2009

This past Sunday, I watched the New England Patriots lose a big game to the Indianapolis Colts 35-34, and everyone is pointing to the coaching decision by head coach Bill Belichick as the cause of the loss.

Towards the end of the game, a critical situation arose that required a key coaching decision, one that meant winning or losing the game. The Patriots were facing a fourth down with two yards to go on their own 28-yard line, with 2:08 left in the game.

Normally, this is a punting situation, no questions asked. But Belichick opted to go for the first down. He clearly didn’t want to give the ball back to Payton Manning and the Colts, and I don’t blame him.

Almost everyone is calling this the worst coaching call he could have made. Of course, if the play had worked, everyone would be tripping over themselves in admiration of Bill Belichik’s guts, noting the confidence that he has in Tom Brady and the offense, how the Patriots must have ice in their veins, and so on.

From my Monday-morning quarterback vantage point, I firmly believe that if the Patriots had punted the football, Payton Manning and the Colts would have driven down field to win the game. In the end, the Patriots lost a highly competitive game, but what the heck does this have to do with software development?

Just like software development, football players execute better than the coaches. I readily acknowledge that there are plenty of ex-players who were darn good players in their day that are now coaching, but there is a difference between knowing the game and playing the game.

While age is a factor on the football field, time and experience with the latest tools becomes a factor in programming. Once someone progresses beyond a junior level, someone who is programming full time should be better able to perform the act of programming better and faster than a manager. A manager's job should involve a variety of other duties that will negate their ability to be as proficient as others who exclusively focus on programming, particularly as time marches on and new tools and technologies are introduced.

This doesn’t excuse a software manager from understanding the software game. To be effective, I am a firm believer that a manager must understand what he or she is managing. True, you may not be able to execute as well as those that you are managing, but that is OK. You do need to understand and manage what goes into producing results in order to be effective.

For example, football coaches watch and measure what goes into making the individuals and the teams better and coach the players on those aspects of the game. They want hand-offs without fumbles, they want everyone coordinating their efforts by blocking when and where they are supposed to be, running pass routes designed to provide the quarterback with options, etc.

The bottom line is that while good coaches absolutely want a higher score, they don’t fixate on the score itself. They sure as hell can’t ask players to work “overtime” in a game in order to bring up the score. The players need to execute well, and they need to review what they did right and what they did wrong, make adjustments, examining their execution from both as a team and an individual basis.

The key is to understand what it takes to be successful, and taking time to make the observations that allow you to see the problem areas in which you need to improve. This means that as a manager, you have to have your eyes on the field, observing your direct reports and the team interactions from the perspective of an informed individual, someone who knows how software should be developed as well as how to manage people and teams.

As an example – and in keeping with my football theme – is to consider what the real cause of the Patriots loss to the Colts. They have a red zone problem. The Patriots settled for two field goals in that game after driving most of the way down the field. Had they scored a touchdown instead of a field goal one of those times, they would not have faced the game-winning, pressure situation that everyone is now crucifying Bill Belichick for. They could have punted the ball away, because giving giving the Colts back the ball at that wouldn't have mattered. The Colts would have needed two scores, not one, to take the game away.

Examining the game films will reveal why the Patriots are failing to score touchdowns in the red zone – which could be a number of causes. The key is not to focus on the score and the one call that led to their losing the game. Sure, maybe punting would have worked. But scoring a touchdown instead of kicking a field goal just once – and earlier in the game – would have guaranteed victory by that now-fateful 2:08 point in the game.

In the software business, look for results, but focus on those things that go into achieving results if you want to be successful.

Task-Switching is for Computers, not Humans

November 13, 2009

Computers are much better at multi-tasking than humans are. Why? Because a computer is designed to multitask. When a computer switches from performing one task to another, it saves the entire state of the task being switched out, enabling the computer to literally pick up where it left of when it returns to that task.

Humans aren’t wired that way.

Consider an activity that requires keeping a lot of details in your head along with applying some creative thought; where mental focus and concentration is essential, like programming. Programmers keep a lot in their heads during the act of programming, like how the code will be organized, what data structures are being used, the naming and use of variables, all of which is centered on the act of translating business requirements into detailed instructions for the computer to follow.

What happens when a programmer is pulled of one programming project to work on another project or problem? Valuable productivity is lost. In his book Quality Software Management: Systems Thinking, Gerald Weinberg proposed a rule of thumb to calculate the waste caused by project switching:

The graphic illustrates how switching between only two projects is very costly: You lose 20% of your time! By the time you get to three or four simultaneous projects, you’re losing significant productivity to task-switching.

This is also supported by recent studies written about in a Wall Street Journal article New Studies Show Pitfalls Of Doing Too Much at Once. One study in the Journal of Experimental Psychology points out that people who multitask are less efficient than those who focus on one project at a time, because the brain has to overcome "inhibitions" it imposed on itself to stop doing the first task in order to take on another tasks.

Not only that, but attempting to manage two mental tasks at once reduces the brainpower available for either task, according to a study published in the journal NeuroImage. This was proven by a relatively simple experiment where subjects listened to sentences while comparing two rotating objects. It was found that the ability to process visual input dropped 29% if the subject was trying to listen at the same time.

How does this translate to my day-to-day software management universe? As a software manager, I have my own personal rule that I manage by: If a project is important enough to start, it is important enough to finish. When someone is allocated to a project, they are expected to see that project through to the end; and I resist pulling anyone off of a project at all costs.

Equally important, I regard interrupting programming time as something that I only do as a last resort, when there is a critical issue. Like when another team is dead in the water and someone else has the skills and knowledge available to help un-stick that team, or when a customer is down and I know that one or more individuals can help. As a rule, I do everything I can to make these situations exceptions that occur very infrequently.

I'll wager that virtually everyone agrees that task-switching of programmers is very counter-productive, but all too often the day-to-day practices of management translate into interruptions and disruptions of productivity! We're all human, and it's easy to think, "What is one day out of a schedule slated to last a few months?"

Let's take a look at an example provided by James Shore and Shane Warden in their book The Art of Agile Development. They note that a programming task estimated at 8 hours could take 2-3 calendar days if a programmer gets interrupted.

While the impact of interruptions on an entire project don't seem significant when they occur, keep in mind that projects rarely have one BIG event that someone can point to as THE cause of a target date being missed. The reality is that software projects get into trouble a little at a time, and schedule misses are almost always due to the cumulative effect of lots of little things going wrong over time, like tasks taking longer than estimated. And if you are interrupting programmers, you are disrupting projects!

Programmers meed concentration time – and blocks of it – to be productive. Leave task-switching to computers.

Six Ingredients for a Successful Team

November 6, 2009

What Makes a Team Successful? I’ve watched different sports teams succeed and fail over the years, and regardless of whether you are talking about high school sports or professional sports, the same key ingredients are present in the teams that are successful. The same holds true in the workplace.
  1. A clear goal with a specific focus. Even teams that should be great will fail if they attempt to take on too much. By keeping the focus narrow, and with clear objectives, teams can achieve success. If you truly have a lot of ground to cover, then cover it in small steps, using specific, measurable intermediate goals. Too many software projects in particular fail because they attempt to cover too much ground all in one shot.

  2. Talent. Without talented players, you can only go so far. Talented people have a gift, the capability to perform, with the potential to reach greater heights than others. On the flip side, there are talented people out there who don’t reach their true potential because they don’t work to develop their potential. They remain on par and competitive with others, but rob themselves because they don’t apply themselves.

  3. Dedication. This is displayed by those who are enthusiastic and willing to apply themselves to their chosen sport or profession. They study and learn. This can make the difference between being darn good and being a superstar. Even an average Joe can compete with others that possess greater talent if those with the talent aren’t working to improve their game. Larry Bird was known for dribbling a basketball over every square inch of a court prior to a game. Why? Because he wanted to find the flat spots where the ball didn’t bounce back as quickly; he filed this away and used this knowledge to his advantage to make a steal. His dedication provided him an edge. In the working world, my take is that dedication is displayed by those who have the “I’m responsible for …” attitude versus those who have the “I work at… ” attitude.

  4. Practice. Talent and dedication do nothing if you don’t practice what you have learned. As a software developer in years past, I did something that I advocate to this day: Put together your own, small applications to learn things outside of whatever tasks you have going on with your current day-to-day job. This will likely be done on your own time, on your own dime, but it will give you practice and experience that broadens your knowledge and abilities. Focus on understanding developing and honing skills without cutting corners. This means using good approaches to development, not hacks. Practice alone doesn’t make perfect. Perfect practice makes perfect.

  5. Coaching. Any team that is successful has a good coach behind it. Coaches motivate, recruit, and set the tone and direction. They get people to understand what they need to do and why they need to do it. They recognize and reward people for a job well done. They call the plays and expect the best from themselves and those on the field. They make the hard choices of benching people or removing them from the team when necessary. It’s about blending the individual talents into a cohesive, unified team, helping everyone to realize their full potential in the context of working collectively as a team. Coaches watch the hand-offs, transitions, communications and interactions between teammates.

  6. Playing as a team. The most successful teams are greater than the sum of the parts. Great teams communicate openly and honestly, they understand each others’ strengths and weaknesses, and they push each other to become better. The team score matters the most, but everyone is expected to contribute. Great teams don’t have weak links, and they don’t tolerate laziness. They want talent, but will take someone who is dedicated, hard-working and willing to practice over someone with talent but no drive or lacking in the desire to be a part of the team. The real need for successful teams is to get everyone flowing in the same direction, dividing the effort – with everyone pitching in – to reach a common goal. Personal goals are second to the team’s goal because great teams understand that they can achieve more than a collection of individuals who are out for themselves.