How Long Will It Take To Deliver?

February 27, 2009

A typical question that arises in the software business – and I’ve had this happen to me literally as a meeting to elicit the business requirements was coming to a close – is, "So, how long will it take to deliver the requested features?"

My default answer in this situation is, “Well,” (my standard lead), “we’ve got a lot to review, I’ll get back to you as soon as we are finished with the estimates.”

Have you ever felt challenged about delivering estimates? There are times in the past when I've had feelings of trepidation, coupled with the thought that those on the business side were being unreasonable and overly demanding. Their urgency was crystal-clear, and they didn't seem to care one bit (no pun intended) about what it would take to produce the software. They wanted it yesterday. Are there unreasonable people in the business world?

Yes, there are. There are some who aren't looking to create that win/win scenario. They want to win at all costs. For them, winning is the thing.

More often than not, however, the business has its demands that cause what appears to be unreasonableness to those of us on the software side. After all, the business has its customers that it has made commitments to, and the business needs to make its dates as well.

This leads to an interesting dynamic when business people interact with software teams. Many times, those on the business side push towards an aggressive date to ensure that they’re getting full value for their money. It’s partly about maximizing value, but there is another reason. Negotiating is a way of life for many business people. It's their default mode of operation.

Unfortunately, too many in the software world spend so much time with computers that they are terrible (or at the very least inexperienced) negotiators. Bottom line: don’t let someone negotiate you into a bad position! Make sure that you have your facts, and by all means hold your ground.

When it is all said and done, no matter how passionate someone is being about the end date, the world won’t stop if your position – one that you can support – is that there is too much being requested given the constraints that you have (resources, time, budget).

No one on either side of the table wants to start a project with the dates that are wrong from the outset. In some cases it would help to understand the key drivers that are pushing the business. Ask questions! Maybe another solution is available, or priorities for delivery can be adjusted to create a win/win scenario. Other times, problems could have been avoided if someone had provided the business side with insight on why exact estimates are difficult to come by.

It’s about educating, building trust, and establishing a true partnership.

Consider the following.

For some reason, software projects always seem to start with an implicit assumption that ALL of the requirements have been fully expressed and will remain stable. Even when everyone involved agrees that there will be variability, once a date gets published…

People forget that the definition of estimate is approximate and everything ends up revolving around the exact end date provided in the estimate. It's not right, but that is what happens.

The truth of the matter is that requirements are rarely as complete and locked down as everyone thinks they are – despite how complete they appear on paper. Even when you think that you’ve nailed the requirements from a business standpoint, the act of writing software will surface the need to define new requirements.

The primary cause of this is that computers require specific, detailed instructions for every task that they perform, and it is virtually impossible to catch all the little nuances of how a business operates at this kind of detailed level up front.

Now add the fact that non-business people (programmers) are translating someone else’s knowledge and understanding of the business into the software. So I ask you, what are the odds that the software will be completely correct on the first pass? I came across a great quote that expresses this reality:

"Software is an opinion about how the business works."

I don’t know who coined this phrase, but I’d love to find out so that I can give credit where credit is due.

The reality is that software projects require adaptation as they progress, and while I feel that you should perform as much due-diligence up front as possible to minimize the unknowns and to help create greater confidence with the estimates, I also recognize that there will in fact be unknowns and change that must be taken into account.

This is also why I advocate small projects. Small projects that target delivering the features that will provide the greatest business benefit are the projects on my “most likely to succeed” list. Small projects keep issues contained.

I also advocate that the people doing the work must provide the estimates. People who will be performing the work must have an understanding of what is being asked and they must be committed to accomplishing that work. The best way to achieve this is to have them estimate the work in the first place.

Another very important concept that I recognize as a manager is that estimates will vary depending upon who will be doing the work. Studies have shown that there are 10x productivity differences between programmers at the same level, something you must take into account if you want an accurate end date!

When it comes to the actual estimation process, I’ve found that the most reliable estimates come from a team effort. The general approach is that the team meets and reviews the requirements one by one, listing the development tasks required to meet each requirement. Everyone gives their estimate, followed by a discussion if someone is low or high – so that the team can understand the deviation.

Sometimes there is someone on the team who knows that they can leverage code in some way, either through a third-party component or existing library routine that can reduce the effort involved. Other times someone may recognize certain considerations that others have failed to notice. Overall, the vetting process by a team results in fairly reliable estimates.

Another benefit of the team approach is that there will be recognition within the team that certain tasks can and should be performed by a specific individual. Someone may have related experience and be willing to write the code – or at least mentor someone who wants to gain expertise in this area.

The final benefit with a team approach is that you've got a bunch of detail-oriented people examining the requirements from the perspective of those who need to translate what has been provided into software. A team estimating session is a great, early way to catch problems with the requirements, particularly those tacit requirements that detail-oriented people will surface.

Many times there is pressure to starting coding – now. And every time that I’ve short-changed what I know to be solid due-diligence, particularly the up-front work, I’ve regretted it. Do your homework and engage those on the business side with meaningful dialog about what it takes to deliver software.

Over time, trust will be built through a demonstrated command of the facts and by meeting key delivery dates – something that can only happen with realistic estimates. In the end, effective, results-oriented delivery that truly satisfies the business requires a true partnership.

Productivity in Software Projects: Requirements

February 19, 2009

In my last blog post, Six Keys to Successful and Productive Software Delivery, two of the six keys really focused on the business/customer side of the fence. Key #1: Know Your ROI is all about considering the value – the measurable benefit – that will be derived from the software development effort. What I’m talking about here is making sure that you are actually doing the right thing – before worrying about doing things right.

Don’t short-change this process! The business must provide input on the true goals for the software – and what will provide the biggest bang for the buck. For those of us who work in the commercial software industry, this is a must. We need to consider what our unique value proposition is relative to our competition, but my position is that this should hold true for in-house software development efforts as well.

After all, why would anyone want to spend time and effort duplicating what is readily available in the marketplace? Thinking more deeply about what you need and why something that is commercially available does not work may lead you to a much more achievable and proper solution. You could find yourself in a situation where the commercial package provides 80% what the business needs, and the real value-add is to augment that solution in some way. And augmenting something is one heck of a lot easier than building everything from scratch!

And for those who are thinking that you can develop a quality application that equals what is commercially available, I’d re-check your estimates. Odds are, you are being overly optimistic somewhere. Think twice before building something that is already available!

Key #2: Customer Involvement is a related and vital component. Yes, if the business is setting the priorities appropriately they are indeed involved, but what I’m talking about here is getting involved at another level. After you’ve decided what the right thing is to do, you need to get it right.

And getting the requirements right is so essential that I would like to reiterate the points I cited from Steve McConnell and Karl Wiegers:
  • Requirements defects in software projects account for approximately 50% of product defects.

  • The percentage of the re-work on software projects due to requirements defects is greater than 50%.
Just having involved customers does not mean that you will get the requirements right, either. James Shore has a great essay called Up Front Requirements w
here he describes a project where he did the requirements “right,” but the project still failed. James, I can relate!

One problem is that business users – the ones who have the expertise and knowledge needed to make a project a success – already have jobs, and it isn’t to sit with development teams for long periods of time to talk about their business and the features that they need. Another problem is that it is very difficult for business users (or anyone) to evaluate – on pieces of paper – what has been described to the development team as being complete and accurate.

On the flip side, the development team needs to know what to build, and computers happen to be the dumbest, most literal devices on the planet. (For those who have raised kids, teenagers run a close second here, but I give the edge to computers; there are plenty of smart teenagers, they just use their intelligence for evil by frustrating adults with their word games.) Computers need to be told – in excruciating detail – what to do, when to do it, and how to do it. And they will only do what they’ve been told to do, no more and no less.

This is why I’m not a proponent of large-scale software projects. The larger the project, the greater the likelihood of failure, or at least significant cost overruns. (A cost overrun alone can push a project into the failure category, particularly if it is significant.)

There are two major problems with large projects:
  • There are more features. The result is that there is less attention paid to each individual feature. In my experience, the smaller projects devoted much more time to defining, designing, reviewing, and iterating back through requirements than the larger projects. The larger projects simply taxed people too much.
  • There are more people involved, and this increases lines of communication, requires more effort to coordinate work, adds more complexity because a greater number of components need interoperate, and there is less time available to scrutinize every aspect in the same way that a smaller project can operate.
Regardless of your approach to software development, I advocate keeping the scope of any project as small as possible; and for those projects that are BIG, at least define a series of projects designed to focus on the highest-valued (by the customer) features first. Shoot for a series of smaller successes.

If you have a complex project mapped out with one little circle that contains the phrase “and then a miracle happens,” your project will not succeed. As crazy as this sounds, there are projects where people intuitively understand that this point exists and even joke about it, but no one will publicly admit it or push back. The project rolls on until it eventually reaches the breakdown point – where the miracle was supposed to happen, but didn’t. And another project failure is added to the record books.

One great resolution (in my mind) to everything that I’ve discussed in this post is addressed with Agile development. One important aspect of Agile is that it focuses on delivering working software in small increments of time so that the software can be routinely inspected. This solves one major problem that many users have: they can’t envision how the software will work for them by reading specifications documents! They actually get to see working software to provide feedback on it.

At my company, we’ve adopted Agile development and had greater success with our projects overall. Adopting Agile also focused our attention on our priorities to a much greater extent. While the switch to Agile had its bumps in the road, all in all, it has been a great move for us.

Six Keys to Successful and Productive Software Delivery

February 14, 2009

This blog follows up on my last post, Performance, Productivity, and Results Defined. At the risk of stating the obvious, I’ll provide my take on the essential keys required to achieve true productivity and to generate value.

Key #1: Know Your ROI
Every software project involves people; and since there will be an investment in time and effort of these people – well-paid knowledge workers – to build the actual software, there should be some up-front thinking and analysis performed to determine that there will be a sufficient return on the investment.

Part of this analysis should consider the high-level features and capabilities desired, with a prioritization of those features and capabilities that will provide the greatest benefit. Why? Because there are far too many software projects are regarded as failures, as reported in Failure Rate.

When it comes to software, smaller projects are definitely better, as smaller projects improve your chances for success. And if they are going to be small, those projects should concentrate on delivering those features that drive the greatest benefit. This will also contribute improving results and productivity as noted in my next key.

Key #2: Customer Involvement
Software projects can go awry for any number of reasons, but one major reason centers around the requirements – the business end that the software is meant to meet in the first place! Customers absolutely need to be involved in the feature specification and design process, and ideally participating with the software team throughout the development cycle.

Consider the following data points drawn from industry gurus Steve McConnell and Karl Wiegers:
  • Requirements defects in software projects account for approximately 50% of product defects.

  • The percentage of the re-work on software projects due to requirements defects is greater than 50%.
These points are significant in their own right, and I feel that this places requirements in a class by itself. Mismanagement or inattention in this area alone will force the overall costs of a project up while driving the quality, satisfaction, and ultimately the value delivered sharply down.

Key #3: Hire and Retain Capable, Competent People
In his book Good to Great (© 2001), Jim Collins asserts “Get the right people on the bus, the wrong people off the bus, and the right people in the right seats – then figure out where to drive it.” So just what do the “right” people look like?

My take is that there are some specific skills and abilities that are exhibited by top performers, regardless of whether they are programmers, quality assurance testers, managers, writers, or anyone else in any other role. My “top skills and abilities list” is as follows:

  1. Motivation.

  2. Critical thinking skills.

  3. Communication and collaboration skills.

  4. Demonstrated drive for continual learning and exploring.

I have two more that are desirable, and contribute to overall higher productivity and effectiveness:

  1. Domain knowledge.

  2. Experience.
When you have great people working for you, you’ve positioned yourself for success right out of the gate. I’ve seen great people working with great teamwork overcome some ridiculous hurdles in the past. Software is a people-intensive business, and you need great people to succeed.

Key #4: Prevent Defects
Key #2 stressed the need for customer involvement and getting the requirements correct, as 50% of the product defects in software projects can be attributed to requirements problems. Preventing the other 50% of product defects – or at least catching them as early as possible – pays a dividend as well. A general industry assessment is that finding and fixing a software problem after delivery is 100 times more expensive than finding and fixing early on, in the requirements or design phases.

Check out the following chart to see how costs drive upward – and increasingly so – the further into the software phase you progress:

There are supporting best practices in the software industry to prevent defects:

  1. Conduct design reviews, checking to ensure that there is alignment between business requirements, system architecture, and test scenarios. There should also be a check to ensure that software re-use and the incorporation of design patterns to solve common programming problems are planned.

  2. Conduct automated and manual code inspections, checking for – and preventing – the implementation of high-risk code. This means ensuring that the code does not contain any overly complex routines, that it is commented, clear and maintainable, that any planned software re-use and design patterns were implemented, and that proper error handling and security requirements have been addressed.

  3. Test, test, test! Test early, test often; keep the quality up by conducting rigorous tests throughout the software life cycle. These include unit tests and feature tests. Naturally, the more automated your tests, the more productive you will be.

Key #5: Teamwork and Collaboration
Today’s world is more demanding than ever before. We’re tackling harder software projects that involve interconnections and interactions between other systems and devices that require people to communicate and work together. In the early PC days, it was much easier to be the single and only expert – hardware, software, telecommunications – but those were simpler systems.

Technology has advanced significantly and will continue to do so, forcing specialization and focusing efforts of individuals. Delivering sophisticated applications that communicate with each other and expose interfaces in a wired (and wireless) world demands teams and collaboration.

While your personal productivity may require some “quiet time,” there is a need to work and play well with others. The days of being a solitary programmer with people sliding pizza under your door – because you can’t or won’t play well with others – is over.

For some people, working as part of a team is a challenge. I’ve seen people who are supposed to be a part of a self-organizing team choosing to optimize their own work to the detriment of the team as a whole. There can be reasons for this. Perhaps management is evaluating people on their individual assignments to the extent that teamwork takes a back seat; other times a simple coaching exercise is required – with the focus on the team and the team’s results being emphasized.

True teams, once they start to gel, can be an overwhelming force. People assume responsibility for their tasks, take on tasks that are outside of their own areas of expertise when a need arises, discuss and white-board options to solve difficult problems, and in general make each other more effective than the sum of the parts.

Key #6: Management and Organizational Support
The tone that is set – the organizational culture – and the methods that management uses to manage people can play a vital role in the continued success of an organization. You can achieve short-term success by driving people with “death march” projects and micromanagement, but in general you run the risk of driving good people straight out of your organization through mismanagement.

Determining and communicating organizational priorities and objectives so that they are understood, setting expectations of individuals and teams – and ensuring that performance evaluations and recognition systems are aligned with the organization’s priorities and objectives – is vital. Equally important is finding that middle ground between micromanagement and being completely disconnected.

Knowledge workers in general do not like to be micromanaged, and in fact find it to be a great disincentive. Personally, I feel that micromanagement introduces productivity problems in general. If you insist on micromanaging, people that work for you will always look to you to provide the definition of the work, but they will not contribute any value-added thoughts that could make a meaningful difference. They will be conditioned to providing what you ask for and only what you ask for – which means everything revolves around the micromanager.

On the flip side, some managers are so distant from the work being performed by those that they manage that little to no important conversations ever take place – a strange dynamic. Too often, people complain that they only have meaningful conversations with their boss at the annual review. And I’ve been involved with management teams in companies where management clearly didn’t understand the specifics of the situations that their people were involved in, but they were very willing to weigh in on what they wanted to see happen. Of course, many times the direction was less than optimal, and certainly could have been much better if those in management had engaged in a dialog designed to truly understand the situation before making a call.

Performance, Productivity, and Results Defined

February 8, 2009

While you can argue that the terms “performance” and “productivity” are synonymous, I’ve chosen to put some strict bounds around the use of these terms. My main premise is built around the following sentence:

A distinction needs to be drawn between measuring productivity and what goes into achieving productivity.

Building on this premise, obtaining great results is a function of prioritizing work to provide the highest possible value. Otherwise, you run the risk of working on all the wrong things the right way.

A picture is worth a thousand words. The following illustration captures my thoughts on defining performance, productivity, and results:
The "Development Organization" box defines the company or business unit that is responsible for creating and delivering the software. External to the development organization is the customer, the entity that helps determine what the software should do and the one who expects to derive some benefit – value – from the use of the software. The arrow signifies delivery of this value to the customer, with everything flowing with, and an intrinsic part of, the total delivery picture.

The customer surrounds the development organization, not only receiving value but also participating at the beginning of the process, making the business case for pursuing a given project and providing valuable input that helps to shape the overall direction.

This is why the arrow starts and ends outside of the development organization box, and why the customer is viewed as surrounding the entire process. In practice, there should be an active partnership between the customer and development organization.

Starting at the bottom left and moving in the direction of the arrow, the Performance category captures the knowledge, skills, and abilities of individuals. Without good, capable, competent people, nothing else matters. This is input helps to drive the next category in the flow: Productivity.

Productivity is about orchestrating and leveraging peoples’ collective abilities and activities through teamwork, processes and tools. Teams comprised of skilled, collaborative people who are working efficiently can realize greater productivity than those working independently.

Ultimately, results are delivered. How much value that is obtained is a function of the return on investment. Ideally, people and teams should be focusing on and delivering work that provides the greatest possible value, and striving to deliver that value as efficiently and effectively as possible. Poor results can be a function of many things, such as performance and productivity problems that drive costs up, or even a consequence of working productively on all of the wrong things.

The arrow gets bigger as the flow progresses towards final delivery because there is a compounding effect as work flows. If the effects are positive in nature, the flow increases in size. If they are negative, flow is constricted and the arrow would shrink in size, gating your ability to be truly productive or deliver highly-valued results.

And there are other factors that affect productivity. Organizational culture is one example. A command and control, task-oriented, metric-heavy organization is not conducive to innovative thinking and can place limits on what an organization can achieve.

The framework that I’m discussing here is broad in scope, but each area can be defined and examined (and my intent is to do so in future posts). The following are some immediate thoughts that come to mind:

  • The business objectives that focus on delivering the highest possible value. How well-defined and communicated those objectives are, how well they match the organizational objectives in terms of providing value – or the potential for profit.

  • The organizational context that the development organization is a part of. Organizational norms, reward/recognition systems, communication and collaboration mechanisms between the development organization and the customers all influence productivity.

  • The knowledge, skills, and abilities of the people working in the development organization. This includes managers, project managers, quality assurance, developers, documentation, build and release engineers – anyone that is a part of the organization that delivers the final software product. Asessing and managing individual performance is a vital area, and often done poorly.

  • Processes and tools used to monitor and facilitate the work. Regardless of what process you use, examining how well you are executing and whether there are impediments that need to be addressed is an on-going process.

  • Metrics to measure and compare yourself against industry norms or an internal baseline are important. You can’t know how well you are doing without some measurements in place.
As always, I welcome your comments and invite anyone to contribute to this blog if you feel so inclined.

Software Results Redux – Would You Like to Contribute to this Blog?

February 7, 2009

I started this blog in January, and I had certain objectives in mind that I will share with you. Consider this post something of a re-launch effort, as I’m feeling the need to put a framework and definition around this blog based on early feedback and some reflection on my part.

I began this blog to explore the topic of being as productive as possible and generating true value with software projects. This blog is not about my staking a claim as an expert, but I have been in the industry for some time now and I do have a couple of objectives in mind with this blog.

First, I feel that there is too little in the way of good information from the trenches in terms of software productivity. Many times, productivity gets expressed solely in terms of fairly simple – but incomplete – metrics. This leads me to the second reason for starting this blog.

As a development manager, I wanted to force myself to think about how I approach managing software development in order to maximize results. I continually poke around book stores and then Internet, but it seems to me that there is a major problem lurking in software today that is largely unaddressed: There are either managers out there who do not have a software background – and thus don’t really understand what they should be focusing on – or there are managers that come from programming who are unprepared for a management role.

A second problem spot is the interaction between business users and the development organization. A common situation in many organizations is that the business is driving IT in general, but the business community lacks an understanding of just what they are driving. I’ve heard a couple of common refrains all too often when dealing with people who don’t understand the software business: “I don’t know what’s taking so long…” or “It should be a simple thing…”

Hence, the mission of this blog: “A blog about achieving productivity and generating value in the software world.” My goal is to provide myself (and hopefully others) with a deeper, truer understanding of what it takes to be productive, how to measure and gauge that productivity, and how to explain it when necessary.

Writing is a great way to clarify your thinking, and this blog forces me to think about the specifics and to clearly articulate those specifics. It also permits me to gain insight from others who comment on my blog posts, whether they happen to agree or disagree with me.

Since I’m not claiming to be an industry guru, I’ll put an open invitation on the table. If any readers would like to be a guest blogger and contribute material, shoot me an e-mail at: Use the subject Guest Blogger and provide me with a short description of yourself and what you would like to cover. I’ll be delighted to add you as an author!

I’ve got another post I’m finishing up right now, and I’ll post it tomorrow. In it, I’ll put my definition around Performance, Productivity, and Results.