Agile Development and the Realities of Software Development

July 30, 2010

In my post, What is Better About Agile Development? (a reprint of the article Top 10 Reasons to Use Agile Development that I wrote for DevX.com), my tenth and final reason for using Agile is that Agile addresses the realities of software development and the needs of the business. This post will expand upon this topic.

One reality of software development is that there is still too much unsustainable development in the industry as a whole. Squeezed projects with significant overtime and stress prevent us from getting smarter, let alone working smarter. Technology and business are continuing to change at ever-increasing rates; keeping pace with the business while keeping our skills current and relevant is a major challenge, yet essential if we are going to improve productivity. Through sustainable development and continuous improvement, Agile development helps teams and individuals do both.

A few tangible benefits associated with Agile development that meet the realities of both software development and the business – contrasted against traditional, plan-driven development projects – can be categorized broadly as: The early bird gets the worm.

Software project teams need feedback about features from the users. No matter what documentation was prepared in advance of development, there is nothing like having the users seeing the working software to obtain final, definitive agreement on whether the software meets their needs. Signing off on a specification and “building to the spec” might meet a project measurement, but it won’t necessarily satisfy the customer.

Why? In my experience, I’ve found that users aren’t particularly good at dealing in the abstract, or at least they aren’t as good at dealing in the abstract as software developers. They need to see things, not conceptualize them. That, coupled with the fact that communication between individuals is always subject to misinterpretation at some level, leads to the ever-present “requirements problems.”

I’ve heard the phrase, “Now that I see it…” quite often over the years, followed by a change request. Agile development’s frequent delivery of features enables users to routinely inspect the (working) software to provide immediate feedback. And because the highest-priority features are delivered first, software project teams get early feedback on the most valuable business features, allowing for course-corrections to occur early, instead of late, in the project.

Because the highest-priority features are being worked to completion early, the business can benefit from an early return on investment. With traditional, plan-driven projects, software is delivered as a complete package, but the business must wait until the end of the project to begin realizing a benefit.

Another, related benefit to using Agile development is that valuable time is not spent on the details of features that may or may not be a priority later. With traditional, plan-driven projects, you run the risk of investing time and energy into defining and designing features that may not be desirable or even needed. Time is money, and I prefer not to waste either.

Early delivery, early feedback, and an early return on investment; the early bird (the business) does get the “worm” with Agile development – or is it really a caterpillar that morphs into a butterfly in the cocoon of the Agile team?

Another reality is that business is rapidly changing. As Faisal Hoque points out in an article, The Speed of Business Today, “Constant change is the new dynamic of the global economy, and makes agility even more necessary than at any point in business history.”

Software development should align itself to the needs of the business by being able to respond change, and Agile development does just that. In the 2009 State of Agile Development Survey sponsored by VersionOne, the number one benefit obtained from implementing Agile development was the ability to manage changing priorities. 90 percent of the respondents said implementing Agile either improved or significantly improved their ability to manage changing priorities. (The survey data included 2,570 participants in 88 countries.) An older survey, the Shine Technologies 2003 Survey, was another global survey of actual experiences using Agile development. The highest ranked positive feature reported in the survey was the ability to “Respond to change over plan.”

In terms of general alignment between the business and IT, the 3rd-highest benefit obtained from using Agile development reported in the 2009 State of Agile Development Survey was an Improved Alignment Between IT and Business Objectives. This is supported by the Shine Technologies 2003 Survey, which reported that 83 percent of those surveyed stated that business satisfaction was better or significantly better.

Satisfaction can be a function of a variety of factors, but one reality is that software is being designed and built for the customer. People like to feel involved, that their needs are paramount in your mind. Traditional, plan-driven projects “respond” to the customer, but usually with change control and a “here’s how the project will be impacted” assessment. A rigid, process-oriented approach isn’t bad, but it can develop into an almost adversarial relationship where the customer feels that they are the one that must press for changes to their software.

This is because traditional, plan-driven projects have a focus on “planning the work and working the plan” where delivering to the specification is the key measurement. While there are project managers who run their projects as a series of negotiations (which helps maintain the relationship between the team and the customer), the very nature of the process causes people to lock in on what is defined – and change is becomes disruptive.

Agile development is designed to handle change; people don’t become wedded to a plan because they haven’t invested a lot of time and effort defining, estimating, and designing everything prior to actual construction. Agile teams wait until the last responsible moment to begin diving into the details of any one feature. And features that haven’t been started yet are considered negotiable at all times. This allows the business to change its mind with ease. And because Agile teams work with the customer throughout the project, the customer feels involved and valued throughout the entire process.

Agile development also stresses the need for the use of sound technical practices as a means of improving productivity. While you don’t have to be on an Agile team to make use of these practices, Agile development promotes them. Automated testing, Test-Driven Development, continuous integration are all geared towards improving throughput without sacrificing quality or sleep.

My final point, and final reality, is that software professionals are all adults, and they should be treated as such. Agile development, with its autonomous, self-directed teams allows people to define and manage their own work, which provides a motivational – and productive – boost.

Dan Pink, in his book Drive: The Surprising Truth About What Motivates Us, cited a study conducted by researchers at Cornell University that examined 320 small businesses, half of which granted the workers autonomy, the other half relying on top-down direction. The businesses that offered autonomy grew at four times the rate of the control-oriented firms and had one-third the turnover.

Top-down, control-oriented management brings its share of problems. In his book, The Way We’re Working Isn’t Working, Tony Schwartz puts this into solid perspective. Schwartz states, “The all-too common dynamic is today’s workplace is parent-child. Most employers tell employees when to come to work, when to leave, and how they’re expected to work when they’re at the office. Treated like children, many employees unconsciously adopt the role to which they’ve been consigned. Feeling disempowered and vulnerable, they lose the will and confidence to take real initiative or to think independently. Doing what they’re expected to do often becomes more important than doing what makes most sense.”

Command-and-control management creates the need to manage people more closely! Is that what we want? I know that I don't. Agile development's final contribution to the realities of software development is that it provides the opportunity for everyone to be treated as valued, professional, adults. That works for me!

Agile Development is Motivating and Engaging

July 27, 2010

In my post, What is Better About Agile Development? (a reprint of the article Top 10 Reasons to Use Agile Development that I wrote for DevX.com), my ninth reason for using Agile is that Agile development is motivating and engaging. This post will expand upon this topic.

My position is that developers are knowledge workers, and as I pointed out in my article, “knowledge workers have the greatest understanding about their own work, and that they are the ones best qualified to plan and organize how they will accomplish that work.”

I’m not alone in this observation. In their book, High-Performing Self-Managed Work Teams by Dale E. Yeatts and Cloyd Hyten, Yeatts and Hyten stated, “Case studies have shown that the decisions made by SMWTs (Self-Managed Work Teams) are extremely effective because those making the decisions – the team members – are the most knowledgeable persons about the work.” (Buchholz, Roth & Hess, 1987; Ray & Bronstein, 1995).

Not only are those decisions effective, but allowing teams to operate autonomously – self-directed – is a great motivator. Why do so many people seek to start their own business? Money, yes, but another very common reason is autonomy. People want to live their lives the way they want to; they don’t want a boss telling them what to do, when to do it, and how to do it. The control factor present in most workplaces is a major factor in the decision to go it alone.

Tony Schwartz discussed this control factor in his book; The Way We’re Working Isn’t Working. Schwartz states, “The all-too common dynamic is today’s workplace is parent-child. Most employers tell employees when to come to work, when to leave, and how they’re expected to work when they’re at the office. Treated like children, many employees unconsciously adopt the role to which they’ve been consigned. Feeling disempowered and vulnerable, they lose the will and confidence to take real initiative or to think independently. Doing what they’re expected to do often becomes more important than doing what makes most sense.”

As a manager, I certainly want people thinking independently and taking initiative. I want people to do what makes sense, and by all means avoid following a procedure because “that’s the way we’re supposed to work” when that procedure doesn’t apply! I want people to understand our business goals and to think and act in accordance with those goals.

A great treatment on the subject of engagement and motivation is given by Dan Pink in his book Drive: The Surprising Truth about What Motivates Us. He relates a theory put forth by Edward Deci and Richard Ryan called “self-determination theory” which argues that we have three innate psychological needs – competence, autonomy, and relatedness – ingredients that Agile development endorses. Deci and Ryan feel that when these needs are met, we are happy, productive, and motivated. (No argument from me!)

In support of productivity gains through autonomy, Dan Pink cited a study conducted by researchers at Cornell University that examined 320 small businesses, half of which granted the workers autonomy, the other half relying on top-down direction. The results were fantastic! The businesses that offered autonomy grew at four times the rate of the control-oriented firms and had one-third the turnover.

While I'm on the subject of engagement and motivation, good morale is vital to positive motivation and engagement. Studies support that adopting Agile development definitely improves morale. The 2009 State of Agile Survey by VersionOne lists Improved Team Morale as the fourth-highest benefit obtained from adopting Agile development. Mike Cohn reports in his book, Succeeding with Agile: Software Development Using Scrum, that fifteen months after adopting Scrum, Salesforce.com surveyed its employees and found the 86 percent were having a “good time” or the “best time” working at the company. Prior to adopting Scrum, only 40 percent said the same thing – a solid improvement to say the least!

A couple of quotes that that capture the concept of autonomy, motivation and engagement:

“Mediocrity is expensive – and autonomy can be the antidote.” – Tom Kelley, General Manager of IDEO

“Traditional management is about compliance. If you want engagement, self-direction works better.” – Dan Pink

Finally, Agile development provides a sense of accomplishment that is very motivating. Through the delivery of working software early and often, there is a sense of meaningful accomplishment on a regular basis. There is also a feeling of belonging and relatedness due to the highly collaborative nature of Agile development.

I’ll summarize by quoting from my article. “This control over their workday, plus operating in a sustainable mode and the feeling of accomplishment that is a by-product of continuous delivery of working software, all combine to provide a solid motivational boost to each and every person on an Agile team.”

What is a manager to do? As Josh Leibner, Gershon Mader, Alan Weiss observed in their book The Power of Strategic Commitment: Achieving Extraordinary Results Through Total Alignment and Engagement, “Outstanding managers can create environments conducive to people motivating themselves.” Agile development creates this very environment – provided that you (as a manager) and your organization endorse and support it.

Agile Development: Continuous Improvement Is Expected

July 23, 2010

In my post, What is Better About Agile Development? (a reprint of the article Top 10 Reasons to Use Agile Development that I wrote for DevX.com), my eighth reason for using Agile is that, Agile development expects continuous improvement. This post will expand upon this topic.

As I noted in my article, “Agile development expects its practitioners to be continually learning and adapting…” This concept is articulated in the final principle of the Agile Manifesto, which states:

“At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.”

As a manager, I want to see everyone to improve and grow, and i feel that continual learning is vital in the software business. If you aren’t learning and growing (you’re “maintaining”), you are effectively losing ground. The world will move past you.

It is equally important that my development organization is utilizing the best practices possible to make individuals and teams as effective and productive as possible. Agile development actually makes a manager’s job easier with respect to learning and improving people and the software development process. You don’t have to explicitly create a “learning opportunity” for your staff (carving out time in busy schedules); the opportunity is created through the Agile process itself.

As I stated in my article, continuous improvement is enabled in part through sustainable development. Improving productivity requires that people grow and change, and in order to accomplish this, people need time to learn, think and reflect. Another component of improving productivity is getting people to accept change. The best way that I know of to accomplish this is to have them initiate the change on their own.

The other enabler is the individuals themselves. People must have the desire to actively learn and grow; they must value learning and be willing and able to act on what they learn, improving their personal and/or team productivity. By applying what they learn and then thinking and reflecting on how well it worked (and tweaking where necessary), new knowledge and expertise is developed. This is a true learning organization at work.

To facilitate this, Agile development incorporates the concept of slack. As James Shore and Shane Warden described it in their book, The Art of Agile Development:

“To keep up with their constantly expanding field, programmers must continually improve their skills. In doing so, they will often learn things that enhance their work on the project.

“Dedicated research time is an excellent way to encourage learning and add additional slack into your iterations. To introduce it, set aside half a day for each programmer to conduct self-directed research on a topic of his choice. Be completely hands-off. I recommend only two rules: don't spend this time on project stories or tasks, and don't modify any project code.”


I’ll make one point about the first sentence, which in part states, “…they will often learn things that enhance their work on the project.” Some people are better learners than others, and often times it is better to learn about specifics that are directly related to the work that is being performed or is about to be performed then something that you might need to know about in the future – there is direct applicability and reinforcement of the learning. It’s an application of the age-old, “use it or lose it” advice.

Team retrospectives are another way that Agile development fosters improvement. A retrospective is a meeting where a team looks back on their past iteration so that they can learn from their experience and apply this learning to future projects. In contrast, traditional, plan-driven projects normally hold something like a “post-mortem” at the conclusion of the project, when it is too late to improve that project.

I like this continuous improvement cycle of Agile development, not only because small improvements are being routinely made, but because the changes are automatically bought into by team members because they are the ones initiating the change. I also like regular retrospectives because they are also a great time to reflect on what went well, and what should be celebrated. Celebrating success helps keep people energized and engaged.

When it comes to improvement, some key points about retrospectives that I feel are important:

No blame. Retrospectives shouldn’t look to lay blame, but to examine – professionally and depersonalized – what needs improving.

Focus on what is in the team’s control. A team retrospective that focuses on exclusively that is outside of their control isn’t particularly useful to the team. Sure, maybe the team could have used an extra resource or two. The question is, did the team maximize the use of everyone? What or how could the current team have done to work more productively?

Drill down to actionable tasks. Don’t settle for high-level statements of a problem and/or vague notions of what could be done. Decompose the issue into actionable tasks that could be taken on in the next iteration.

Prioritize and select a task. A team doesn’t have to take on a suite of improvements all at once. In our organization, we use two-week iterations, so every two weeks there is an opportunity for improvement. Teams need to ask themselves, what would have the greatest impact on productivity for the team?

There are some managers who get uncomfortable with the concept of slack during an iteration and time dedicated to routine retrospectives. They don’t feel that they are maximizing productivity because team members aren’t engaged in “producing” software on a constant basis. This is counter-intuitive, in my opinion.

If you want productivity gains out of your knowledge workers – increasing the capacity of your existing workforce – then those people need to put knowledge into their knowledge banks. Knowledge workers cannot improve their capacity without investing time and effort to do so. This is a two-way street, however. The organization must recognize and provide the opportunity for this to happen, and the employees must take responsibility and take the initiative to follow through.

Agile development creates this opportunity. And it is up to the teams and the individuals on the teams to exploit the opportunity that is presented to them.

Building Trust Through Agile Development

July 19, 2010

In my post, What is Better About Agile Development? (a reprint of the article Top 10 Reasons to Use Agile Development that I wrote for DevX.com), my seventh reason for using Agile is that Agile development builds trust and relationships. This post will expand upon this topic.

My article highlighted two key areas where trust and relationships are built using Agile development:
  • Between the team and the business (customers and/or stakeholders). 
  • Between members of the team.
Building Trust with the Business
James Shore and Shane Warden explained this issue in their book, The Art of Agile Development:

“Many organizations I've worked with have struggled with an ‘us versus them’ attitude between customers and programmers. Customers often feel that programmers don't care enough about their needs and deadlines, some of which, if missed, could cost them their jobs. Programmers often feel forced into commitments that they can't meet, hurting their health and relationships.”

The business side has its drivers; sometimes they are the ones dealing with an imposed date – like meeting regulatory requirements. Other times, the business is responding to a competitive threat where delay is costly in very real terms, such as revenue and market share. And in other cases, there is an opportunity that the business wants to capitalize on to drive growth – a “first to market” scenario. Whatever the situation, the DATE that customers have in mind is very real.

On the software project side of the fence, other dynamics can come into play. In response to the business pressure, for example, a software project team can be driven into “optimizing” a tradition, plan-driven project before the project has even started. Why? Because the schedule – and indirectly, the team – isn’t trusted. Typically, it is management that is examining project schedules, and they are usually responding with one or more of the following motivations:
  • A strong desire to complete the project as quickly as possible so that the team can move on to other, pressing projects.
  • Pressure to meet the business needs within certain timeframes to win the business and drive revenue (or prove the worth of an in-house organization).
  • A strong desire to ensure that the team is motivated by “just enough” schedule pressure to work up to their potential.
The reality is that everyone is concerned about whether the project team can deliver working software even close to the expected timeframe and anticipated budget. Documents and plans are fine, but it is the long gap between project inception and the first delivery of working software creates distrust with traditional, plan-driven projects.

Let’s face it, after months of dialog, reviews, and eventual sign-off to “get the requirements right,” customers are still going to feel uneasy, particularly in the early software construction phases when there is nothing to see. And despite everyone’s best efforts to fully comprehend the needs of the business, there will be questions during the development of specific features, and some of those questions will reveal changes – or even new features – that should have been a part of the requirements, but weren’t.

What can happen next is “the case of the imposed date” from a development point of view. The customer thought that they had expressed what they needed to development, and development provided an estimate. The problem is, as the team gets into the project, a deeper understanding occurs and the original estimates are in need of revision.

It is at this point when someone (usually a manager) reminds the team that they created the estimates – which are now considered cast in stone and not the approximation that they really were – and the team is held to its “commitment.” Doubt starts creeping in, and the question that is on everybody’s mind is: Will this project make it?

These days, it is no secret that a great many projects suffer schedule overruns. And when projects become challenged, conversations invariably begin about trimming features from the planned delivery or extending the delivery date, even before the customers have seen much – if anything – of the software that is under construction. At this point the customer’s fears are realized, and trust takes a hit.

If customers can see it, they can believe it – and ultimately trust those producing it. This is precisely the point that my article made, the benefit of trust that Agile development provides: “Through early and often delivery of working software to the business – even if this is nothing more than demonstrating the features – the business will grow to trust the development team. In addition, the continual dialog and the ability for the business to adjust and adapt the software gradually changes the ‘us versus them’ dynamic into a ‘we.’”

Referring back to The Art of Agile Development, James Shore and Shane Warden summed this up nicely: “Stakeholders don't know how to evaluate your process, but they can evaluate results.”

Customers don’t understand all of the ins and outs of software development, but they do measure us. Their ultimate measure is the end result – working software that either meets or fails to meet their needs. Their satisfaction can be – in part – a direct result of how much participation and input they experienced during the software project.

This is where Agile development lends itself to establishing trust and relationships. Agile development expects and embraces continual customer involvement with the development team. Instead of a large amount of time spent up front defining and refining all of the requirements, Agile development spreads this involvement out over time, accomplishing the same goal, but in different ways. Agile development also includes regular iteration reviews that demonstrate new features to the customer – a very visible display of progress that is much more convincing than reaching a milestone marker on a project plan.

This increased trust and improved relationship provides a couple of other benefits. If the customer wants add new features that affect the schedule, the trust and relationships that are built allow for candid conversations about what the customer will receive and when, including discussing trade-offs because new features are being added.

Even if the development team is struggling with a feature that is proving to be more complex than anticipated, the visible progress and constant contact with the team makes the “we’re struggling to deliver this feature” conversation is much easier to have. Because of the connected nature of the relationship, the customer has a greater appreciation of the effort involved.

The other benefit of trust and the frequent delivery of working software is that a significant amount of documentation is simply not required. In sequential projects, mounds of documentation exist to inform and guide development efforts because it is assumed that conversations will not occur. The requirements document is intended to capture all that there is to know prior to it being handed off to development. Continual conversations cut down on the need for comprehensive documentation; and contrary to what some contend, Agile development does NOT mean no documentation.

Building Trust between Team Members
The traditional, plan-driven approach to development organizes and optimizes its work functionally – with formal hand-offs between functions. People can start to identify with their area of specialty and not with the project or worse, not with the customer. The notion of specialists and optimizing work isn’t bad, and this model does work. Given that work centers around human beings, problems can and do arise.

When I first became a development manager, one problem that our organization had was that there was a wall between Development and QA. I worked hard with the QA manager and everyone involved, breaking those barriers down.

This involved coaching – casting QA as a benefit by focusing attention on the fact that our job was to prevent defects from getting out the door and into our customers’ hands. I began working on setting higher goals for our developers, asking more in terms of design, code reviews, unit testing, and the like. The QA manager worked at raising the QA game and pointing out that developers weren’t being sloppy just because bugs were showing up – at least not if they were using sound practices like conducting design and code reviews.

We also held project meetings that fostered and encouraged more open dialog. Eventually, everyone began to gain an appreciation for the skill and work required on both sides of the fence, and despite that fact that at the time we were still a plan-driven, functionally-oriented shop, the teamwork and rapport significantly improved between the Development and QA organizations.

Agile development takes building teamwork and rapport to a higher level. Project work matters more than a functional role, and as I noted in my article, “Agile teams make the most effective use of the team members as dictated by the needs of the team to meet its commitments. The shared goal becomes more important than each individual working strictly within his or her area of expertise, with defined policies and procedures for handing off work between functional roles. This breaks down barriers between functional disciplines, enabling the team to reach higher levels of productivity.”

This means that a developer can and should help test (particularly if that testing involves creating automated tests) to help the team finish its work. Meeting an iteration commitment shouldn’t be an individualized, “I’ve completed my tasks,” but a, “How is the team doing in meeting our commitments?” With Agile development, there is a strong expectation that the work is all about the team getting across the finish line together.

Because everyone is working closely together – preferably co-located – and focused on completing the team’s highest-priority items first, trust is established through continual interaction and delivery. The daily stand-up meeting reinforces individual commitments to accomplishing meaningful work each and every day, and the expectation that Agile development places on continuous team and individual improvement builds trust as team members and teams improve their game.

Agile Development Enables Emergent Innovation

July 16, 2010

In my post, What is Better About Agile Development? (a reprint of the article Top 10 Reasons to Use Agile Development that I wrote for DevX.com), my sixth reason for using Agile is that Agile development enables emergent innovation. This post will expand upon this topic.

It is no secret that companies desire greater innovation to differentiate themselves in today's marketplace. Emergent innovation is all about deriving innovation organically from within the organization as a natural course of performing day-to-day work.

I’ll admit that this particular reason for using Agile development was of my own choosing, and I cannot point to specific studies that support this as a benefit. Since I was responsible for writing the article, I wanted to give my opinion on a benefit that I feel should be a part of Agile development.

Step One in enabling emergent innovation is to provide a sustainable development environment. As I noted in my article, “Teams that are working constant overtime to meet schedules simply lack the time or inclination to think about anything else other than the difficult schedule in front of them. In sustainable development environments, people have the time to think more about the business and explore – creating the potential for innovation that did not exist previously.”

Step Two is to have conversations as a team. Requirements (User Stories) should not be considered cast in concrete, but should serve as a basis for a conversation about what the business wants and the anticipated benefit(s) the business expects as a result. As I noted in the article, “…the dialog between business experts and the technical experts can yield unexpected results, like turning complex, difficult features into elegant, differentiating features.”

We’ve experienced this. We’ve had teams review User Stories and discuss how certain features could be implemented – a back-and-forth conversation about the business workflow and what was realistic for software to handle automatically along with how the user needed to interact with the software to accomplish his or her goal. We’ve been able to eliminate unnecessary steps and simplify the software while still providing a great user experience that met the needs of the business.

Sometimes, users are asking for features that are based on other software that they have experience with. The trick is they don’t always know how to separate how they perform tasks today with what is possible tomorrow. This is why understanding the anticipated benefit is an important component of the User Story, and why conversations between those who understand the business and those who understand software construction are highly beneficial. You can end up i a much better place than either side could have imagined by working in a sequential, isolated hand-off development approach.

Emergent innovation can also involve an entire product. We initiated one project that was little more than a vague notion of the actual problem that we wanted to tackle. This was a classic R&D effort where we gradually – and experimentally – tackled the issue of exposing complex, back-end insurance processing to web portals.

We initially felt that we needed to build a portal itself, but as the team immersed itself into the effort, a shift occurred. The result was a middleware set of tools and technologies that enable us to interrogate a back-end system and generate schema and meta-data that describe the workings of the back-end system. (And an even better, recognition for the developers in the form of a patent application.)

The result is that we don’t create web portals, we allow the definition and management of templates, rules, and workflows that govern the semantic display and behavior of information that can be contained within web portals. We make it easy to access back-end systems and propagate changes out to the web, with our middleware facilitating the dialog between the web portal and the back-end system(s). Our customers have full control over the look and feel of their web portals, leveraging our technology to make their lives a lot easier.

Naturally, this type effort required some of our best people. People who have a proven track record of creative thinking, the ability to deal with ambiguity, and that inquisitive, exploratory mindset that enabled them to define the goal and then work towards achieving it.

What are your thoughts? Does emergent innovation belong among the Top 10 Reasons for Using Agile?

Agile Development Should Be Sustainable Development

July 13, 2010

In my post, What is Better About Agile Development? (a reprint of the article Top 10 Reasons to Use Agile Development that I wrote for DevX.com), my fifth reason for using Agile development is that Agile development creates a sustainable development environment. This post will expand upon this topic.

As a software development manager, my responsibility is to make sure that our development organization produces working software day in and day out, year after year. And like many other managers out there, it seems that every year I’m being asked to do more with less. I can accomplish this in one of two ways:
  1. Squeeze my staff for all they are worth (and replace them out when they burn out).
  2. Work smarter.
I personally like creating a collaborative, fun, sustainable work environment that is focused on retaining good employees and not working them into the ground to meet imposed, aggressive (ridiculous) deadlines that are based on little more than hopes and dreams. Call me crazy, but I like treating professional knowledge workers as professionals. And I enjoy seeing people develop and grow – it’s truly rewarding to see people reach new heights!

Sustainable software development is all about meeting the needs of the business on an ongoing basis, which is typically translated into meeting project schedules. There are other ways of meeting the needs of the business, which I discussed in Agile Development Embraces Business Agility. The issue of software project schedules and the usual overtime response in order to meet those schedules is worth examining in term of productivity and results.

In my article I stated that, “Software projects rarely meet their initial schedule. Overtime with the mandate to ‘work harder’ is frequently used as it becomes apparent that a project will be running longer than anticipated.”

This has been an all too-common experience for many software professionals, and a great deal of anecdotal evidence and various studies that support that schedule overruns are frequent:

In a Dynamic Markets Limited Independent Market Research Report commissioned by Tara Consultancy Services in August 2007, 62 percent of the IT projects failed to meet their anticipated project schedule.

The latest Standish Group report in 2009 report shows that only 32 percent of IT projects were considered successful – meeting the schedule, budget, and delivered with the expected features.

In his book, Software Runaways: Lessons Learned from Massive Software Project Failures, Robert Glass found that schedule overruns were very common, pegging schedule overruns at 89 percent. (The book profiled 16 of the largest and most publicized software failures of the 1990s.)

Using overtime as a mechanism to deal with software project schedule issues creates far more problems than it solves, and generally inhibits the creation of a viable, productive, long-term organization. The longer-term risks of burnout, turnover, and overall sustainability are commonly acknowledged (and frequently ignored), but the short-term effects are rarely mentioned.

In the first place, the productivity of knowledge workers (like programmers) plunges with heavy doses of overtime. Research from The Journal of American Epidemiology – studying people who work greater the 55 hours – found that individuals had problems with memory, problems with functioning appropriately and problems with reading and comprehension.

From a software development standpoint, a major short-term issue associated directly with overtime is quality. In a paper, Impact of Overtime and Stress on Software Quality by Balaji Akula & James Cusick, Baleji and James focused on the impact of project teams working on an aggressive schedule, studying four projects over a two-year period (three of which had an aggressive schedule).

The chart below demonstrates a dramatic difference in defect rates when no overtime is involved:


Project #1
Project #2
Project #3
Project #4
Estimated person hours
784
416
416
528
Overtime person hours
175
167
0
198
Defects
2038
1545
142
2000

I’m quite sure that I know why defect rates are low when no overtime is present; I’ve experienced this many times as both a programmer and as a manager. Unrealistic schedule pressure and overtime not only introduces fatigue that produces more defects, programmers stop thinking as deeply and carefully about their work in response to the pressure to meet the deadline. Corner-cutting begins to creep in at every turn as output becomes a major focus – at the expense of practices that help to produce more reliable code.

Overtime has other problems as well. Since programmers are knowledge workers, they need to putting knowledge into their “knowledge banks,” something that will pay dividends later. Routine overtime on routine project tasks eliminates the time for the professional growth that will be of future benefit. (And in other cases, people are doing themselves a great disservice by failing to continually acquire new knowledge, as Pawel Brodzinski pointed out in a post, People Don’t Want to Learn.)

Then there is the whole work/life balance issue. My wife, Lauri-Ann, has referred to herself as a “software widow” at times when I’ve been very involved – and I mean working some very long hours – on software projects. You can tell when you are tipping the scales in favor of work too much when your wife pleads with you to put the computer aside to watch your daughter play soccer on a Saturday morning. (True story, Lauri-Ann almost had to hit me over the head.) In the long run, this isn’t healthy…

Sustainable Development
I support Agile development because it promotes sustainable development as one of its 12 Principles, as found in the Principles behind the Agile Manifesto. The principle states, “Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.”

Agile development doesn't try to predict the future by estimates alone, let alone try to make that future happen via overtime. With Agile development, the future can be predicted based on actual team velocity. Agile teams size User Stories in points and tracks team velocity – the User Story points that are completed in each 2-4 week iteration.

Stories are considered done when they are delivered as working software with high quality. Over a period of time, it becomes crystal clear how much work can be realistically accomplished by a team on a sustainable basis. Future delivery can be predicted by contrasting the team velocity against the active product backlog and the size of the User Stories contained in that backlog.

Once you have a clear picture of what is realistic and sustainable, how should you seek to improve team performance? It shouldn’t be about working harder, it should be about working smarter. How is the collaboration? Are there personnel issues or friction? Are practices such as Test-Driven Development being utilized? I’ve witnesses Agile teams working very productively, accomplishing more than they were when they were working overtime through the proper implementation of Agile development. The output was higher, it was sustainable, and morale was better.

When it comes to having developers maintain their skills and working smarter, one contribution that Agile development brings to the table (if you choose to use it) is the concept of slack as a technique to provide for research time each iteration. I first encountered this concept in the book, The Art of Agile Development by James Shore and Shane Warden. They state:

"To keep up with their constantly expanding field, programmers must continually improve their skills. In doing so, they will often learn things that enhance their work on the project.

"Dedicated research time is an excellent way to encourage learning and add additional slack into your iterations. To introduce it, set aside half a day for each programmer to conduct self-directed research on a topic of his choice. Be completely hands-off. I recommend only two rules: don't spend this time on project stories or tasks, and don't modify any project code."


What about teams meeting their commitments? Shouldn't they work overtime to meet what they committed to? If you have a team that struggles to meet its commitments every now and then, you actually have a great situation! It means that they are taking on work that tests and stretches them. It should be a judgement call whether to roll the work into the next iteration or deal with it now. It really depends upon the urgency and circumstances.

If there is a critical customer deliverable with a tight timeframe, a good team will recognize that and get the job done, and they will set aside their research time to make things happen. And yes, there are times that as a manager I've had to remind teams that a customer was waiting on us, and that yes, working over the weekend was necessary. There is nothing wrong with a spike of activity from time to time.

There are other circumstances when it is OK to let the work slide. Software development isn't highly predictable, and what appears simple when it is estimated can turn out to be more difficult in the actual implementation. In this case, why punish people by mandating overtime? Since it is a difficult problem, taking the extra time to deal with it – keeping people fresh at the same time – will improve your odds that it will be dealt with correctly the first time.

I've seen comments on LinkedIn and on other blogs that lead me to believe that there are some very wrong implementations of Agile development out there. Instead of sustainable development, the term "Agile" is being misused and abused to drive a series of short-term, "mini death marches" where the goal is to code quickly (without thinking) and to drive the output of the team in unrealistic and unhealthy ways. Unfortunately, this is the very issue that Agile development seeks to overcome.

Don’t be misled, though. Agile development takes discipline. It takes knowledgeable, capable, engaged people. It takes a supportive management team. It takes training and time. It is work and it is change, but in the end I believe that it is worth it. Regular use of overtime should be viewed as a serious problem.

Agile Development Increases Productivity

July 9, 2010

In my post, What is Better About Agile Development? (a reprint of the article Top 10 Reasons to Use Agile Development that I wrote for DevX.com), my fourth reason for using Agile development is that Agile development increases productivity. This post will expand upon this topic.

Before I begin, let me state that I understand that traditional projects can be successful and productive. Many people point out – rightly so – that many aspects of Agile development are not new. However, the question of productivity as it relates to Agile development is worth exploring. Even if you don’t want to be an Agile shop, perhaps there are aspects that you can adopt that will improve your situation…

Surveys and studies have attempted to determine to determine if Agile development is improving– or least providing a strong perception of improving – productivity. While everyone is not reporting productivity improvements, a high percent certainly are. Let’s take a quick look at some percentages:

82 percent reported a “somewhat higher or much higher productivity improvement” in Dr. Dobb’s February 2008 Agile Adoption Rate Survey Results.

73 percent reported “improved or significantly improved” productivity in VersionOne’s 2009 State of Agile Survey.

83 percent reported an improvement in productivity at Lockheed Martin, as reported at the Agile Vancouver: Much Ado About Agile, Sep 2, 2007 by Mike Zwicker in a presentation titled War Stories: Fighter Jets and Agile Development at Lockheed Martin.

Reaching further back, 93 percent stated that productivity was “better or significantly better” in the Shine Technologies 2003 Survey.

How is this happening?

Practices
Scrum is taking the lead as the Agile method of choice, but there is growing acknowledgement that a Scrum/XP hybrid provides more along the lines of total productivity improvement. My article focused attention primarily on the autonomous, teamwork orientation that Scrum provides – although I admit that I should have mentioned something about technical practices.

Some key practices of Agile teams employ that improve productivity:

Automated Testing: Ideally, these are small tests that are capable of being executed many times a day and can confirm that a newly-developed feature is working correctly as well as confirming that pre-existing features are continuing to function as expected.

Automated Unit Tests/Test-Driven Development (TDD): Agile development insists upon developer unit testing, and the best way to accomplish this is through automated testing. TDD is a “test first” approach where a unit test is written before the code is written; the test fails and the developer must code to make the test pass. This approach has the advantage of forcing developers to think more about their interfaces, helping to make the software more understandable and self-documenting.

We’ve been increasing our attention on automated unit tests, and the first benefit we obtained was that our developers were delighted that they could re-factor with confidence. And QA was more than grateful that automated tests quickly and easily proved that the behavior of the software did not change, with no manual testing required on their part. In the end, our velocity improved, allowing us to develop additional features that we could not have completed without the initial investment to build our suite of automated unit tests. (This investment was made gradually as we built out new functionality – although we did need to invest time in creating tests for code that was already created without automated unit tests.)

Code Reviews/Pair Programming: The benefits of code reviews have been known to the industry for many years; equally known is that the earlier a defect is detected, the cheaper it is to correct. Catching coding problems is best accomplished by a code review. Pair programming is one way to accomplish a real-time code review.

A recent Forrester study, The Value and Importance of Code Reviews, published in March, 2010 (commissioned by Klocwork) found that code reviews are considered to be of high value by those who conduct code reviews. The benefits cited:

  • Bugs are uncovered earlier in the life cycle.
  • Best practices are shared.
  • Refactoring and code simplification are encouraged.
  • Reduction in wasted time (later in the project).
There is a way to combine the development of automated unit tests and code reviews. We have not tried this out in our office (yet), but I did attend a recent training course where we conducted a side session that combined Test-Driven Development and Pair Programming. One programmer wrote the test, and the other programmer implemented the code. There was continual dialog about the business feature and the design and implementation as we worked through the example. The end result was a shared understanding of the code, solid, simpler code (we simplified the instructor's sample code), and an automated unit test. Does anyone have real-world experience with this technique that they would like to share on a guest post?

Continuous Integration: Instead of keeping portions of a system separate until the end of a project, Agile projects integrate and deliver working software early and continuously. This avoids problems of late-stage integration that some projects suffer, and helps to build a greater degree of quality into the system through the testing of features through the architectural layers.

Insistence on Quality: Agile development has a philosophy that quality comes first. Features should not be considered done until they are of high quality; defect-fixing is not an activity that occurs late in the project cycle. In the event that a defect is found with a feature that was declared as done, work on a lower-priority feature should be deferred in favor of making sure that the higher-priority feature is of high quality.

Teamwork
As I noted, my article focused on the team-oriented aspects of how Agile improves productivity. I made the statement that, “Agile development focuses the team's attention like a laser on delivering the highest-priority, highest-valued features in short increments of time using the most productive mechanisms to accomplish the work.”

Part of the “most productive mechanisms” are the practices noted above. Focusing on the highest-priority, highest-valued features is just that – a concentrated, team-based effort. We’ve found through our own experiences that teams should limit their work in process to maximize productivity.

We use two-week sprints our organization, and teams make commitments to accomplish certain work within that two-week period. It is tempting for teams to start as much of that work as early as possible, but I advise caution. There can be side-effects:
  1. Less collaboration occurs, and you end up with a team that is a collection of individuals working on their own tasks, leading to point number 2.
  2. Teams may end a sprint with a lot of work in process, but nothing actually completed.
Limiting work in process forces teams to truly discuss the highest priority items and finding ways to collaborate and truly share and own the work as a team. You might get pushback on this, as people tend to react with “there isn’t anything that we can all work on,” but there is usually ways in which people can collaborate.

We’ve also experienced the benefits of establishing a truly collaborative team environment. We had one team that did nothing other than change how they worked. We had a classic case of a “team” that was simply a collection of individuals working individually on their tasks. There was little to no collaboration, functional hand-offs, and plenty of overtime.

We started over by grooming the backlog and discussing – really discussing – each feature. We found that some features were vague, and others were unnecessarily complex based on the goals that the business had in mind. By having a real dialog with the Product Owner, we were able to make changes that met the business needs more effectively, and in a way that was simpler to implement from a development standpoint.

The day-to-day work of the team changed as well. Everyone tasked out each User Story, discussed the validations, and collectively estimated the work (including discussing any deviations in the estimates). And during the course of implementing a User Story, the team collectively discussed the impact of any change – yes, issues surfaced as the team got into the details of some features. But there was a collective understanding and full team ownership of each and every feature.

The result? Without adding or changing any technical practices, focusing solely on teamwork and building true collaboration, the team achieved a little over 30 percent increase in productivity. Our measure was based on the team’s estimate of when they would complete the work based on their old approach and velocity. We simply beat it, and this included eliminating the overtime that the team was working.

This brings me to another point that I made in the article. Productivity increase comes from teams operating autonomously, where the teams make informed decisions about their day-to-day work without the need for constant managerial direction. Not only does this expedite certain decisions by keeping them localized, employees who have more control over their day-to-day work are more energized and engaged about their work.

A final note about improving productivity through teamwork that I did not mention in my article (one that we’ve also proven in our experience), is that small teams are more productive. This has also been proven in other studies, such as those made by Quantitative Software Management (QSM), a company that maintains a comprehensive database of software development project metrics.

Here’s a pop quiz for you, based on QSM data: There are two software teams that must deliver a project with a size that is estimated to be about 40,000 lines of code. One team, Team Large, has 29 resources compared to Team Small, with 3 resources. How much faster do you think Team Large will be?

According to the QSM data, the difference would be a mere 12 calendar days! The other significant differences would be the fact that Team Large would consume 151 more person-months more Team Small, and Team Large (on average) would create six times as many defects! The additional lines of communication and overhead associated with large project teams create more problems than are solved.

Hyper-Productivity
Jeff Sutherland coined the term Hyper-Productivity, defining it as a 400 percent higher velocity improvement. It’s an impressive marketing term, but does Hyper-Productivity exist?

It’s possible, based on studies. In his article, An analysis of XP, TDD, Pair Programming, and Scrum (Using Real Options), Dr. David F. Rico arrived at the following figures:



Low
Median
High
Productivity Improvement
14%
122%
712%
Quality Improvement
10%
70%
1,000%
Schedule Improvement
11%
71%
19%

I don’t think that everyone can improve by 400 percent or more; my personal opinion is that this depends entirely on where you start from. If you have a good sense of software development and you are following a sound methodology that includes good technical practices, I still feel that you can improve, but not to a dramatic degree that places you in the Hyper-Productivity category. What are your thoughts about Hyper-Productivity?

Agile Development Reduces Risk

July 6, 2010

In my post, What is Better About Agile Development? (a reprint of the article Top 10 Reasons to Use Agile Development that I wrote for DevX.com), my third reason for using Agile development is that Agile development reduces risk. This post examines the topic in greater detail.

Every software project has risk associated with it, ranging from business, project, and technical risks. I’m sure that a variety of risks come to mind, but the key points are:
  • There isn’t a software development project that doesn’t face risk.
  • Reducing risk involves mitigation tactics on multiple fronts. 
The first paragraph in my article under reducing risk stated one problem with software development is the “delivery of software where the users ‘got what they asked for, but it isn’t what they wanted’ because the requirements were not understood in the first place.” I’ve experienced this myself more than once, and others that I’ve talked to over the years have the same experience.

Requirements problems account for approximately 30-40 percent of a software development budget, based on figures by Dean Leffingwell (reported in his book, Managing Software Requirements, 2003). This includes the experience that Salesforce.com reported at the 2007 Agile Conference, where Steve Greene and Chris Fry noted that Salesforce.com “…experienced problems that most plan-driven projects experience, such as late feedback on features at the end of the release cycle...”

Late feedback at the end of the release cycle isn't bad, unless the feedback is about what you don't have right. When project teams are faced with enough of these requirements problems at the end of a project, discussions typically begin to either push out the project date or to drop features in order to meet the date.

Using an Agile development approach, on the other hand, significantly reduces the risk of this problem from occurring. Because there is continuous involvement from the business, there is less opportunity for misunderstandings to occur. And because of frequent delivery and inspection, a requirements problem will be detected early in the project, allowing the business to weigh the cost and prioritize a requirements change against other features.

Even if a new requirement emerges (a.k.a., scope creep), Agile development accommodates this through rigorous prioritization of the features based on value. Everything can’t be a priority, but if a new feature is determined to be important and valuable mid-way through a project, Agile development allows the feature to be prioritized and worked – with lower-priority items moving down in the queue (backlog).

Controlling scope creep itself if managed through continuous involvement and dialog. The benefit of collaborative development and managing scope creep was written about in an article by Capers Jones in 1996 (Strategies for Managing Requirements Creep), where joint development between user representatives and development was noted to cut requirements creep nearly in half. It is interesting to note that Capers Jones was reporting this in context of Joint Application Development that has been around since the 1970s. While the concept of collaborative development is not new, but Agile development exploits it.

Another way to mitigate risk is to improve project visibility. Understanding exactly what challenges and obstacles that a project team needs to overcome, plus getting a realistic appraisal of their true progress reduces risk because it become possible to intervene to help the team. In the 2009 State of Agile Development Survey sponsored by VersionOne, the second-highest benefit obtained from implementing Agile development was improved project visibility. 83% of respondents said implementing Agile showed either improved or significantly improved project visibility.

We’ve used increased visibility to mitigate risk at my own company. Those of us on the management team formally review each project weekly with the Product Owners and Scrum Masters. We also make every effort to attend the daily stand-ups. This way, we can get a good sense for what is going smoothly and what is not – and we can take action to do whatever needs to be done to help the team move forward.

This counts for the smaller, day-to-day problems and the larger, “we’re not going to make when we thought we were” problems. For example, we encountered one problem where it was apparent that the team was simply too big that is was literally tripping over itself, so we split the team in two and assigned Agile coaches to each team and improved our velocity as a result.

As I noted in my article, the concept of delivering working software can keep schedules and budgets in check, because the software is usable from the outset. If the project begins to take more time than planned, the business has the business has the option of discontinuing making further investment and using what has been delivered to date. Cutting an Agile project short doesn’t mean complete failure. You have some usable software, if you choose to use it.

Again, because Agile development delivers early and often, it gets around another all too common problem: the last 20% of the project takes as long or longer to complete as the first 80% of the project. A number of reasons can cause this: poor up-front estimation, late-stage integration issues as the project team attempts to “bring everything together,” requirements problems, quality issues (because a suite of testing was waiting for a code complete phase), deployment problems, etc. Here’s a couple of links to those who talk about the last 20% occupying large amounts of project time:

http://www.sqlmag.com/article/sql-server/the-80-20-rule.aspx

http://stackoverflow.com/questions/608748/how-to-avoid-the-80-20-rule-in-software-development

In terms of studies that that specifically quantify the risk reduction component of Agile. The 2008 and 2009 State of Agile Development surveys sponsored by VersionOne, contains an actual risk reduction category. 65 percent reported a significant improved or improved risk reduction in project risk, which was consistent with the 2008 figure of 64 percent.

There wasn’t a specific explanation in the studies about what risk covered, but another report, Agile Development: Results Delivered (by VersionOne, 2007), based on their 2006 State of Agile Development survey notes the following about risk:

“Agile development provides empirical control processes required when precision is needed and complexity or change is at a high level. Regular assessments through daily meetings and iteration reviews provide visibility throughout a projects lifetime. As a result, risk is reduced.

“Because planning occurs every iteration, the project risk falls because teams see actual results and reprioritize work to align with the business goals on an ongoing basis. Agile projects can dramatically reduce risk through delivering working software that meets the business goals faster. In contrast, with traditional development processes, risk remains high throughout the development process and is not significantly reduced until project completion.”


In general, the continual delivery of working software and the highly collaborative, visible nature of Agile development reduces many of the risks that traditional, plan-driven projects face. Agile development may not always eliminate the risk, but it should allow you to detect problems earlier, allowing you to and the team to adapt.

Agile Development Embraces Business Agility

July 2, 2010

In my post, What is Better About Agile Development? (a reprint of the article Top 10 Reasons to Use Agile Development that I wrote for DevX.com), my second reason for using Agile development is that Business agility is embraced. This post will expand on the topic.

Wikipedia defines business agility as: “The ability of a business to adapt rapidly and cost efficiently in response to changes in the business environment.”

Given today’s global, competitive economy, the ability for the business to respond to change is critical. Does it feel that change is continuing to accelerate to you? If so, at what rate?

If you want an extreme take on the subject, Ray Kurzweil states in an article, Understanding the Accelerating Rate of Change that, “The whole 20th century, because we’ve been speeding up to this point, is equivalent to 20 years of progress at today’s rate of progress.”

Sometimes, the consequences of failing to adapt is failure of the business. Faisal Hoque makes the case for business Agility in an article, The Speed of Business Today, asserting, “Constant change is the new dynamic of the global economy, and makes agility even more necessary than at any point in business history.” Faisal then states:

  • Only 74 of the original 500 companies in the S&P Index are still on the list 40 years later―a mortality rate of more than 10 per year. 
  • The average life span of an S&P 500 company has steadily decreased from more than 50 years to fewer than 25. 
  • Projecting forward, it’s likely that only about one-third of today’s major corporations will survive as significant businesses for the next quarter century. 
I think that it is safe to assume that technological progress is due to slow at some point in time, but none of us really know when. Relating business agility back to software development, the observation that I made in my article was simple: “Software development practices shouldn't run counter to business needs by forcing the business to choose and adhere to a set of pre-determined features that will be delivered months, if not years, into the future.”

This is contrasting Agile development against a traditional, plan-driven software development approach. As the rate of business change continues to increase, if only seems prudent to utilize a software development approach that mirrors the needs of the business, particularly since the purpose of software development is to support the business. Agile development welcomes and adapts to change, which means that Agile software teams can adapt in accordance with continually changing business conditions.

My brother Brian made an interesting observation about software projects at his company (he works for a large company that I won’t name here). They struggle with containing costs and overall project throughput because they still use a plan-driven approach and they want everything “done right.” The problem is, doing things right in their context means adding features up front so that they “don’t have to come back to the project later.”

In an effort to nail every project and then move on, my brother’s company is costing themselves time and money. The notion of getting in and out quickly – failing fast in some cases – is not present. They COULD be introducing another problem for themselves as well. (If they aren't, others do have this problem.) In their attempts to be thorough, could be spending some of their valuable time building out features that they don’t really need.

Does this sound like your situation? If so, take a look at a widely-quoted study by the Standish Group, a typical system has a variety of features/functions that are rarely or never used:


The message here is simple: What you think you need at the outset of a project may not always be what you really need. Large, plan-driven projects with a large feature count will increasingly be at odds with the need for adaptability on the business side. A smaller project with a well-defined scope and feature set where you “get in and get out quickly” certainly helps, as does a methodology that supports adaptability.

Using an Agile development approach, the most valuable, prioritized features will be delivered in short periods of time with continuous involvement from the business. The business gets to use the software and provide feedback, both in terms of the actual features delivered and about what the future priorities should be. Agile development projects are thus able to easily adapt in accordance with changing business conditions.

In my own company, we frequently review priorities as a management team, and we’ve certainly seen our priorities change throughout the year. As time marches on, information becomes available, information that you did not or simply could not have at the beginning of the year. Did those prospects purchase our software? Did some new competitive threat emerge? In addition, getting software into the hands of the users as early as possible yields valuable feedback and can help identify new, valuable requirements, something that we’ve experienced.

Other companies value this capability as well. In the 2009 State of Agile Development Survey sponsored by VersionOne, the number one benefit obtained from implementing Agile development was the ability to manage changing priorities. 90% of respondents said implementing Agile either improved or significantly improved their ability to manage changing priorities. (The survey data included 2,570 participants in 88 countries.)

An older survey, the Shine Technologies 2003 Survey, was another global survey of actual experiences using Agile development. The highest ranked positive feature reported in the survey was the ability to “Respond to change over plan,” which came in a 47.3%.

Another great, real-world example of improving business agility is the story of Salesforce.com. Salesforce.com was growing rapidly, but it was becoming increasingly difficult for them to deliver. Salesforce.com experienced problems that most plan-driven projects experience, such as late feedback on features at the end of the release cycle, and long, unpredictable release schedules. The frequency of Salesforce.com’s releases went from 4 per year to 1, meaning that customers were waiting longer to get less from the company. This is far from embracing business agility!

Salesforce.com undertook an all-in, major transformation to Agile development. Within one year, Salesforce.com delivered 94% more features, and Salesforce.com calculates that it delivered 500% more value to its customers compared to the previous year. And within 2 years, Salesforce.com’s revenue had doubled. (These specific numbers are reported in Mike Cohn’s book, Succeeding with Agile: Software Development Using Scrum.)

I’ve embedded the complete Salesforce.com presentation by Steve Greene and Chris Fry that talks about Salesforce’s Agile transformation given at the 2007 Agile Conference so that you can view Salesoforce.com's story for yourself.

While traditional, plan-driven projects have certainly succeeded (and I’ve been a part of them), what are your thoughts on meeting the competitive demands of today’s world? Is software development adaptive and response enough? Is Agile development the right solution to this problem?