Agile Development Can Improve Your ROI

June 29, 2010

Introduction
In my post, What is Better About Agile Development? (a reprint of the article Top 10 Reasons to Use Agile Development that I wrote for DevX.com), I listed Superior ROI as the first reason for using Agile development. This post will explore the topic in greater detail.

I set a condition in my article that Agile projects provide a superior return on investment compared for other software projects where “the business is forced to wait for the completion of the entire project before it can begin deriving a benefit from that investment.” I’m therefore comparing traditional, plan-driven projects to Agile.

I recognize that there are hybrid solutions in use, but I’m using these comparisons to highlight the benefit of Agile development against traditional development in the purest sense – to examine what is working that we should all be considering, regardless of the development methodology in use.

Just to keep definitions clear, a traditional, plan-driven approach is one that operates with formalized phases of phases of requirements, design, construction, testing, and maintenance, completing each phase before moving to the next. These types of projects strive to be predictive. Conversely, Agile development is an umbrella term that encompasses several actual methods (XP, Scrum, etc.), but in general Agile projects are incremental and adaptive.

And the Survey Says...
Leading off, Scott Ambler’s December 2008 Software Development Project Success Survey specifically asked respondents about their ROI and included this in the results. Scott used a formula to derive an effectiveness score, and Agile development’s ROI score was 3.9 versus 0.2 for traditional development on a scale of -10 to +10.

In addition, a detailed treatment of various studies has been performed by David F. Rico in a couple of articles, and he includes a long list of references (and commonly cited sources) that serve as a handy reference if you want to examine a variety of Agile development studies for yourself.

In his article, What is the Return on Investment (ROI) of Agile Methods?, David Rico’s stated purpose was “to examine and identify the ROI of agile methods. More specifically, its purpose was to identify the ROI of using agile methods in their entirety, not just some of the tools and techniques of agile methods like pair programming.”

His conclusion: “The benefits of using agile methods range from 10% to 100% for increased cost-effectiveness, productivity, quality, cycle-time reduction, and customer satisfaction. The use of agile methods as a new product development approach does result in increased ROI. This begins to dispel the notion that agile methods result in lower ROI than traditional methods.”

In another article, What is the ROI of Agile vs. Traditional Methods? An Analysis of XP, TDD, Pair Programming, and Scrum (Using Real Options), David Rico’s goal was “examine scholarly studies of Agile Methods and survey the range of quantitative costs and benefits associated with the use of Agile Methods. Data were compared to costs and benefits of Traditional Methods such as CMMI®.”

David concluded that, “On average, studies of Agile Methods reported 29% better cost, 91% better schedule, 97% better productivity, 50% better quality, 400% better satisfaction, and 470% better ROI than CMMI®.”

He also made another good observation: “The latest trend is to mix-and-match Scrum and XP to tap into practices like Pair Programming (PP) and Test-Driven Development (TDD) to increase productivity and quality.” One Agile method alone is not the recipe for success, and like a lot of things, we need to look under the covers to discover what is really working. Some of these are practices that can be employed on any project.

The Numbers...
In terms of working the numbers, John Scumniotales has a nice post, Why incremental development is better - An ROI perspective, that demonstrates how delivering early and often improves ROI. John also provides an ROI/NPV Excel spreadsheet calculator that can be downloaded so you can play with the numbers for yourself, as your mileage may vary. John’s sample project contrasts a traditional (non-incremental) approach versus an Agile (incremental) approach:


Dennis Stevens made a solid observation in the comments section of John’s post, stating, “Some features are more valuable than other features… Higher earlier benefit actually creates a bigger spread.”

These comments address another observation that I made in my article and something that is important for improving results: “Agile does ask for discipline and participation from the business as part of the deal, such as rigorously prioritizing the features…” and, “In return, the business gets its highest-valued features delivered early…”

Our Experience...
We’ve experienced superior return at my own company as we’ve transitioned to Agile. (My posts don’t delve too far into the specifics of my own company to avoid any entanglements with competitive disclosure or breeching of confidences.)

In the past, we’ve used the traditional plan-driven development approach. We performed the classic requirements elicitation, refinement and development, and then moved on to design and build out the architecture in its entirety prior to plugging in the business features. I’ve been involved with projects that have required – based on the defined feature set and staffing – over one year to complete before we could release and begin generating a return.

In comparison, we initiated the development of a brand new product after our adoption of Agile development. This product was released much earlier, allowing us to begin generating returns much sooner than would have been possible with a traditional approach. In fact, we delivered this product to one customer before our first, official release. We’ve benefited greatly from an early return on investment along with obtaining early feedback.

The feedback that our customer provided us allowed us to make adjustments to features that we thought we had right, but didn’t. We thus obtained two benefits in one: we generated a return very early in the product cycle, and our official product release was stronger as a result.

We’ve also taken advantage of another aspect of Agile development: adaptability. We appeared to have a solid feature set defined, but feedback from customers have led us to adding new, compelling features and re-prioritizing our product backlog to work on those valuable features now. We aren’t measuring ourselves against “planning the work and working the plan”; instead, we’re adapting and responding to the needs of the market.

How Does Your Project Portfolio Look?

ROI from a single project is one thing, but many companies have a portfolio of projects, and it is worth considering what the return of the entire project portfolio. A failed software project obviously provides no return and can easily drain significant capital from a company, placing a greater emphasis and need for other projects to succeed.

Every year, the much quoted Standish Group Chaos Survey reports outright project failure or “challenged” projects – projects that fail to meet the classic definition of success in one or more ways, such as incurring a budget or schedule overrun, or having features trimmed from the final delivery, as noted in the table below:

2009
2006
2004
2002
2000
1998
1996
1994
Successful
32%
35%
29%
34%
28%
26%
27%
16%
Challenged
44%
46%
53%
51%
49%
46%
33%
53%
Failed
24%
19%
18%
15%
23%
28%
40%
31%

The Chaos Survey paints a dismal picture of the software industry, but it is worth noting that the very definition of software project success can and is disputed, as demonstrated in Scott Ambler’s December 2008 Software Development Project Success Survey. There were 279 respondents to Scott’s survey, and their definition of success in comparison to the classic definition is as follows:

  • On Time: 58% believe that delivering when the system is ready to be shipped is more important than delivering on schedule.
  • On Budget: 70% believe that providing the best ROI is more important than delivering under budget.
  • Of High Quality: 82% believe that delivering high quality is more important than delivering on time and on budget.
  • Delivered with the Expected Features: 83% believe that meeting actual needs of stakeholders is more important than building the system to specification.
Given that definition of success is disputable, it is certainly in the best interest of everyone to have a candid conversation about the definition of success before taking on a project! But the main point is that one disputes that delivering all of the software features, on time, with high quality, and on budget is a problem. Scott Ambler’s survey simply challenges the definition of success. (Projects that are challenged can also generate more than the anticipated benefits, but nothing is certain until your product is in use.)

Standish Chairman Jim Johnson was quoted about the improvement in success rates saying, "The primary reason is the projects have gotten a lot smaller. Doing projects with iterative processing as opposed to the waterfall method, which called for all project requirements to be defined up front, is a major step forward." (This comment was made prior to 2010, and I don’t have anything definitive about the dip in success rates. Does anyone have any insight on this?)

Focusing on the industry averages of Agile project success rates, the success rates are not perfect, but they are very good. The 2009 State of Agile Development Survey sponsored by VersionOne, includes data from 2,570 participants from 88 countries and reports that 23 percent of the respondents indicated that they had not experienced a failed Agile project. This is an improvement from 17.4 percent who reported no Agile project failures in VersionOne’s 2008 report, and in contrast to the Chaos Report’s downturn in overall software project successes.

I also took a look at the figures from Scott Ambler’s survey. Success in Scott’s survey was defined by the respondent, and not subject to a stricter definition such as the “on time, on budget, with quality and with the expected features” criteria used by the Chaos report. I’m fine with this, since a great many are arguing that the Chaos criterion is flawed.

I’ll stick to the Agile versus Traditional (plan-driven) comparison. In keeping with the spirit of “what’s working,” Scott’s survey included examining success rates by location – the closer the team, the greater the odds of success – summarized as follows:


Success Rates
Agile
Traditional
Co-located
79%
73%
Near located
73%
69%
Far located
55%
55%

Co-located means that everyone, including primary stakeholders, are in the same room.

Near located means that everyone is in the same geographic area, and could be on different floors, in different buildings, or working at home, but could potentially get together each day for a group meeting if need be.

Far located means that people would need to plane to attend a physical meeting of the entire team.

Overall, these studies indicate that nothing is perfect, but Agile improves your odds of success, and greater project success contributes to the overall return of your organization.

Conclusion
Summarizing what’s working in general for Agile software development projects, I’d have to say that the findings support that smaller, co-located projects that deliver incrementally allow a greater and earlier return on investment, and have a slightly greater chance of success than larger projects and/or distributed project teams. Finally, practices such as pair programming and test-driven development should be utilized to achieve productivity gains.

What is your take?

Agile Development: Where’s the Proof?

June 25, 2010

Have you ever participated on a forum or e-mail exchange that became a great example of terrible communication? This happened to me recently with on Reddit when I submitted a link to my post, What is Better About Agile Development? (This post was a reprint of an article that I wrote for DevX.com titled, Top 10 Reasons to Use Agile Development.)

There are times when face-to-face communication is much more effective, where a conversation only minutes in length can save hours of electronic chatter. This Reddit dialog felt like one of those times.

It started with this comment about my article: “Methodology marketing - no 'softwareresults' - no evidence, just assertion.”

I responded with, “It's an opinion about how Agile development can provide a number of benefits – and improve the results of software development efforts. I structured this piece with reasoning to support the assertions; but no, I did not provide specific evidence. Does the reasoning fail to convince you? What evidence would work for you?”

Things went downhill (and around the hill) from there.

I’ll admit that my motivation was to explore whether the assertions and brief reasoning offered in the article held up with someone who clearly:
  • Has experience in the software development field.
  • Reads a great deal on the subject.
  • Participates in forums and comments on blogs.
My attempts failed. My agenda was to explore my article on its own merit, while this individual wanted me to supply proof of my assertions. This individual eventually summed up my article with this: It's not even wrong.

I believe this to be a harsh judgment, and our dialog failed to convince me that anything that I said in the article was in fact wrong. This individual demanded evidence from me, but he didn’t present any evidence that my assertions were wrong based on research that he has clearly performed, either.

I do agree (in part) with his answer to this question that I asked: “How should the industry be approaching software development, in your opinion?”

He answered, “With humility and measurement.

"The Principle of Dispassionate Methodology: Don’t get your method advice from a method enthusiast. The best advice comes from people who care more about your problem than about their solution."

I happen to care about software development and meeting the needs of the business, and I’ve become enthusiastic about Agile development because I believe that when it comes to working with the business to meet the needs of the business, the approach is excellent. And I certainly don’t prescribe using Agile methods if they don’t fit your situation.

This individual was right about one thing. My blog is about software results, and I could back up my assertions more thoroughly than I did, and I will do so in my next series of posts. In advance of this, let me ask you a few questions so that I can learn to write more compelling and informative articles.

Please keep in mind that writing articles places some constraints on you as a writer. The editor had already determined the topic in this case. He wanted a “Top Ten” article (lists and “Top Reason” articles tend to be popular). There is a word limit. I was given a range of 800 words to 1600 words, and I came closer to the upper limit in this case.

I simply lacked the space to delve into details of each assertion. I had just enough room to explain each one as succinctly as possible. My hope for the article was that it would get someone excited enough to consider to want to explore more, or perhaps provide a convenient reference for those who understood and supported Agile development, but needed to explain the benefits of Agile development to others.

My questions to you:

In your opinion, does my article capture the benefits and reasons for using Agile, based on your understanding of software development and your experiences? Why or why not?

What would you recommend that I change?

What other articles would you like to see written?

Book Review: Get Rid of the Performance Review

June 22, 2010

Get Rid of the Performance Review!: How Companies Can Stop Intimidating, Start Managing--and Focus on What Really Matters I’ve been a software development manager for several years, with plenty of time invested each year on the annual review process. And like everyone else, I’ve been on the receiving end of the annual review process throughout my career. Being on the receiving end is the very foundation of the entire book!

Samuel Culbert points out that the typical performance review is not an interaction between manager and employee, that there is only one opinion that counts: the boss’s. Furthermore, the entire conversation is set up for failure, because the employee believes he or she is negotiating pay and readiness for advancement while the boss is focused on missed opportunities, skill limitations, and relationships that could use enhancing.

Culbert continues, noting that everyone lacks something as measured by other people’s metrics. Yet despite this lack, they are accomplishing a lot in their lives anyway. And when it comes to managing knowledge workers in particular, so-called “objective” reviews aren’t. You can get excellent and poor reviews by different managers, even if you are doing the same job. (Have you ever had this experience? I have.)

The result of this – and Culbert does a great job of articulating the problems – is that the performance review does exactly the opposite of its intended purpose. It actually prevents workers from improving. Culbert calls it “a dehumanizing process that leaves workers demoralized, unwilling and unable to address weaknesses. It makes them hate coming to work, let alone inspire them to turn themselves into better employees.”

One example Culbert uses is from an engineer, who said, “My desire to please everyone, to receive a stamp of approval from everyone, is my weakest point.” But, according to his boss, the problem is that he wastes too much time accommodating requests from other departments that should be doing their own work. In this guy’s mind, he’s just trying to help others out. But in his boss’s mind, he’s insufficiently focused. Who’s right? They both are. Everybody is right and everyone loses. There is no column on the performance review for “tries to please everybody,” but there is one for “lacks focus.”

What should be done instead? Culbert recommends using a performance preview.

The goal is to have an ongoing dialog, not a monologue, to align the manager and employee to accomplishing a common goal. This makes good sense to me. As a manager, why judge someone after the fact, from “on high?” Where were you before the act?

Some key observations about the differences between a performance review and a performance preview:

  • Performance reviews are one-sided-accountable and boss-dominated monologues. Performance previews are two-sided conversations, with both sides accountable. 
  • Performance reviews mean that if the subordinate screws up, the subordinate suffers. Performance previews put both the subordinate’s and boss’s skin in the game.
  • Performance reviews lead to bullshit. Performance previews lead to straight talk. 
  • Performance reviews allow the big boss to go on autopilot. Performance previews force the big boss to become involved.
  • Performance reviews create a competition between boss and subordinate. Performance previews create a team where both teammates inform and learn from each other. 
  • Performance reviews focus on deviations from some ideal as weaknesses. Performance previews celebrate differences.
  • Performance reviews are about comparing employees. Performance previews treat people as individuals. 
Another component of the performance process that I think is appropriate, yet a strong deviation from the standardized review process in most organizations is the notion that “one size doesn’t fit all.” If you read this book and you feel that changes should be made in your organization, be prepared that specific recommendations aren’t available. It will get you thinking about the need for change and provide some solid guidance on what change should look like, but there won’t be a new procedure for you to follow. As the book noted:

There is no magic formula, no silver bullet. Ordering people to use a procedure in lockstep fashion would be the kiss of death because you would be replacing an antiquated formula with a new mandated formula. If people don’t have to think about the need that generates their activity, mindless stuff happens. Each management unit and each manager must come up with a format that they own.

The book is well-written and really makes an excellent case for getting rid of the performance review! (Of course, this is easier said than done.)

Pausing

June 15, 2010

My recent posts have been made from a backlog that I established, but I'm beginning to run out. This is a temporary condition, one due to my father spending the last six weeks in the hospital, ultimately passing away. As a result, I am taking a short break until I can get back to writing more content. I have other material in various stages of preparedness, and I expect to continue in a week or so.

How Would You Define Agile development?

June 11, 2010

I recently participated in a discussion on LinkedIn, where someone asked, “Is Agile a methodology or a set of guidelines?” This provoked a number of responses, including:

“Agile is a framework rather than a methodology.”

“It is a set of principles rather than a methodology.”

“Agile is definitely just a framework.”

“Agile is not any single one thing.”

“Agile is not a methodology or a framework.”

“Agile IS 4 values and 12 principles, with frameworks that adhere to those Agile Values and Principles. “

So, just what is the proper definition of Agile development? Is it a framework, methodology, or something else? Is it nothing definitive at all?

Let’s start by examining what it means to be agile. Wikipedia defines business agility as: “the ability of a business to adapt rapidly and cost efficiently in response to changes in the business environment.” Wikipedia expands on this, stating, “Agility is a concept that incorporates the ideas of flexibility, balance, adaptability, and coordination under one umbrella. In a business context, agility typically refers to the ability of an organization to rapidly adapt to market and environmental changes in productive and cost-effective ways. The agile enterprise is an extension of this concept, referring to an organization that utilizes key principles of complex adaptive systems and complexity science to achieve success.”

Being agile in a business context equates to one of the primary benefits of Agile software development: that it strives to be responsive and adaptive to the business. While the purpose of Agile development may be well-understood; people get wrapped around the axel when they attempt to succinctly define Agile development, and disagreements surface when terms like “framework,” “methodology,” “guideline,” and “principles” become a part of the definition.

Wikipedia defines Agile software development as, “A group of software development methodologies that are based on similar principles.”

This is accurate, but it doesn’t quite work for me. I’ll come back to this definition in a moment. Let’s examine whether we should be using words like framework, methodology, guideline, and principles in the definition of Agile development. The formal definitions of each:

Framework – (Wikipedia was light on this definition, so I used Wiktionary.org) A framework is a basic conceptual structure used to solve or address complex issues.

Methodology – (Wikipedia) The study of methods used in a field.

Principle – (Wikipedia) A law or rule that has to be, or usually is to be followed, or can be desirably followed, or is an inevitable consequence of something, such as the laws of nature or the way that a device is constructed.

Guideline – (Wikipedia) Any document that aims to streamline particular processes according to a set routine. By definition, following a guideline is never mandatory (protocol would be a better term for a mandatory procedure).

Confused yet? I’ll keep going. Other definitions of Agile Software Development tend to be descriptive, like Scott Ambler’s: “Disciplined agile software development is an iterative and incremental (evolutionary) approach to software development which is performed in a highly collaborative manner by self-organizing teams within an effective governance framework with “just enough” ceremony that produces high quality software in a cost effective and timely manner which meets the changing needs of its stakeholders.”

Describing software development is fine. However, my objective is to provide a succinct definition, separating the what from the how, and to make a determination about whether Agile development is a framework, guideline, or methodology. In writing this blog, I’ve spent some time thinking about the terminology and my LinkedIn response, which in part was:

“Agile SOFTWARE DEVELOPMENT refers to a group of software development methodologies, whereas a specific implementation such as Scrum or XP IS a reference to a specific framework, methodology, or guideline (all of which I take to describe a process or approach to Agile development). I lean towards referring to something like Scrum as a methodology versus anything else. A guideline strikes me as too weak, and a framework invokes an image of static building-blocks rather than a process flow.

“My other two cents is that Agile development and the Agile Manifesto should not be confused, terminology-wise. But what the Agile Manifesto brings to the table is worth noting. The Agile Manifesto is an instrument to succinctly state the mission of Agile development. The values and principles of Agile development should be embodied within a specific Agile methodology, since those are the key tenets of Agile development. The values and principles reflect the spirit of Agile, and the implementation is the methodology.”

I was wrong!

Well, in part. I still prefer to stay away from defining Agile development as a framework or a guideline because Agile development does come in various flavors, including Scrum, XP, Feature Driven Development, etc. – and each have their own specific approach.

I also stand by my take on the Agile Manifesto. Agile development is more than a set of principles, although thanks to the Agile Manifesto we have a documented set of values and principles for Agile development. Ultimately, the definition of Agile development must be a general term that allows for specific implementations.

The use of the term methodology fits, if appropriately used. And this is where I was wrong and where I feel Wikipedia’s definition is slightly off. The term methodology has been used – inappropriately – as a substitute for method. Here’s a quick take from Wikipedia on the use of the term methodology:

Etymologically, methodology refers to the study of methods. Thus the use of methodology as a synonym for methods (or other simple terms such as means, technique, or procedure) is proscribed as both inaccurate and pretentious.


Agile software development is thus the study of methods – a methodology, whereas a specific implementation such as Scrum is a method. Wikipedia therefore should not refer to Agile software development as a “group of software development methodologies,” but as a “group of software development methods.”

What is my definition, using one sentence?

Agile development is a methodology that refers to a group of adaptive, responsive software development methods that value delivering working software in the shortest period of time in a sustainable manner.

Ten Rules for Better Management

June 8, 2010

Managers are more important to an employee’s decision to remain with a company than any other factor. The book, First, Break all the Rules, takes the position that “…if your relationship with your manager is fractured, then no amount of in-chair massaging or company-sponsored dog-walking will persuade you to stay and perform. It is better to work for a great manager in an old-fashioned company than a terrible manager in a company offering an enlightened, employee-focused culture.“

Since being a good manager is this important, here's my personal list of rules for better management.
  1. Know your staff, and align their strengths and preferences with the needs of the organization. This means getting to know each person as an individual, seeking to understand their strengths, weaknesses, preferences, and career aspirations. If you can leverage peoples’ strengths and assign them to projects that are in line with their stated preferences, they will be motivated to perform, and more likely to be successful in the process. This will also result in less directing and cajoling on your part – a real win/win.

  2. Understand what good performance looks like. As a manager, you don’t have to be the best performer, but you do need to know what stand-out performance looks like in order to judge it accurately. To do this, look to your best performers; you cannot understand what excellence looks like by studying failure.

  3. Provide clarity and context relative to expectations. Make sure that you have you clearly communicated your performance expectations in terms of what needs to be done and how to go about it. This should be in the form of a regular, two-way dialog and not a "it's my way or the highway" mandate. As a part of this dialog, make sure that the expected results are understood in context of the organizational goals and values.

  4. Stretch people, but do so carefully. To get high standards of performance you need to set tough, yet attainable goals.

  5. Pay attention. I’ve seen articles and books dating back decades that discuss the fact that people don’t get the time and attention from their bosses that they feel they need. Some managers deliberately distance themselves from their staff, not wanting to be "too familiar" or too close because they feel it will undermine their authority. This doesn't work, as you cannot build personal influence and strong relationships by distancing yourself.

    Another part of paying attention to people is finding ways to provide praise and recognition for things that people are doing right on a regular basis. People need to know that you are paying attention to what they do, and that they can count on frequently hearing from you. Finally, giving praise and recognition are not only great motivators, but it makes any corrective conversation that needs to be conducted a whole lot easier.

  6. Provide opportunities to learn and grow. Knowledge workers need to expand their horizons to be of continuing value; putting knowledge in the knowledge bank allows you to make those continual withdrawals, paying dividends in the process.

  7. Identify and remove impediments. People sometimes need a little assistance and guidance to make them more productive. A few key questions to ask that can surface typical bottlenecks:

    • Does your staff have the proper materials and equipment required to do their work?
    • Are there things that you as a manager or another part of the “system” are doing to impede progress?
    • Is too much being demanded of too few?

  8. Take the long view. Yes, we all have urgent tasks that creep into our day, but keep the longer-term objectives in mind when dealing with those day-to-day pressures. Continual firefighting and unchecked multi-tasking keeps people very active and busy, but it diverts you and your organization from achieving meaningful results. Ultimately, this lack of meaningful accomplishment will be a significant de-motivator.

  9. Provide confidence and optimism. This is particularly important when the pressure is on. People and teams that don’t allow themselves to get rattled under pressure can make all the difference in terms of successful delivery and failure. Under times of stress, managers can shore up everyone by projecting confidence.

  10. Assess employees on specifics, not on generalities. Give people something concrete to work with. “Rudeness” is general, whereas specifying the improper behavior of interrupting a co-worker when she was making a suggestion is specific. “Not a team player” is general, where pointing out that not volunteering on several occasions to take on extra work is specific. Having a “good attitude” is general, whereas smiling at people and never complaining during crunch time is specific.
Did I miss anything that you view as important?

What is Better About Agile Development?

June 4, 2010

This post is a reprint of the article Top 10 Reasons to Use Agile Development that I wrote for DevX.com, published on April 27, 2010 and used here with permission. Given the comments generated from my last post on this site and on Reddit, I thought it would be appropriate to provide my thoughts on the benefits of Agile development. And I'm very interested to learn your thoughts on the subject!

You’ve heard the buzz: Agile development is all about solving the problems associated with traditional software development. The Agile Manifesto states, “We are uncovering better ways of developing software by doing it and helping others do it.” But just what is better about Agile development?

As a software development manager, I’ve had to consider this very question as we’ve progressed from testing the Agile waters to a full-blown implementation of Agile across our organization in the past few years. My thinking and this list are the result of my personal experience with Agile, numerous conversations, training, and plenty of reading on the subject of Agile development written by other experienced, very capable practitioners.

1)      Superior ROI.
With many software projects, the business is forced to wait for the completion of the entire project before it can begin deriving a benefit from that investment. Given the challenges in meeting software schedules, those on the business side of the house not only have to wait for delivery of the software, they have the potential to be disappointed in a couple of ways:

a)      Features are trimmed from the final delivery.
b)      Quality trade-offs are made.

(There are other ways to be disappointed, and many are addressed in this list.)

Agile emphasizes delivering early and often, enabling the business to begin generating a return on its investment much earlier. Agile does ask for discipline and participation from the business as part of the deal, such as rigorously prioritizing the features and being available to answer questions during the development cycle. In return, the business gets its highest-valued features delivered early, and delivered with quality.

2)      Business agility is embraced.
In order to capitalize on opportunities, the capability for a business to adapt and respond to change is critical. Software development practices shouldn’t run counter to business needs by forcing the business to choose and adhere to a set of pre-determined features that will be delivered months, if not years, into the future.

Agile development welcomes and adapts to change. Because software is delivered in short iterations (measured in a few weeks) from a prioritized backlog of features, Agile development projects are able to easily adapt in accordance with changing business conditions.

3)      Agile development reduces risk.
There are a number of risks (a.k.a., challenges) with every software project. Schedules, budgets, scope creep, and one of the most demoralizing for everyone involved – particularly if you’ve exceeded your planned schedule and budget – delivery of software where the users “got what they asked for, but it isn’t what they wanted” because the requirements were not understood in the first place. (This sometimes masquerades as “changes in requirements.”)

Agile development seeks to avoid these issues with frequent delivery of working software that allows the business users the opportunity to provide feedback based on frequent inspection. This permits the team to make immediate corrections if there was a misunderstanding.

The frequent delivery of working software also keeps schedules and budgets in check. There is complete transparency and certainty about the progress of the team because the features delivered slice vertically through the architectural layers. This eliminates the late-stage integration and quality issues suffered with software projects that use different delivery mechanisms.

Finally, the business has the option of discontinuing further investment and retaining what has been delivered at any time, instead of being forced to make a full (or greater) investment in order to build out the software to a state where it is suitable for use.

4)      Agile development increases productivity.
Producing software that meets the needs of the business requires the involvement of knowledge workers from a variety of disciplines – business and technical – to work together. Agile development focuses the team’s attention like a laser on delivering the highest-priority, highest-valued features in short increments of time using the most productive mechanisms to accomplish the work.

As part of this delivery, Agile development goes beyond using directed teams that are in reality nothing more than a collection of individuals working on assigned tasks. Agile teams embrace collaboration in the truest sense of the word; there are shared goals, shared knowledge, shared learning, shared progress, and a shared responsibility by the team to meet its commitments.

Another productivity increase comes from teams operating autonomously, where the teams make informed decisions about their day-to-day work without the need for constant managerial direction. Not only does this expedite certain decisions by keeping them localized, employees who have more control over their day-to-day work are more energized and engaged about their work.

When an autonomous, highly collaborative, multi-disciplinary team is firing on all cylinders, the productivity gains can be extraordinary.

5)      Agile development creates a sustainable development environment.
Software projects rarely meet their initial schedule. Overtime with the mandate to “work harder” is frequently used as it becomes apparent that a project will be running longer than anticipated. Often times, frustrated managers feel justified in holding those who provided the estimates at fault for not meeting their commitments – in spite of the fact that an estimate is supposed to be an approximation and not a precise figure.

There are a number of problems with the regular use of overtime, including more mistakes being made by tired employees, risk of costly turnover, and the simple fact that a constant overtime model is not sustainable in the long run.

Agile development sizes User Stories in points and tracks team velocity – the User Story points that are completed in each 2-4 week iteration. Over a period of time, it becomes crystal clear how much work can be realistically accomplished by a team on a sustainable basis. The goal then becomes one of working smarter to improve, not harder.

6)      Agile development enables emergent innovation.
Big innovations are easy to recognize, but hard to come by; they typically materialize from work outside of day-to-day projects. Because Agile creates a sustainable development environment, a greater opportunity exists for innovation to emerge from the employees. Teams that are working constant overtime to meet schedules simply lack the time or inclination to think about anything else other than the difficult schedule in front of them. In sustainable development environments, people have the time to think more about the business and explore – creating the potential for innovation that did not exist previously.

Even on “routine” tasks, the collaborative nature of Agile development creates the opportunity for smaller innovations to occur. Requirements are discussed as User Stories, involving what the business needs along with the benefit that it hopes to realize. The important aspect of this is that the requirements aren’t considered to be cast in concrete; there is a discussion about what the business is hoping to achieve.

The discussion yields important insight and understanding for the entire team. The worst-case scenario is that everyone will be on the same page. At other times, the dialog between business experts and the technical experts can yield unexpected results, like turning complex, difficult features into elegant, differentiating features.

7)      Agile development builds trust and relationships.
Through early and often delivery of working software to the business – even if this is nothing more than demonstrating the features – the business will grow to trust the development team. In addition, the continual dialog and the ability for the business to adjust and adapt the software gradually changes the “us versus them” dynamic into a “we.”

The same will happen for the members of the Agile development team. Instead of everyone divided by functional roles, Agile teams make the most effective use of the team members as dictated by the needs of the team to meet its commitments. The shared goal becomes more important than each individual working strictly within his or her area of expertise, with defined policies and procedures for handing off work between functional roles. This breaks down barriers between functional disciplines, enabling the team to reach higher levels of productivity.

8)      Agile development expects continuous improvement.
Agile development expects its practitioners to be continually learning and adapting, and is something that is enabled in part through sustainable development. Sustainable development provides the time and energy for the development team to expand their working knowledge of their profession through self-learning.

In addition, Agile teams conduct regular retrospectives at the end of each iteration to review what is working well and what can be improved.  Team members are expected to assess their teamwork and their use of (or lack thereof) technical practices and to make adjustments in the upcoming iteration to improve.

9)      Agile development is motivating and engaging.
Agile recognizes that knowledge workers have the greatest understanding about their own work, and that they are the ones best qualified to plan and organize how they will accomplish that work. Agile development utilizes an autonomous, self-directed work environment that is a powerful motivator. This control over their workday, plus operating in a sustainable mode and the feeling of accomplishment that is a by-product of continuous delivery of working software, all combine to provide a solid motivational boost to each and every person on an Agile team.

10)   Agile addresses the realities of software development and the needs of the business.
The challenges that every software project faces stem from the fact that we’re not producing pre-defined widgets; there is uniqueness involved with every software project. Agile development’s entire approach addresses the major problems encountered with software projects and managing talented knowledge workers while being aligned with, and responsive to, the business.

I’ll close where I opened. There is a reason that the Agile Manifesto states, “We are uncovering better ways of developing software by doing it and helping others do it.” Agile is a better way.

Agile is Fragile

June 1, 2010

I find it fascinating when I hear someone state that “Scrum doesn’t work” or that it isn’t all that it is advertised to be, particularly in organizations that have embraced and adopted Agile/Scrum development. It’s always worth exploring why that person feels the way he or she does, particularly if Scrum isn’t working for a software development project.

If a lightweight process like Scrum isn’t working, you’re seeing the fragile side of Agile. It’s all too easy to read about Agile development and listen to people like me – I’m an optimist by nature – champion the wonders of Agile development and how it will improve your development universe. While Agile sounds simple on paper, a proper implementation requires a deep understanding, discipline, and organizational support. And even if you get it right initially, it's easy to go astray.

I embrace Agile/Scrum development because I am firmly convinced that it addresses many of the problems that plague software development along with providing working professionals with the autonomy that they should have in the first place. If something isn’t working, most likely it isn’t the process that is at fault. I’d wager that there is an implementation or execution problem.

We’ve implemented Scrum at my company, and we’ve been fairly successful with the basics to date. Unfortunately, I keep hearing and reading about other organizations that are failing on the basics, hindering their successful adoption of Agile/Scrum. This is the classic ScrumBut problem of, “We’re doing Scrum, but…”
  • We don’t have a product owner.
  • We don’t have a product backlog.
  • We don’t bother with daily standup meetings.
  • We’re not doing ... some fundamental activity that is and should be a part of the Scrum process.
Many organizations struggle with implementing the basics. Jeff Sutherland has spoken frequently about the Nokia Test and how upwards to 80% of teams that claim to be doing Scrum don’t perform the basics (some don’t even know who the product owner is, let alone have a prioritized backlog).

I’ve witnessed the benefits of Agile/Scrum, seeing teams improve productivity, reduce overtime, and generally enjoy their day. But we have experienced problems as well.

A common problem is that teams will implement the mechanics of Scrum in the context of old habits. In the software world, functional specialties become the focal point: analysts create the requirements, developers develop the software, and QA tests the software once development is “done.”

Functional hand-offs turn Agile projects into a series of “mini-waterfall” efforts – bypassing the collective, collaborative aspects that Agile development brings to the table to boost productivity. The risk with this approach is that the User Stories may be in various stages of completion at the end of the sprint, but aren’t “done done.”

Even when teams are collaborating well, they sometimes take on more work than they should. This problem surfaces in different ways.
  • Teams fail to track their velocity – skimping on an important aspect of Scrum – and then continually misjudge their capacity to take on work and end up committing to more User Stories in a sprint than they should.
  • Teams start work on too many stories at the beginning of the sprint instead of focusing on the high-priority stories, possibly feeling external pressure to get a set amount of work done in a specific time frame. This again leads to ending with User Stories in various stages of completeness, none of which qualify as finished.
The point about external pressures is also important relative to the successful adoption of Agile development. The surrounding organization can have an impact. Martin Proulx provided some great examples in a recent post about Agile mis-adoptions he's encountered as a consultant that are as much fun as they are scary. One of my favorites:

Client: We have identified a list of issues that we need help with. Here’s the list. Can you help us?
Martin: Possibly. Let me look at your list. Who came up with the items on this list?
Client: Me and my direct reports.
Martin: Has the team been involved in putting this list of issues together?
Client: Absolutely not. We asked them to put together a list of issues they were facing and most of the items were related to lack of trust, micro-management, and bad communication so we threw out their list and put this one together for them…

Or this one from an overly-enthusiastic manager:

Client: This Agile thing is great! I’m going to impress the management team with our success.
Martin: How so?
Client: The development team asked me if they could use Agile for their next project and from what I read, Agile can help them improve their performance and reduce the time to market.
Martin: Yes, if it’s done right you may get those benefits.
Client: Wonderful! After I gave them the go ahead to start immediately, I told them I now expected to project to be delivered in 9 months (instead of 18 months) and cut their budget by half…

When it comes to problems with Agile development, people tend to blame the process when in reality the problem(s) are in their implementation and/or support of the process. While it takes more than you realize to fully implement Agile development correctly, the potential that it creates for greater productivity along with aligning development with the business makes in investment worthwhile. It forces you and your organization to contend with the very issues that are preventing you from improving today.