Agile is Fragile

June 1, 2010

I find it fascinating when I hear someone state that “Scrum doesn’t work” or that it isn’t all that it is advertised to be, particularly in organizations that have embraced and adopted Agile/Scrum development. It’s always worth exploring why that person feels the way he or she does, particularly if Scrum isn’t working for a software development project.

If a lightweight process like Scrum isn’t working, you’re seeing the fragile side of Agile. It’s all too easy to read about Agile development and listen to people like me – I’m an optimist by nature – champion the wonders of Agile development and how it will improve your development universe. While Agile sounds simple on paper, a proper implementation requires a deep understanding, discipline, and organizational support. And even if you get it right initially, it's easy to go astray.

I embrace Agile/Scrum development because I am firmly convinced that it addresses many of the problems that plague software development along with providing working professionals with the autonomy that they should have in the first place. If something isn’t working, most likely it isn’t the process that is at fault. I’d wager that there is an implementation or execution problem.

We’ve implemented Scrum at my company, and we’ve been fairly successful with the basics to date. Unfortunately, I keep hearing and reading about other organizations that are failing on the basics, hindering their successful adoption of Agile/Scrum. This is the classic ScrumBut problem of, “We’re doing Scrum, but…”
  • We don’t have a product owner.
  • We don’t have a product backlog.
  • We don’t bother with daily standup meetings.
  • We’re not doing ... some fundamental activity that is and should be a part of the Scrum process.
Many organizations struggle with implementing the basics. Jeff Sutherland has spoken frequently about the Nokia Test and how upwards to 80% of teams that claim to be doing Scrum don’t perform the basics (some don’t even know who the product owner is, let alone have a prioritized backlog).

I’ve witnessed the benefits of Agile/Scrum, seeing teams improve productivity, reduce overtime, and generally enjoy their day. But we have experienced problems as well.

A common problem is that teams will implement the mechanics of Scrum in the context of old habits. In the software world, functional specialties become the focal point: analysts create the requirements, developers develop the software, and QA tests the software once development is “done.”

Functional hand-offs turn Agile projects into a series of “mini-waterfall” efforts – bypassing the collective, collaborative aspects that Agile development brings to the table to boost productivity. The risk with this approach is that the User Stories may be in various stages of completion at the end of the sprint, but aren’t “done done.”

Even when teams are collaborating well, they sometimes take on more work than they should. This problem surfaces in different ways.
  • Teams fail to track their velocity – skimping on an important aspect of Scrum – and then continually misjudge their capacity to take on work and end up committing to more User Stories in a sprint than they should.
  • Teams start work on too many stories at the beginning of the sprint instead of focusing on the high-priority stories, possibly feeling external pressure to get a set amount of work done in a specific time frame. This again leads to ending with User Stories in various stages of completeness, none of which qualify as finished.
The point about external pressures is also important relative to the successful adoption of Agile development. The surrounding organization can have an impact. Martin Proulx provided some great examples in a recent post about Agile mis-adoptions he's encountered as a consultant that are as much fun as they are scary. One of my favorites:

Client: We have identified a list of issues that we need help with. Here’s the list. Can you help us?
Martin: Possibly. Let me look at your list. Who came up with the items on this list?
Client: Me and my direct reports.
Martin: Has the team been involved in putting this list of issues together?
Client: Absolutely not. We asked them to put together a list of issues they were facing and most of the items were related to lack of trust, micro-management, and bad communication so we threw out their list and put this one together for them…

Or this one from an overly-enthusiastic manager:

Client: This Agile thing is great! I’m going to impress the management team with our success.
Martin: How so?
Client: The development team asked me if they could use Agile for their next project and from what I read, Agile can help them improve their performance and reduce the time to market.
Martin: Yes, if it’s done right you may get those benefits.
Client: Wonderful! After I gave them the go ahead to start immediately, I told them I now expected to project to be delivered in 9 months (instead of 18 months) and cut their budget by half…

When it comes to problems with Agile development, people tend to blame the process when in reality the problem(s) are in their implementation and/or support of the process. While it takes more than you realize to fully implement Agile development correctly, the potential that it creates for greater productivity along with aligning development with the business makes in investment worthwhile. It forces you and your organization to contend with the very issues that are preventing you from improving today.

12 comments

Harold Fowler said...

Wow thats incredible. Who woulda thunk it?

Lou
www.anon-posting.at.tc

June 1, 2010 at 9:49 AM
bragadeesh said...

Nice read!! I recently shifted from the classic mode of development to a totally agile environ. Your article really helped.

June 1, 2010 at 9:53 AM
Anonymous said...

Could you just fix "We use Scrum at my company, and we’ve at least be able to get past some of the very basic issues" in 4th paragraph...makes you look bad.

(hey, not knocking you personally, I had a good laugh at myself when I proofread and realized I asked you to fix it in your 34th paragraph)

June 1, 2010 at 12:34 PM
Dave Moran said...

Thank you, Anonymous proofreader! I made an adjustment. Funny, sometimes what you think you wrote and what you actually wrote are two different things. I don't have anyone proofreading, and I appreciate the feedback.

June 1, 2010 at 12:52 PM
John Hunter said...

Well said. I find many management failures are do to very poor implementations of a style - compared to just really bad ideas in the first place (though that also exists). I think agile has great potential but you can't just say we are doing agile and expect everything to work. Poor implementation of the ideas will lead to failure.

Also you have to realize, in many organizations, you have to make numerous modifications. In my organization we have had to modify things because of what the organization is willing to do. It is working well for us but we have to watch where we are weak and attempt to improve our ability to actually practice agile methods.

June 1, 2010 at 1:50 PM
Anonymous said...

Here perhaps is another typo.

Many organizations struggle with implementing the basics. Jeff Sutherland has spoken frequently about the Nokia Test and how upwards to 80% of teams that claim to be doing Scrum don’t perform that basics (some don’t even know who the product owner is, let alone have a prioritized backlog).

Should be: don't perform the basics

June 1, 2010 at 4:14 PM
Dave Moran said...

John,

You make a good point about modifications. Agile is a "plan-do-adapt" model, and there should be regular retrospectives and a "what can we do better" mindset.

June 1, 2010 at 5:05 PM
Anonymous said...

I've been using Agile/Scrum for several years now. I believe in the process if I'm a consultant charged with building a product from beginning to end. I don't agree with using Agile fuel a team indefinitely. Using Agile every day is like perpetually falling forward; orbiting around the planet, unable to land, it gets old fast.

Because Agile/Scrum was chosen as the production model and people produce sprint, after sprint, the process gets a lot of credit. People are not manufacturing robots, designed to produce day in, day out. Robots cannot creatively produce solutions to Product owner specs and iterate. Robots cannot discern functional test cases or offer perspectives to solution architects. Agile will have you thinking different about humans.

Agile puts people into roles with objectives for present and future work. Agile does not concern itself with details of software construction like maintainability of code, implementation design, test iteration or refactoring. Agile is a process designed to produce a product in the shortest period of time, in small chunks, and gain acceptance on those chunks. Sounds great if you're a brick layer. I write software, which is a complex, abstract product. Because I care more about producing quality software than I do "efficient production", it's hard for me to swallow the Agile pill.

I am happy meeting deadlines and milestones and coding like crazy; I am not happy working like a robot.

June 2, 2010 at 12:59 AM
Dave Moran said...

Anonymous,
Your comments concern me, because it sounds like you have experiences that suggest the very thing that my post is attempting to point out. There are far too many improper implementations of Agile out there. I embrace Agile development because I feel that it in fact addresses the realities of software development, more so than other methods.

I’ll try to address some major points as I see them:

"People are bricklayers or robots."

The last thing that Agile should advocate is the notion that people are bricklayers. In fact, I responded to this very notion of developers as bricklayers towards the tail end of a recent post: http://www.softwareresults.us/2010/04/counterpoint-agile-baby-talk-kills.html. People are not robots, either. Agile development recognizes that producing software that meets the needs of the business requires the involvement of knowledge workers from a variety of disciplines – business and technical – to work together. This should be done is a sustainable manner, and not grinding people into the ground by treating them like coding machines.

There also should be conversations throughout the process of designing and building the software. This is the collaborative aspect that Agile brings to the table. Conversations with the business, with architects, etc. Talk about what the best solution should look like. Agile development values people over process!

"Agile does not concern itself with details of software construction like maintainability of code, implementation design, test iteration or refactoring."

The use of sound technical practices is a must! Agile teams should be developing automated unit tests. Having these automated unit tests should enable refactoring with confidence.

And while there isn’t a Big Design Up Front, there should be design. The trick is to design vertical slices (understanding where you think that you are going), but don’t spend precious time implementing and testing a large framework that might have to change later because the business needs change. By all means make use of industry-standard design patterns.

There also should be conversations throughout the process of designing and building the software. This is the collaborative aspect that Agile brings to the table. There should be conversations with the business, with architects, etc., talking about what the best solution should look like.

And when it comes to actual coding, writing maintainable code is vital. Why would anyone strive to do anything else? If the code isn’t maintainable, you are only hurting yourself in the long run. Eventually a team will not be able to deliver with the same velocity, because the existing code can’t be maintained – you will lose the ability to adapt it.

"Because I care more about producing quality software than I do "efficient production", it's hard for me to swallow the Agile pill."

Quality is a major component of Agile development. Teams are supposed to be finishing sprints with code that is “done done.” The software should meet the business need, it should be well designed and maintainable, and it should be thoroughly tested (with automated tests).

Ultimately, the goal is to deliver quality, working software is short time frames, but long-term efficiency should be emphasized.

June 2, 2010 at 6:21 AM

Dave, You make strong points. But one should realize that to get the basics right around self organizing teams coming from the age old water fall world is a serious cultural change and does take time to get it right. The comment of creating mini-waterfall is something that I have seen very closely :-)

June 7, 2010 at 2:32 AM
Dave Moran said...

Prasanna,

You are most certainly correct, getting the basics right does take time (and effort)!

June 7, 2010 at 9:16 AM
Vinisha said...

What an exciting experience!/Hilarious! Delightful! True!/wonderful stuff! thank you!

Scrum Process

April 8, 2011 at 1:40 AM

Post a Comment