Agile Development: Where’s the Proof?

June 25, 2010

Have you ever participated on a forum or e-mail exchange that became a great example of terrible communication? This happened to me recently with on Reddit when I submitted a link to my post, What is Better About Agile Development? (This post was a reprint of an article that I wrote for DevX.com titled, Top 10 Reasons to Use Agile Development.)

There are times when face-to-face communication is much more effective, where a conversation only minutes in length can save hours of electronic chatter. This Reddit dialog felt like one of those times.

It started with this comment about my article: “Methodology marketing - no 'softwareresults' - no evidence, just assertion.”

I responded with, “It's an opinion about how Agile development can provide a number of benefits – and improve the results of software development efforts. I structured this piece with reasoning to support the assertions; but no, I did not provide specific evidence. Does the reasoning fail to convince you? What evidence would work for you?”

Things went downhill (and around the hill) from there.

I’ll admit that my motivation was to explore whether the assertions and brief reasoning offered in the article held up with someone who clearly:
  • Has experience in the software development field.
  • Reads a great deal on the subject.
  • Participates in forums and comments on blogs.
My attempts failed. My agenda was to explore my article on its own merit, while this individual wanted me to supply proof of my assertions. This individual eventually summed up my article with this: It's not even wrong.

I believe this to be a harsh judgment, and our dialog failed to convince me that anything that I said in the article was in fact wrong. This individual demanded evidence from me, but he didn’t present any evidence that my assertions were wrong based on research that he has clearly performed, either.

I do agree (in part) with his answer to this question that I asked: “How should the industry be approaching software development, in your opinion?”

He answered, “With humility and measurement.

"The Principle of Dispassionate Methodology: Don’t get your method advice from a method enthusiast. The best advice comes from people who care more about your problem than about their solution."

I happen to care about software development and meeting the needs of the business, and I’ve become enthusiastic about Agile development because I believe that when it comes to working with the business to meet the needs of the business, the approach is excellent. And I certainly don’t prescribe using Agile methods if they don’t fit your situation.

This individual was right about one thing. My blog is about software results, and I could back up my assertions more thoroughly than I did, and I will do so in my next series of posts. In advance of this, let me ask you a few questions so that I can learn to write more compelling and informative articles.

Please keep in mind that writing articles places some constraints on you as a writer. The editor had already determined the topic in this case. He wanted a “Top Ten” article (lists and “Top Reason” articles tend to be popular). There is a word limit. I was given a range of 800 words to 1600 words, and I came closer to the upper limit in this case.

I simply lacked the space to delve into details of each assertion. I had just enough room to explain each one as succinctly as possible. My hope for the article was that it would get someone excited enough to consider to want to explore more, or perhaps provide a convenient reference for those who understood and supported Agile development, but needed to explain the benefits of Agile development to others.

My questions to you:

In your opinion, does my article capture the benefits and reasons for using Agile, based on your understanding of software development and your experiences? Why or why not?

What would you recommend that I change?

What other articles would you like to see written?

16 comments

Tony Morris said...

You are begging the question here. That is the reason you make no progress with your detractors. It is not so much that Agile has no evidence for its efficacy -- it's that it has no definition -- it is a projection bias only. This is why we hear "he's not doing real Agile, I am!" and other such nonsense.

It's an epistemological error to even believe that such a concept exists in the first place. Worse, to correct it, requires significant discussion and retraining in concept formation, which is an incredibly laborious task, with poor prognosis (ever tried to get a religious person to drop the bias?).

Agile doesn't exist, therefore, I am "not a fan" and neither are you.

June 25, 2010 at 7:24 AM
Anonymous said...

Tony Morris: Agile does indeed have a definition, but as with any new idea it is still taking shape. Saying "Agile doesn't exist" is missing the point completely.

Returning to the question at hand, I believe it is difficult for most people to accept a new idea or way of doing things based solely on logical or anecdotal evidence. You must provide concrete evidence in the form of facts to persuade people to believe that the ideas behind Agile are better than whatever they are doing currently. Case studies are an excellent way of showing how certain business practices actually lead to things such as "Superior ROI".

June 25, 2010 at 8:41 AM
coderoom said...

I think it would have helped a lot if you'd:
1. Briefly defined what you meant by 'Agile' at the start of the article
2. Compared it to some specific other methodology in each point, e.g. waterfall development, ad-hoc development
3. Stripped out all the buzz words. "Business agility is embraced?" Surely you mean "It's easier to change direction when you find out you're writing the wrong thing"
4. Use specific examples. They don't have to be real case studies; people think in examples. Joel was great at making up ridiculous examples to put a point across; the quality of the example isn't important, it's just a communication mechanism.

Even if you just set up straw men (which they'd probably have to be given the word count limit) it'd have made the case and thrust of the argument clearer.

June 25, 2010 at 9:12 AM
Hal said...

I think Tony's correct in his analysis. "Agile" has become a sort of hat by which group members recognize each other. Agile is so useful to consultants because it appears to offer concrete, actionable solutions -- but lacks any way of separating agile from non-agile (whatever *that* means). Agile is, therefore, another word for "cool" for a certain group of people. What is "cool"? If you have to ask, you're not. Agile proponents love to hand-wave about the benefits, but when the term itself is so ill-defined, they're really just blowing hot air (and charging while doing it).

June 25, 2010 at 9:14 AM
Anonymous said...

You say that the definition for "agile" is still emerging but then want to provide evidence for its benefits. This is silly. It's like arguing seventy years ago that "quantum mechanics is true, but we're still discovering what it is."

If you want to make progress, break your vision of "agile" into specific definable claims and defend those individually.

June 25, 2010 at 9:29 AM
Anonymous said...

I see merits to Agile principles in development, but I immediately get uneasy when someone suggests that a methodology is going to end world hunger. Further, I think that when you see someone go from "cowboy coding" to Agile, or some other methodology, the results are not necessarily because of "Agile" so much as they are because more thought and planning was put into the process of writing software.

June 25, 2010 at 10:56 AM
Dave Moran said...

Interesting thoughts, all!

@Tony: Agile doesn't exist? It does, at least in the minds of many out there. After all, there are people all over the world who believe that they are developing software using an Agile approach, and they have been responding in surveys like the annual “State of Agile Development” survey that VersionOne has been sponsoring. Although I have to say that Agile is a broad term that encompasses many actual methods

@Everyone: Good points and discussion on defining and differentiating Agile from other methods. I have a series of posts in the works that explores each of the ten claims that I made in my article, so I will be defending those separately. Based on the feedback here, I’ll work in greater definition of Agile into the equation as well.

Thanks again, I appreciate the comments!

June 25, 2010 at 11:55 AM
Amber Shah said...

I have received the criticism "where is your proof?" before when I've made assertions. I usually just blow them off since, like you, I've already stated my reasoning and experiences that back up my assertions.

It's a losing game to try to answer a comment like that. We are bloggers, not research scientists, and even if we were to cite research studies, it's so easy to poke holes in them as well.

Most of the time this appears when we do something against the norm. So in a way, it's a compliment that you are not just spewing the same thing as every other blog out there. Of course, no one bothers to ask someone to back up an "established" opinion (even if it is wrong). Yet somehow if we suggest an alternative, we need an army of supporters and a 100% bulletproof logical argument and a plethora of research studies? Talk about a double standard. Those are people that would rather fail the "established" way than take a chance to succeed a new way.

June 25, 2010 at 12:51 PM
Dave Moran said...

Thanks, Amber. You are right, we are bloggers and not research scientists. Like you, I find it interesting that people want to poke holes at things, and yes, I also agree that it is particularly easy to poke holes at research studies!

I am going to use this as an opportunity to explore my article in greater detail and cite research (and likely draw some individuals who want to refute that research). Maybe, just maybe someone will have a good point in the mix of comments, one that will provide a new direction for future posts.

June 25, 2010 at 1:06 PM
Isaac Gouy said...

1) "... my motivation was to explore whether the assertions and brief reasoning offered in the article ... My attempts failed."

Facts can be harsh:

- only one person thought it worth making any comment at all on the article

- they (I) clearly did not think "the assertions and brief reasoning" held up

Whether or not you like that answer, it is telling you something about the article.


2) "Where’s the Proof? ... wanted me to supply proof of my assertions"

Where did I demand proof? Here are my words which you quote - "no evidence, just assertion."

Where's the Evidence? Not - Where's the Proof?


3) "This individual demanded evidence from me, but he didn’t present any evidence that my assertions were wrong..."

I assert that that 95% of the Moon is green cheese and I insist that you accept that to be true unless you can present evidence that my assertion is wrong.


4) Let me try to be constructive, I've found "Being Logical: A Guide to Good Thinking" a very useful reminder of what might go wrong with my "reasoning".

http://books.google.com/books?id=9CMmAAAACAAJ&dq=being+logical&hl=en&ei=xOYkTMurKsncnAf07rTEBA&sa=X&oi=book_result&ct=result&resnum=1&ved=0CCUQ6AEwAA

June 25, 2010 at 1:28 PM
Anonymous said...

I read through all the references you provided via links and am repeatedly noting your insistence on inability to provide evidence due to lack of space. I think the individual you mentioned was not really questioning why you did not provide evidence in your article, but why you are still not able to provide evidence in the discussion that ensued or for that matter in the article above.

He had very valid questions that can tell us if the author is just requoting/rehashing from other popular articles or has real understanding of the subject. Real understanding can come from experience and study. Your response including questioning the individual motivation sadly shows that you are unable to defend your views. Maybe do a better job next time.

June 25, 2010 at 2:11 PM
Anonymous said...

Sorry but this wasn't a great example of terrible communication, this was a great example of you getting schooled. He turned your own reasoning against you, while you tried to duck the actual issues he raised. Well played, igouy, well played.

June 25, 2010 at 3:26 PM
JS said...

The post on the 10 benefits of agile doesn't have anything wrong with it really. It reads more like a marketing piece written by one of the agile methodology vendors or a biased Wikipedia entry.

What it really comes down to is that most of the benefits of agile are accrued to the company - "the customer" - and almost no benefits to the software practitioners. It's a fools' bargain nearly as dreadful as the one proposed by the User Experience Design folks such as Alan Cooper. (I'm not knocking UXD professionals - their work is extremely valuable; I just think their views of where UXD should fit into a software project is ridiculous from the perspective of a programmer.)

From your list, the benefits of agile may be summarized as:
1) The company gets to continually hold out the threat of wrapping up the project at the end of the next iteration if they're displeased with the team, reminding the team that they may not have any more work for them if this happens.
2) The managers get to wring additional work out of the development team by beating it over the head with its own self-selected goals and team-provided time estimates.
3) The managers get near real-time updates on what each team member is working on and how they are going about that work, enabling far greater micromanaging of the ongoing work efforts than would otherwise be possible.
4) The managers have development team members provide them with all the ammunition they need to scuttle any pay increases or bonuses with poor performance reviews.

What exactly does the software practitioner gain in the bargain? They get longer hours for meeting their own self-imposed deadlines, boring work focused on maximizing company ROI instead of solving interesting problems, and finally having to endure the sales and marketing guys as their new best buddies.

No thanks, waterfall methods work just fine: requirements cycles, ridiculous documentation requirements, lengthy development cycles, even longer integration and testing cycles, clueless managers, numerous status update meetings for those managers. To top it all off before the project fails after several years, the pay is better, and such big budget projects with their accompanying stable multi-year work histories look better on a resume.

June 26, 2010 at 2:32 AM
Matt said...

We are 30 years into this game of choosing our software development techniques without solid evidence to back up those choices. 30 years of trying and 30 years of failing. We are all desperate for something we can trust. We have heard logical arguments based in experience over and over again. I for one am finished making decisions that way.

June 26, 2010 at 8:32 AM

Dave, an interesting fork to hit. You have been passionate in your thoughts and how you achieved results by following agile. Agile is clearly not a one size fits all solutions, I say this because it needs not only the manifesto but more importantly a cultural change in letting go of control and in empowering teams. This is hard and time consuming for individuals and teams themselves. So there are proof of this pudding failing and succeeding. Waterfall does have similar stories. It is definitely another strong option - but personally, I continue to believe that this brings "joy to the journey" much better than the waterfall as the journey is much longer in waterfall as I wrote a few weeks back.

June 26, 2010 at 1:13 PM
Dave Moran said...

To all,

Thanks everyone for their comments! I’m taking the article, the Reddit dialog with Isaac, and the comments here and on Reddit to heart. In retrospect, I never should have engaged Isaac on that Reddit dialog, at least not at the time that I did. It was a huge mistake in judgment on my part; I was doing it as a diversion, but it was dumb. Let me explain.

My father had been in the hospital for weeks leading up to that post. I published that article on my blog because I was running out of content, and I wanted to keep things going. During the Reddit dialog with Isaac, my father’s condition worsened, and he passed away. This was draining emotionally and it hampered my ability to concentrate. I did a terrible job of engaging someone who clearly has a passionate interest in software development, and someone who is taking the time to participate and provide feedback.

I apologize, and I want to continue to explore that article and honor my commitment in providing background information to support my assertions. However, I’m thinking that I should do so in a way that is constructive for everyone. There are plenty of people who are rubbed the wrong way about Agile for various reasons, one of which is that they are being told (either directly or through inference) that their way of developing software is “wrong.” I’m a fan of Agile, but I’ve been successful in the past without it, as many others have. I’ll explore my article contrasting Agile versus plan-driven approaches, but I’ll be sure to add that going forward, we should be discussing what works and why.

June 27, 2010 at 7:22 AM

Post a Comment