Working Smarter Means Measuring Smarter

March 26, 2010

Last year I posted a blog entry titled Essential Metrics. If you read the post, you’ll discover that I was not advocating measuring the outputs of software project work as much as I advocating measuring what goes into producing results. Software Development = Knowledge Work, and we need to measure those things that matter to knowledge work, some of which is generic to all knowledge workers and some is specific to software development.

The difference-makers in knowledge work like software development are the people. It is their knowledge, their skills , and the technical and teamwork practices produce the results. Unlike a manufacturing plant producing standardized, pre-defined widgets, the outputs of knowledge work like software development are variable; attempting to measure outputs is an exercise in futility.

Some of the more traditional software metrics like measuring lines of code, function points, and defect rates don’t work for me. If you want lines of code, you’ll get more lines of code! But will this tell you that you’re more productive than ever before? And what about a case where someone refactors code that actually reduces the lines of code, making it more readable and maintainable?

When it comes to software metrics, there are simply too many variables involved to make good comparisons across different projects in one company, let alone making comparisons across a world full of software developers and projects. I also prefer direct, lightweight approaches that don’t detract from the most important activity of producing working software.

We’ve adopted Agile/Scrum development, using the delivery of working software as the primary measure of progress. How do we know that we're improving? We certainly didn't change our development process because we felt like it; we want to improve.

The term used to describe how fast a Scrum team produces software is known as velocity, and this can be used as a realistic measure of a team’s improvement.

Since Agile/Scrum teams are supposed to be delivering with quality at all times, and if each user story is accepted as complete and “done,” then the size of the story counts towards the total velocity of the team. If a team improves its practices, the velocity should increase – providing a simple and direct measure of improvement.

The caveat is that velocity is only applicable to a given team. User story sizes are relative to each other, making them comparable to other stories that the team is working on. This makes the resulting velocity figure for each team unique to that team.

I’m good with this, because other factors come into play when measuring productivity. The nature of the work can vary from project to project, as can the size and composition of the team itself. Changing the composition of a team will change its velocity. The conclusion is that one of the real measures must involve the practices used to produce results. The better that teams are with their practices, the better their results – the outcomes – will be.

We need to make sure that we’re building the right thing.

We need to make sure that we’re doing things right.

Building the right thing is all about providing a solid return on investment – business value. This requires a product backlog that is prioritized by business value. And before a team commits to a collection of user stories, they should make sure that they have a clear understanding of those user stories. A second part of this is to ensure that customers (or proxies) are available to the project team to answer questions that arise about business functionality as the project progresses.

Everything else falls into the category of making sure that we are doing things right. Use of technical practices such as Test-Driven Development and Pair Programming ensure that common development mistakes are avoided. A practice like Root-Cause Analysis will inform teams where process breakdowns are occurring. Scrum practices like Backlog Grooming, Sprint Planning and Daily Stand-Ups help teams understand and monitor their work.

At the individual level, employees who are knowledgeable about their profession, who are technically proficient and embrace continuous learning and improvement, who can communicate and collaborate well, and who are willing to hold themselves and others accountable are equally essential.

Measure what matters, and with knowledge work like software development, measure – assess might be a better word – those skills, abilities, and practices that go into producing results.

(Photo by aussiegall)