The Product Backlog as Communications Hub
With agile development using Scrum (the most popular agile framework in use today), we overlap product definition and delivery activities by incorporating a coordinating mechanism known as a Product Backlog. Conceptually, the Product Backlog fills that previously blank overlap area between product definition and delivery that I included in my Cone of Uncertainty diagram in my last post. Figure 3.5 shows the Product Backlog occupying this area:
This model is useful to illustrate that with agile development, we are striving to keep the costs down in comparison to traditional approaches. The left-hand side of the cone – the Define area – where the cone is the widest and where we have the greatest variability and uncertainty, so we want to avoid overinvesting this area by creating large batches of detailed requirements along with plans and schedules that are cast in concrete based on those requirements. This area is where a great deal of waste originates, as any planning that we are doing is relying on information that most likely will change when we get to the brass tacks of actual implementation.
With agile development we will have many, smaller versions of the Cone of Uncertainty with each short cycle. However, because an agile team is gaining certainty through continually learning and validating its work, the cone should narrow as we increment our way forward in later cycles, as shown in Figure 3-6:
- Get to work sooner by deferring on defining details until they are actually needed.
- Deliver in short cycles to obtain feedback on actual results.
- Allow for changing requirements and new priorities to be accommodated for the least amount of effort and cost.
- Allow for learning from one delivery cycle to be incorporated in future cycles, maximizing the value being delivered while keeping the overall costs to integrate that learning low.
- Our priorities.
- Our intent.
- The value/benefit doing the work brings to the customer and our business.
All of this is in support of the values, principles and characteristics covered in The Depth of Agile section of Chapter Two’s What is Agile Development post. Consider how the Product Backlog and the collaborative shaping of the solution are in support of one of the values of the Agile Manifesto: “Individuals and interactions over processes and tools.” (The Manifesto for Agile Development, 2001) A couple of principles easily apply here as well:
- The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
- The best architectures, requirements, and designs emerge from self-organizing teams.
An agile development team takes its work from the top of the Product Backlog, talking with the Product Owner and/or customer about what needs to be delivered and why and collaborates on how that can best be implemented in code and what that design should look like. This creates an opportunity for the development team to suggest options to meet the goals of the business in ways that weren’t previously considered, opening the door for some emergent innovation.
This shared understanding supports one characteristic of learning organizations: mental models. The Product Backlog – and techniques used to create a prioritized, estimated list of work items (requirements) that can be built and validated – is really all about defining what a shared mental model of what a great customer and business outcome looks like.
From an agile team perspective involved with shaping the solution, the communication steps involved are what Ron Jeffries, a co-creator of XP (eXtreme Programming) coined as the 3Cs: Card, Conversation, and Confirmation. (Jefferies, 2001)
It begins with a card, where the statement of intent is small enough to fit on an index card or sticky note. Next, there is a conversation to explore and flesh out the details which are captured as tests that will confirm for the Product Owner and the team that the User Story has been implemented correctly. The conversation and confirmation activities create a shared understanding about what is to be built, as depicted in Figure 3.7:
I can’t emphasize enough the usefulness and insights that result from these conversations. This is where a deeper understanding of the customers’ perspective, the business drivers, and constraints – like budgetary and time constraints – become very real for the team, creating greater alignment and commitment to the effort.
I’ve seen some solid contributions made during these discussions. While a Product Owner may have ideas about how to meet a given need, sometimes there are other options that should be considered. When teams are engaged in truly collaborative, robust discussions, I’ve seen challenging features get simpler to implement while continuing to meet the needs of the business. A few wins like this, and you have found one way to maintain scope and schedule! But this doesn’t happen when developers are relegated to mere implementers instead of being participative members of a product development team.
What Should Agile Teams Document?
If you are used to a more traditional approach to software development, you may feel that the “fleshing out of the details” should involve creating a detailed requirements document, that way development and testers can get to work in their respective domains as quickly as possible. However, there is another way, one that increases efficiency by combining tasks while keeping everyone on the same page.
What is the purpose of crafting a detailed requirements document? Referring back to Figure 3-7, it is to create a shared understanding that programmers and testers can use to create and validate the finished product or feature, right? Why not combine traditional tasks and collectively document or – even better – create the automated tests (we want to stay away from manual testing as much as possible) as part of the collaborative process?
This puts QA in the lead up front as opposed to being squeezed at the end. QA can help guide the Product Owner and the rest of the team towards defining the high-level Acceptance Criteria and the more detailed behavior involved with implementing the Acceptance Criteria.
A quick note: Acceptance Criteria should be high-level, bullet-list expressions that provide an additional level of behavioral detail about a User Story. Acceptance Criteria is not all there is to it, because as we all know, the devil is in the details. This is where more granular test cases oriented towards testing expected behavior and results help to provide greater specification of a given feature. The ultimate here would be to capture these tests in a test automation tool, creating what is known as “executable specifications.”
Just as the act of development always raised questions about system behavior, the creation of these tests will raise questions, particularly if tests are being created prior to diving into development. (Some teams may start development while QA builds out the detailed tests, but it is vital that teams remain highly communicative about the questions and the answers.)
From an expectation standpoint, it is critical that the Product Owner be available in real-time as much as possible to answer these questions! Delays in decision-making will slow the team down. Short agile delivery cycles increase the need for fast decisions. A one or two day delay when a two-week (10 business days) Sprint is involved is a substantial delay. Some of these decisions won’t be easy, either.
There will be routine opportunities to add scope. Determining what not to do, keeping the minimal marketable feature concept in mind, takes real discipline. The easy out is to tell the team that they need to do everything that has been thought of, but the aggregate effect of many small changes will increase the total delivery time, and we need to be mindful that there are other items on the Product Backlog waiting attention.
Let’s complete the picture with design work. Like everything else, the key is to have conversations and design what is needed now, without designing (and certainly not building!) items that we are anticipating that we will need in the future. This is not to say that we shouldn’t consider where we appear to be going and short-change what we are doing now, not by a long shot. A system design should focus on implementing the functional requirements laid out in the Product Backlog, but no more.
Another class of work concerns itself with “illities” such as scalability, manageability, operability, continuity, and security. Some of this work should be considered as part of a Product Backlog where specific items need to be addressed outside of the business features already defined. Other work will be common, shared with all of the business features contained in the Product Backlog. This includes things that involve secure code and validating application security, coding standards and code complexity, for example.
We don’t want spend time and energy replicating this information as part of the Acceptance Criteria or test cases for each and every feature. Items like this that the team agrees to be important and applicable to the product as a whole should be incorporated into a common Definition of Done and factored into the overall product development cycle from a time and effort standpoint. This Definition of Done should be as visible and transparent as the Product Backlog so that everyone – team members and non-team members – are clear about what the full scope of work required to complete a feature.
Communication is an Engagement Enabler
The key takeaway is that we don’t want an overreliance on communicating through documentation because this limits and constrains the critical interactions required for productive, innovative product delivery. With agile development, we document only what is useful and necessary at the point in time when it is most effective to create that documentation. We focus our attention on capturing information that contributes to downstream productivity (understanding that some documentation may be required for compliance purposes in some cases).
The heart and soul of agile development is the engagement of people in well-timed, robust conversations that are supported by documentation that lead to the greatest possible customer and business outcomes.
This post is a draft of content intended for an upcoming ebook: Agile Expectations: What to Expect from Agile Development, and Why. Blog posts will be organized under the “Agile Expectations” content on the left-hand side for easy reference. I welcome your comments and feedback! – Dave Moran
Jefferies, R. (2001, August 30). Essential XP: Card, Conversation, Confirmation. Retrieved from XProgramming.com: http://xprogramming.com/articles/expcardconversationconfirmation/
The Manifesto for Agile Development. (2001). Retrieved from The Agile Manifesto: http://agilemanifesto.org/