I was reviewing one of Dean Leffingwell’s slide presentations and I particularly like a comment he made that “Agile is only a tool – it’s not a religion”. It inspired me to write this blog post. (Thanks Dean!)
There is a lot of religious fervor about Agile and there’s a lot of polarization between proponents of Agile and more traditional plan-driven project management approaches. There’s a lot of good reasons why Agile makes sense in many situations but some people seem to just do it mechanically – becoming Agile becomes a goal in itself without a real understanding of why it makes sense in a given situation.
Don Reinertsen has written a good book called “The Principles of Product Development Flow” which I think provides some valuable insight into the underlying principles of why Agile makes sense. Since the principles are general enough to apply to any product development effort (agile or plan-driven), it also provides an objective foundation for comparing Agile and alternative plan-driven approaches so that someone can make an intelligent and objective assessment regarding how these two seemingly competitive approaches.
The book goes a little too far in my view in developing a quantitative and mathematically sound basis for the principles in the book based on statistics. In the real world, I don’t think very many people would really apply that kind of rigor to analyzing these principles; but nonetheless, an understanding of the principles, at least at a high level, is very valuable and it does provide an objective foundation for understanding the benefits of Agile at a deeper level. I’ve summarized the eight principles here:
- Economics – Take an Economic View
- Queues – Actively Manage Queues
“Why do we want to change the product development process? The answer: To increase profits. As obvious as this might seem, this economic view of product development is surprisingly uncommon. Instead, product developers create a large number of proxy objectives: increase innovation, improve quality, conform to plan, shorten development cycles, eliminate waste, etc. Unfortunately, they rarely understand how these proxy objectives influence profits.”
I’ve seen a number of instances where companies blindly pursue some of those “proxy objectives” such as “increasing innovation” without really understanding the economic impact. “Increasing innovation” should not be an end-in-itself and it reaches a point of diminishing returns at some point and it might begin to impact other proxy variables such as quality. For that reason, it is good to put things in an economic context. The economic context provides a framework for making intelligent decisions about how much to focus on these “proxy variables”.
Here’s another example of how the economic context can provide a sound framework for decision-making: It is generally best to defer decisions on product features as long as possible; however, some decisions should be made early and should not be significantly deferred because if they have a significant economic impact.
“Queues matter because they are economically important, they are poorly managed and they have the potential to be much better managed. Queues profoundly affect the economics of product development. They cause valuable work products to sit idle, waiting to access busy resources.… queues hurt cycle time, quality, and efficiency.”
Developing requirements far-in-advance that sit in a queue waiting for development can be inefficient and wasteful because:
- The requirements may change prior to going into development and much of the effort involved in developing the requirements might have been wasted, and/or
- Speculation in the requirements that are done too far into the future can result in erroneous assumptions that make their way into development without being questioned
But, on the other hand, we shouldn’t blindly accept the notion that developing requirements far-in-advance never makes sense. There are a lot of reasons why at least developing high-level requirements early might make sense to guide the overall direction and architecture of the project and I’m sure that there are even some circumstances where developing more detailed requirements upfront also makes sense (you have to use the economic impact as a guide for making that decision).
“There is probably no aspect of product development that is more misunderstood than variability. Because variability sometimes leads to bad outcomes, it is inevitable that some observers will incorrectly generalize that variability always leads to bad outcomes.”
Reducing variability will many times improve efficiency but that isn’t always the case. For example, Breaking up large requirements into smaller ones that are of a more uniform size reduces variability and can improve flow, however, at some point further attempts to reduce variability do not have economic value.
For example, if we might use a rule-of-thumb that if a user story is greater than 13 story points, it needs to be broken down, but there’s probably little value in breaking down a story with less than 13 story points into smaller story points just to reduce variability.
“Ask a typical product developer what their batch size is, and why, and you will get a puzzled look…Product developers simply don’t think in terms of batch size. As a result, they underutilize one of the most important tools to improve flow… many of the most important improvements in product development such as concurrent engineering, rapid prototyping, and agile software methods, are recognizable as batch size reductions. However, because developers fail to recognize the underlying mechanism of action behind these improvements, they lose the chance to apply this principle more broadly throughout their development process.”
Examples of batch size inefficiencies include:
- Project scope – taking on more in a single project than is truly necessary
- Project Funding – The entire project is conceived and funded as a single large batch proposal
- Requirements definition – the tendency to define 100% of the requirements before the project starts
Use Work in Process (WIP) constraints to manage overall flow. For example,
- Control the number of projects taken on at any one time to avoid over-saturating development resources
- Use specialized resources wisely to maximize their impact on overall flow
“Cadence is the use of a regular predictable rhythm within a process. This rhythm transforms unpredictable events into predictable events. It plays an important role in preventing variability from accumulating in a sequential process…”
Examples of cadence are using fixed-length sprints to establish a pace for the development effort. Examples of the use of synchronization include:
- Concurrent development on multiple paths at the same time
- Concurrent testing of multiple subsystems
Fast feedback can lower the expected loss by truncating unproductive paths more quickly or raise the expected gain because we can exploit an emergent opportunity by rapidly redirecting resources.
Fast feedback combined with selecting appropriate measures of performance enables rapid learning and ongoing continuous improvement.
“Sophisticated military organizations can provide very advanced models of centrally coordinated, decentralized control. There is an impression that military organizations seek robotic compliance from subordinates to the orders of superiors. In reality, the modern military focuses on responding to a fluid battlefield, rather than executing a predetermined plan. It views war as a domain of inherent uncertainty, where the side that can best exploit uncertainty will win.”
Source: Reinertsen, Don, The Principles of Product Development Flow, Celeritas Publishing, 2009
I recommend Don Reinertsen’s book for anyone who wants more in-depth understanding of these principles (this is just a very high-level overview). Many people get immersed in the mechanics of Scrum and forget about the values and principles of Agile. Don Reinertsen’s book goes even deeper into the principles of product development which provides a good basis for understanding both Agile (adaptive) approaches and Waterfall (plan-driven) approaches at a deeper level.