Investigating the Central Claim of Agility — Does Frequent Delivery Create More Value?
The Claim
Agile and Lean claim to have a critical economic benefit. This claimed benefit can be summarized as — by delivering work in smaller chunks, we can quickly learn what holds value and what does not. As a consequence, we can apply this learning to increase the value we deliver over time. All that without changing the money we spend on human capital costs. In other words, we can improve returns without increasing investment simply by working in smaller batches.
Let us investigate this claim. We will remain framework—or method-independent and assume that all Agile methods work towards faster feedback and utilization of that feedback. In other words, Agile approaches reduce the amount of time it takes us to get feedback and use the learning to make better decisions for the future.
We will model the ‘reduce the amount of time’ part using something called ‘Right Sizing.’ This is where we break up large pieces of work into smaller deliverable pieces that the customer can consume and give us feedback on. We will reduce the range in which Cycle Times (how long it takes us to get feedback after we start work) fall.
You can learn more about Right Sizing in the Drunk Agile video below.
(You can also find training on how to do right-sizing for your work here — https://www.tickettailor.com/events/danielsvacantiinc/)
We will also model the ‘learning to make better decisions for the future’ part in a later step. We will do this by assuming that the team gets better at selecting higher-value work with every delivery. Since every delivery is an opportunity to get feedback, our decisions on what we work on should improve with every bit of feedback. More on this later.
The question we are trying to answer is — Does delivering smaller pieces of work more frequently result in better economic outcomes?
Setting Up The Simulation
We are going to rely on our old friend Monte Carlo to help figure out the effects of working in small batches and learning. The simulations will have a team work through multiple items, over the course of a year to figure out how much ‘value’ they have delivered. We run these simulations 10,000 times and record the value returned after every simulation. Value here is a representation of real money.
Let us make some assumptions explicit. These are assumptions that are for simplification of the simulation. Their violations, if consistent, should not change results by much. For our purposes we only want to change one variable at the time and these assumptions help us do that -
- We have a magical system where after delivery, we can somehow measure value. The exact same method of measuring value will be applied in each case.
- We are working with a Team that is very good at maintaining a stable system. This is an assumption we are making, but one whose occasional violation might not be a huge problem.
- There are upper bounds on how long work can take. We are assuming the team does not let things linger for ever.
- Every item we start gets finished and is delivered as soon as it is finished.
In our simulations there are two values that are (as they should be) stochastic — How long it takes for an item to get done and the value that item delivers. Since we are treating these as random variables, we will change the bounds of these to Model cases where Right Sizing is applied and where it is not.
We are going to run two sets of simulations. One that has Large Items and another that has Right Sized items. The Cycle Times and Delivered Value for these are calculated as follows
Large Items — 85% of the items get done in 40 days or less. The upper bound (Cycle Time for the longest item) for these items is 60 days. In this case the WIP is limited to 1 item at a time. The delivered items can create anywhere between 1 and 1000 dollars of Value.
Right Sized Items — 85% of the items get done in 10 days or less. The upper bound (Cycle Time for the longest item) for these items is 15 days. In this case the WIP is limited to 4 items at a time. The delivered items can create anywhere between 1 and 250 dollars of Value.
We are assuming some good slicing techniques where items that are made smaller can still deliver value and the value is pretty evenly distributed amongst the smaller items. This is also a simplifying assumption which helps us simulate the effects of Right Sizing on value returns.
An Aside
Astute observers and folks who have studied Little’s Law will notice that going to Right Sized Items from Large Items is increasing WIP and reducing Cycle Time at that same time. It almost seems unfair to compare these two systems. On the surface this is absolutely true. If you dig deeper though, you will see that the systems are actually comparable. A 40-day large item, if it can be, through proper slicing techniques broken up, is the same as four 10-day right sized items. So, In the Large Item case, we are already working in a WIP of 4, the Right Sized case just makes it explicit.
Results of Right Sizing(Smaller Deliveries)
As we mentioned earlier, we ran these simulations 10,000 times and recorded the value returned after every simulation. We can plot these values on a histogram to see, over the course of a year (365 days) how much value do these simulations return. The graph below is the value returned by running the simulation for Large Items —
For the most part, we are seeing 5,000 to 9,000 dollars as the range for value generated in these simulations. Pretty impressive, given that every delivered item can generate between 1 and 1000 dollars of value.
How does the simulation for Right Sized items compare? Here are the results—
The range for value delivered is much higher now. In most cases, we are seeing 26,000 to 30,000 dollars as the range for value generated in these simulations. That is very impressive, since each delivered item can generate between 1 and 250 dollars of value.
Despite the range of single item value being lower, we are seeing a lot more value being generated overall. This is primarily because the lower cycle times are resulting in more frequent deliveries. We can compare the results at different percentiles to see how much better the results are for Right Sized items.
In the table above the first row can be interpreted as — in 15% of the cases, the Large Items system created 8,867.15 dollars of value or more, while the Right Sized system, created 29,489 dollars of value or more. The ratio of these two was 3.33. As you follow this down the table, you can see that in most cases the system working on Right Sized items creates about 4 times as much value as the Large Items system. That is also the ratio of the size of the items.
Another way to state this result — With all other things being held the same, reducing the size of your value items will result in an increase in value proportional to the reduction in size.
Adding Learning
Now, we are going to introduce one more variable into the system — Learning. Most proponents of Agile claim that real power of Agile is in the learning it provides to the teams and organizations. As the teams learn, they are able to make better decisions. These better decisions lead to ‘working on the right things’ and thereby increasing the value delivered.
The way we will model this is to say that our upper bound on the range of value we can deliver will increase by some percentage over time. Since learning can only happen with each delivery, this increase will take effect after every delivery. Both systems (Large Items and Right Sized Items) will be effected the same way, with every delivery, the upper bound (1000 for Large Items and 250 for Right Sized Items) will increase by the same percentage.
We will try this with three different percentages — 0.1%, 1%, and 5%.
Value increase due to learning = 0.1%
In the first case we will assume that the team gets 0.1% better at selecting value items after every delivery and the upper bound on value increases by 0.1% after every delivery.
Below is the histogram for the system processing Large Items —
There seems to be a slight shift here. The value generated has received a small bump. The percentile-ratio table below will show the small difference the learning made. Before we get to that though, lets take a look at the graph for the system with Rigth Sized items —
The shift is a bit greater here. We are hitting the 34,000–35,000 region a lot more than we were before. The 28,000–29,000 marks that were in the middle of the distribution without learning are now on the lower side of the distribution.
Let us take a look at the percentile-ratio table when learning is included.
The system working on Right Sized items definitely is able to take greater advantage of the learning. The frequent delivery of value, brings the value of learning at a faster rate back to this system. The overall ratio has started to shift from around 3.5-4.0 to 4.5–5.0. That is a major difference in margin. Compared to the simulations with no learning, the value delivery numbers for Right Sized system show about a 12% increase. For Large Items, this increase is about 0.5%.
Value increase due to learning = 1%
Let us turn this up. What happens when we learn at a pace that helps us improve by 1% in understanding what to work on with each delivery?
Here is the histogram for the Large Items. Things have definitely gotten a bit better. We see more numbers in the five-figure range start to show up on the graph.
How do things look for the system with Right Sized Items?
The increase is almost exponential. For the Right Sized Items, the value generated is 350% greater than the base case with no learning bonus.
A quick look at the percentile-ratio table shows how far ahead the system with Right Sized items is performing. It is 13–14 times better than the system with Large Items.
Value increase due to learning = 5%
Now this is where things get absolutely ridiculous. Below are the graphs and the percentile-ratio table for the case with 5% learning bonus.
The numbers are way beyond what my mind can handle. In the case where each delivery results in a 5% learning, we are expected to be 12,000 better at producing value if we Right Size items.
As an interesting aside, without Right Sizing being in the picture, if we only focussed on learning — Large Items produce about 40% more value if we can get to the 5% learning increase with every delivery.
Conclusions
Right Sizing is a great way to improve value delivery. With just Right Sizing (the way we did it here) and taking no learning into account, you can increase value delivery by 400%.
With aggressive learning (5% with each delivery) alone, without rightsizing you can expect a 40% increase. So, if you had to choose between learning and Right Sizing, these simulations suggest that you fous on Right Sizing.
The combination of the two is absolutely incredible. Even with a mediocre learning system (1%), a team working on Right Sized items is 13–14 times better at producing value than a system with Large Items.
So…regardless of the framework you use — Right Sizing and delivering in small chunks, along with learning from feedback have massive economic benefits. Of course, we started with a major assumption of a stable system — Controlling WIP. We also added in a Continuous Delivery assumption, no batching to wait for feedback.
You can pretty much ignore almost all other guidance from Agile frameworks. If you can create a stable system where WIP is controlled, Right Size your items, deliver as soon as an item is finished and learn from feedback, all other ‘framework’ structures can be removed. You do not need Sprints, PIs, estimation, prioritization etc. at least from an economic point of view.
Check out ProKanban.Org’s Accelerating Product Value course for greater insight into the concepts discussed here.
