On Deterministic Fixed-Length Timeboxes

Yes, the terms ‘deterministic’ and ‘fixed-length’ are redundant.

Once upon a time (also known as yesterday in some cases), there were big long-term plans, commonly referred to as Roadmaps or Gantt Charts. These plans had a precise, deterministic date—usually far out in the future—a date that created a nice, well-defined timebox. A year, a quarter, a month — something neat and well-defined. There were significant problems with these plans. There was very little allowances for feedback from customers. The long time horizons did not allow for responding to changing conditions. They set up a fixed goal or target for a fixed period of time. Since the timebox was fixed, the plans ignored the natural variability in knowledge work.

Along comes Agile(Scrum, to be precise) and says, "We have an excellent idea!" The main problem with these timeboxes is that they are too long. If we can make these one month or less, we would be in a much better place. And… they were absolutely right. The shorter feedback cycles did make things better. Getting feedback and replanning every couple of weeks was much better than doing this once a quarter or yearly. Every time the timebox expires, we can evaluate stakeholder or customer feedback, assess market conditions, and respond. A key aspect of this was setting a goal for this iteration. “We will achieve X for our product/service/customer.”

Setting Goals

Forget about Agile/Scrum/Kanban/OKRs for a bit. How do you set goals? The usual format is “I want to achieve X by date Y.” Another way to state that is “This year I will do X.” For example, one of my current goals is — “Be able to run 13 miles by March 31st, 2025”. Both the 13 mile number and the March 31st date are arbitrarily picked. I have some reasons for these numbers: my German Shepherd, Nisha, lived to be 13, and my birthday is in April. Those might be decent sentimental reasons but they are still arbitrary. When I say arbitrary, I mean not determined by an analysis of my past performance. We will come back to this point.

Photo by Emma Simpson on Unsplash

To begin with, the goal is too long-term and too large. Let us break this down a bit. Currently, I can comfortably run 2–3 miles. My first goal should be to make 3 miles a consistent distance on my runs. I am writing this in the second week of September, it sounds like a good timebox for this is the month of September. I would state the goal as “I want to consistently be able to run 3 miles by the end of September.” I can then build on that goal in October, then November, and so on. Part of accomplishing this goal would be rest days and strength training days, as well as finding the right shoes and routes. Everything that goes into sustainably achieving the goal. It sounds logical and perfectly fine. Now three things can happen with this short-term goal -

1. I meet the goal exactly on September 30th
2. I meet the goal before September 30th
3. I am unable to meet the goal by September 30th

Outcome number 1 is perfect. It lines up with what is stated. The only problem is, unless I consciously contrive to make it so, it is unlikely. It is possible, but unlikely. I say that because September 25th, 26th, 27th, 28th, and 29th are all possibilities as well. So are October 1st, 2nd, 3rd and later dates. The likelihood of meeting the goal precisely on the 30th is very low compared to one of the other dates.

This is the primary problem with a fixed-length timebox. It is deterministic. It keeps us rooted in this deterministic thinking of “There is a fixed end date.” Goals, knowledge work, life does not work that way. Most of the time, we are going to meet (or cancel) our goals before or after the set date. The only way we ensure we meet it precisely on the expiration date of the timebox is by contriving to do so. That contrived scenario can be created in two ways. First, setting a low bar and stretching the time for our efforts to meet the timebox length. Second, working overtime towards the end of the timebox to make things fit. Agile or Scrum do not recommend either, but Scrum’s Sprint implicitly suggests both.

Meeting Goals(not on the timebox boundary)

Let us forget about Agile/Scrum/Kanban/OKRs again for a bit. What happens if I meet my goal of consistently running 3 miles on September 25th? What do I do for the next five days? Do I start my next goal—consistently running 4 miles? Or do I stick with the current one for the next five days? Or create a new goal for the remaining time in the timebox? Or maybe something else?

Photo by Estée Janssens on Unsplash

I'm not sure what you would do, but for me, the choice is obvious. I would start on the next goal. I have some empirical data now that could help me set the next goal as well. Seems that I was able to get the value I wanted earlier than I expected. Might as well start moving towards the next encapsulation of value- my next goal. Lets see if getting consistent with 4 miles is doable within the next three weeks. Who knows, we might hit that goal early as well.

What about the other side of this…what if I am close to meeting my goal on September 30th, but haven't met it yet. I am relatively certain, based on my current empirical observations that in the next 3–5 days I will be able to be consistent with the 3 mile distance. What do I do then? Do I continue with the current goal for a few days? Since the timebox has expired, should I figure out a new goal for the month of October? I probably knew that I was not going to meet the goal a few days before September 30th, should I have adjusted the goal or the timebox?

Knowing that the goal is likely to be met soon, I would adjust the expiry of the timebox.

Variability and Goals

There is variability in hitting goals. As long as we are within an acceptable degree of variability, extending the timebox makes sense. If we are not within an acceptable degree of variability, then we have to figure out figure out why and remove that reason. We have to figure out whether this was common cause variation or special cause variation and apply the correct type of intervention. More on this in the video below:

All of this applies to the work we do as well. If we set a goal and it takes us longer or shorter than the allotted time to finish it — that is not necessarily a cause for alarm. We need to look at the Cycle Times of these to figure out if these are just normal common cause variations or special cause variations. A team missing a Sprint Goal is not necessarily a problem. My argument is that the Sprint being a fixed length is the problem.

There is a certain level of cognitive dissonance required to say “Software Development is too variable” and then teach “Set a goal to accomplish in exactly two weeks.”

We can not have it both ways.

Flow Metrics for Goals

This is a newer thought process for some readers. The Agile community has been pretty good at accepting that the individual items that teams work on can have flow metrics applied to them. The fact is that flow metrics do not become less useful when applied to other constructs.

Here is a simple thought experiment — Imagine you(or your team) are working towards a single goal. Now add two more goals that the team is actively working towards. Now add 20 more goals for the same team, all of them active at the same time. In which of those set ups is your team most likely to accomplish goals quickly? We have just walked through Little’s Law for goals. Work in Progress(WIP), Cycle Time, Throughput follow the same relationships for items and constructs at all levels.

Another way to think about it — If a construct has a defined start and a defined finish, it has flow metrics. I am sure there is a defined time we start working towards a goal. I am also sure there is a defined point when the goal has been met or abandoned. With these start and end points, we can calculate -

1. How many goals are active at the same time — Work in Progress
2. How long it takes to meet or abandon a goal — Cycle Time
3. How many goals do we accomplish per unit of time — Throughput
4. How long have we been pursuing an active goal — Age

Scrum does a great job of controlling the WIP of goals. It explicitly limits it to one goal at a time. It does a terrible job of acknowledging the variability in knowledge work. It tries to predetermine the Cycle Time and Throughput of goals as well. For a two-week sprint, Throughput is expected to be 1, and Cycle Time is expected to be two weeks. The length of the Sprint forces the Cycle Time for a goal. Now if we accept the non-deterministic nature of the work we did and the length of the Sprint was derived from the Cycle Times of goals, we might be closer to a more “Agile” solution. This would make the length of the Sprint a lot less arbitrary.

Pros and Cons of a Sprint(or a PI)

A Sprint is a great step in the evolution away from long-term planning. My friend Dan Vacanti likes to say — “Scrum makes complete sense only as a response to waterfall” and I wholeheartedly agree. It makes the feedback cycles shorter, it focuses the team by limiting the WIP of goals to one, it gives the team the autonomy to figure out how to accomplish the goal.

The problem is it does not go far enough. A fixed-length timebox still suffers from some of the old issues. It tries to artificially control variability instead of harnessing it. It creates a “Push System” of goals where teams get a goal thrust upon them every two weeks. This is in opposition to a “Pull System” which allows the team to pull a new goal as soon as it is ready to do so. It potentially delays the feedback on a goal completed too early since the review meeting happens only at the end of the Sprint.

That last point might not have been such a big deal in response to roadmaps that stretched out for months or years. It is a bigger deal when teams can push something they started working on to production in a couple of hours and observe the results the same day. Every day lost waiting to start the next goal, now has a much greater cost.

I can already hear people talking about daily sprints. Sure, if your Cycle Time tells you that, go for it. The ideal for me is to continue limiting the WIP on goals, and create a pull system where the next goal starts as soon as the last one was met or abandoned. If your Goal Cycle Time is 4 days or less 80% of the time, daily Sprints might hurt more than help. In fact, once you acknowledge the variability in accomplishing goals and start measuring how long it takes — any fixed-length timebox will hurt more than help. The idea is much simpler than finding the right length of the timebox. It is to create a pull system where the team can start a new goal as soon as they finish the last one. Of course, based on the observed Cycle Time, we create an upper bound where we start questioning if the goal is still worth following or not. In fact, with observing Goal Age, we can ask that question pretty regularly.

Another supporting argument for Sprints is that it gives stakeholders and management a fixed time to meet with the team. Creates a nice cadence. To quote Dan again “If your stakeholders can not make time to review what you are working on, the work is not important enough.” That sounds harsh, but is absolutely true. If your stakeholders have too much WIP, you either need new(more) stakeholders or find out the true priority and stop working on goals that they cannot prioritize enough to review. Another option is to just do the work, let the end user be your reviewer when they examine your product in production(or a beta/test environment).

Classic Agile vs Classic Lean

Classic Agile overcame a lot of hurdles as a response to Waterfall. It still has a lot to learn from Classic Lean. Scrum would be a little less simple and a lot more Lean if it promoted variable-length Sprints. A Sprint Goal is a work item masquerading as a separate construct. Even if you do not agree with that — it has a start and a finish. We should treat it as an item that flows and respect the variability. That will allow us to create a pull system that will help our teams serve our customers better. A pull system where the team does “Goal Planning” just-in-time. It allows them to work with the best available information and pick the most important goal based on what they have learned after completing the last goal.

The work we do has variability. There is variability in the amount of time required to meet goals. What if we created a pull system for goals that harnessed this variability instead of a push system that uses a deterministic timebox to push a new goal every few weeks?

I focused a bit more on Sprints in the second half of this post. The same points are true for PIs or any other fixed-length constructs. Figure out the WIP of the higher level items we are trying to manage and harness the variability to our advantage. PIs have a greater disadvantage as they are larger in timescale, but that is a matter for another day.