Tuesday, June 5, 2012

Agile Estimating: Story Points and Decay

I’m re-reading Mike Cohn’s Agile Estimating and Planning. It's the best book I've found on this and worth reading, even if he gets too Scrummy at times, and even if you don’t agree with everything he says. Which I don’t.

For example, I don’t agree with him that Story Points are better for estimation than Ideal Days. When we do estimates, we use Ideal Days, because thinking this way is more natural and more useful, and because these estimates are more meaningful to people inside and outside of the team.

Estimates in Ideal Days Decay

One of the reasons that Cohn recommends Story Points is that, unlike estimates in Ideal Days, Story Point estimates “have a longer shelf life” because they don’t decay – the meanings of the estimates don’t change over time as the team gets more information.

If you’re estimating using Ideal Days, as you learn more about the tools and language and platform, as you have more code to build on and work with, your understanding of how much work you can do in an Ideal Day will change. You may be able to get more work done – or less, because it turns out that you need to spend more time testing or whatever. As you learn and get better at how you work and understand it better, the definition of an “Ideal Day” will change. You’re learning and getting more information, so why not use this information and make better-quality decisions as you go forward?

But this means the estimates that you did earlier aren’t as useful any more – they’ve decayed. You’ll have to re-estimate this work at some point, because you’re mixing before-the-fact and after-the-fact information.

At some point you may have to do this anyways, whether you use Ideal Days or Story Points. Because relative-size estimates in Story Points also decay over time.

Story Points Decay too

Estimates in Ideal Days change as you learn more about the solution space: about how much work it actually takes to build some thing. Story Point estimates change as you learn more about the problem space: as you learn more about the domain and the problems that you need to solve, as you learn more about how big something really is in comparison to other things.

Take Cohn’s “Zoo Points” exercise, where people attempt to estimate the relative size of different wild animals. Everyone knows that a blue whale is much bigger than a rabbit or a fox. But if you’ve never seen a blue whale in real life, you have no real idea just how much bigger it is. 10x? 100x? 500x? 1000x? What about a tarsier or an okapi – some of the rarest animals in the world. You’re unlikely to ever see one. How the heck could you know how big they are relative to a rabbit, or to each other? You don’t know, so you’ll need to make a reasonable guess and use that for the time being.

Later, as you learn more about the problem space, especially if it’s a deeply technical domain, your understanding of size and complexity and risk will improve, and could change a lot.

So your idea of whether a thing is twice as big as something else, or about 1/5 as big as another thing, whether it's worth 5 points or 15 points, will change as you understand more about what these things actually are. As you continue to estimate the changes and new requirements that come in, your new estimates won’t align with earlier ones. The kind of work that you thought to be 5 points earlier you now know is 15 points. Unlike an Ideal Day, you haven’t changed what “5 points” means, but you have changed your understanding of what work is actually “5 points” worth.

On a project that’s run for a long time, people will join the team bringing new knowledge and experience, and people will leave taking knowledge and experience with them. Which means that estimates that you made earlier, before you learned something, or when you knew something that you’ve now lost, can't be the same as the estimates that you're making now, regardless of whether you’re sizing work in days or points. It’s unavoidable – you can’t not use what you have learned, and you can’t keep using what you’ve lost.

There may be reasons that Story Points are a better choice for estimating, for some teams at least, but “decay” isn’t one of them.

1 comment:

GeoffK said...

I personally prefer Ideal Days over Story Points, and I agree completely that both can decay if they are left alone. One important point regarding this, in terms of process, is that the remaining estimates are supposed to be reviewed after each sprint, to see if everyone still agrees with them (or at least the estimates on what is going into the next sprint). The goal isn't to just review velocity, but to say "are these estimates still good". This is specifically because decay will occur, and estimates done earlier on are likely to have changed once more is known and understood.

I'm still not sold that Story Points are "better", in part because (as you said) they don't necessarily translate outside of any particular group (which makes using them as historic guides in new estimates less valuable). I think people can wrap their head around the concept of an "ideal day", and while it could vary some from person to person, I'm not sure it will vary as much as Story Points can.

In reality, both are somewhat arbitrary, so it's like arguing over two shades of blue, and which is "better". They're both blue, get over it. At least with an ideal day, I can explain the concept quickly, and lot of people will get it the first time. Having presented Story Points before, that one takes some explaining, and usually requires a lot more in terms of visual aids for people to get the joke (usually by explaining using objects in the room and some arbitrary dimension like volume).

Site Meter