Thursday, September 13, 2012

Can you get by without estimating? Should you try?

Estimating remains one of the hardest problems in software development. So hard in fact that more people lately are advocating that we shouldn’t bother estimating at all.

David Anderson, the man behind Kanban, says that we should stop estimating, and that estimates are a waste of time. In his case study about introducing Kanban ideas at Microsoft, one of the first steps that they took to improve a team’s productivity was to get them to stop estimating and start focusing instead on prioritizing work and getting the important work done.

Then you have experts like Ron Jeffries saying things like

I believe that most estimation is waste and that it is more common to use estimation as a replacement for proper steering, and to use it as a whip on the developers, than it is to use it for its only valid purpose in Release Planning, which is more like "decide whether to do this project" than "decide just how long this thing we just thought of is going to take, according to people who don't as yet understand it or know how they'll do it”

and

Estimation is clearly "waste". It's not software…If estimation IS doing you some good, maybe you should think about it as a kind of waste, and try to get rid of it.

And, from others on the “If you do bother estimating, there’s no point in putting a lot of effort into it” theme:

Spending effort beyond some minutes to make an estimate "less wrong" is wasted time. Spending effort calculating the delta between estimates and actuals is wasted time. Spending effort training, working and berating people to get "less wrong" estimates is wasted time and damaging to team performance.

In “Software estimation considered harmful?” Peter Seibel talks about a friend running a startup, who found that it was more important to keep people focused and motivated on delivering software as quickly as possible. He goes on to say

If the goal is simply to develop as much software as we can per unit time, estimates (and thus targets), may be a bad idea.

He bases this on a 1985 study in Peopleware which showed that programmers were more productive when working against their own estimates than estimates from somebody else, but that people were most productive on projects where no estimates were done at all.

Seibel then admits that maybe “estimates are needed to coordinate work with others” – so he looks at estimating as a “tool for communication”. But from this point of view, estimates are an expensive and inefficient way to communicate information that is of low-quality – because of the cone of uncertainty all estimates contain variability and error anyways.

What’s behind all of this?

Most of this thinking seems to come out of the current fashion of applying Lean to everything, treating anything that you do as potential waste and eliminating waste wherever you find it. It runs something like: Estimating takes time and slows you down. You can’t estimate perfectly anyways, so why bother trying?

A lot of this talk and examples focus on startups and other small-team environments where predictability isn’t as important as delivering. Where it’s more important to get something done than to know when everything will be done or how much it will cost.

Do you need to estimate or not?

I can accept that estimates aren’t always important in a startup – once you’ve convinced somebody to fund your work anyways.

If you’re firefighting, or in some kind of other emergency, there’s not much point in stopping and estimating either – when it doesn’t matter how much something costs, when all you care about is getting whatever it is that you have to do done as soon as possible.

Estimating isn’t always important in maintenance – the examples where Kanban is being followed without estimating are in maintenance teams. This is because most maintenance changes are small by definition - maintenance is usually considered to be fixing bugs and making changes that take less than 5 days to complete. In order to really know how long a change is going to take, you need to review the code to know what and where to make changes. This can take up to half of the total time of making the change – and if you’re already half way there, you might as well finish the job rather than stopping and estimating the rest of the work. Most of the time, a rule of thumb or placeholder is a good enough estimate.

In my job, we have an experienced development team that has been working on the same system for several years. Almost all of the people were involved in originally designing and coding the system and they all know it inside-out.

The development managers triage work as it comes in. They have a good enough feel for the system to recognize when something looks big or scary, when we need to get some people involved upfront and qualify what needs to get done, work up a design or a proof of concept before going further.

Most of the time, developers can look at what’s in front of them, and know what will fit in the time box and what won’t. That’s because they know the system and the domain and they usually understand what needs to be done right away – and if they don’t understand it, they know that right away too. The same goes for the testers – most of the time they have a good idea of how much work testing a change or fix will take, and whether they can take it on.

Sure sometimes people will make mistakes, and can’t get done what they thought they could and we have to delay something or back it out. But spending a little more time on analysis and estimating upfront probably wouldn't have changed this. It’s only when they get deep into a problem, when they’ve opened the patient up and there’s blood everywhere, it’s only then that they realize that the problem is a lot worse than they expected.

We’re not getting away without estimates. What we’re doing is taking advantage of the team’s experience and knowledge to make decisions quickly and efficiently, without unnecessary formality.

This doesn't scale of course. It doesn’t work for large projects and programs with lots of inter-dependencies and interfaces, where a lot of people need to know when certain things will be ready. It doesn’t work for large teams where people don’t know the system, the platform, the domain or each other well enough to make good quick decisions. And it’s not good enough when something absolutely must be done by a drop dead date – hard industry deadlines and compliance mandates. In all these cases, you have to spend the time upfront to understand and estimate what needs to get done, and probably re-estimate again later as you understand the problem better. Sometimes you can get along without estimates. But don’t bet on it.

7 comments:

genehughson said...

Great post. It's amazing that some people try to make these blanket pronouncements when the context others work in can vary so much. Of course you shouldn't bother estimating if you're not going to use the estimate to determine anything. At the same time, refusing to estimate when someone's attempting to see if the value gained is worth the cost will only cause problems.

itoctopus said...

What would you do if the client asks you for an estimate directly? Like "How much time will it take for you to finish my project?"

Most clients won't commit if you don't give them a clear estimate that you will stick to no matter what, so you have to do that, or you may risk having no cash flow.

Peter Hofman said...

It is funny to see articles with a title ending in a question mark, especially if the questions are suggestive.
When applying Betteridge's law of headlines the answer to the questions is "NO!".

When reading this article, it shows that it probably is not the answer you are expected to give...

Jim Bird said...

@itoctopus, Good point. It's obvious that another time when you can't get by without estimates is when whoever is paying for the work wants to know how much it is going to cost, or at the very least whether the work looks like it will be cheap and easy or not. I should have been clear about that.

@Peter: I hadn't heard of Betteridge's law of headlines. Thank you for that. I think it does apply here: the number of cases where you can get by without estimating isn't that large. So "No" is usually, although not always, the right answer.

Neil Killick said...

If you have a deadline then what's the point of estimating? It's the deadline - there's no point estimating how long remaining scope will take. You need to deliver the best possible product by that date.

The point you're missing is that the main reason estimation is unnecessary is that you get far more predictability by measuring throughput rather than asking the team to guess how long work will take to complete.

It doesn't matter how big the project is, if teams are working iteratively they are breaking down the work into small, roughly equal sized chunks. By measuring how many of these chunks get done every week or fortnight you can quite easily predict how many cards/stories can get done by the delivery date without ever having to spend time in estimation sessions.

This focuses the team and PO on the really important activities of doing the work and ensuring that the highest value undone work is done next.

Jim Bird said...

@Neil, I agree with you that if you have to deliver "something" "soon" or "as much as you can" by some date, then estimating isn't important. Unless of course whoever is paying for the work needs to know how much it is going to cost before they let you start. It's more important to figure out what you need to do, what's most important, and do it.

But if you are coordinating your work with other teams and other projects - you're part of a larger program where work needs to be handed off and the work of other teams depends on what you get done and when it gets done - or if you must complete specific things by a certain date - industry-wide changes happen frequently in financial services for example - then you need to break the work down and understand it and estimate it and do some planning and run through design alternatives upfront. You need to understand what you have to do and how big it is and what the risks are and what different approaches you can take and whether you need more people or different people to do the job, and how to schedule it.

You're correct, I don't understand how you get predictability through measuring throughput if you don't understand the size of the work in front of you. You know you can deliver between x and y points in a 2 week sprint. But if you don't know how many points of work are in front of you, you can't know how long it is going to take to get what you need to get done done. Throughput is only half of the equation.

Maria burton said...

I estimate everything even as far as the my weekly grocery shopping bill. I estimate the cost of each item. Add it up before I leave. I then know if I will fall within my budget but more importantly and this might sound trivial. If problems arise like the smaller washing powder is out of stock and I have to buy the really big one at double the cost. I can then make rational decisions that allow me to make that purchase and what I will need to not purchase so I can stay within budget. I know that sounds stupid and its only a shopping trip but its just the same with large scale projects.

Site Meter