Risky Business

Project Management Porn

Project Management Porn

Anecdotally, I had always known that Software Engineers are terrible at estimation.  I had never realised exactly how bad we are.

Some rules of thumb which seem to pop up now and again, is to take your engineers best estimates, and double them.  Then you’re actually in the ball park.
However Jørgensen (2009) would seem to suggest that this doesn’t go far enough. He details several experiments in his paper, but Experiment C resonates most with me.  It is under a similar context that I have had to create estimates and risk assessments for previous software projects.

In this experiment, he takes 50 software engineers and randomly divides them into two groups (LESS and MORE).  He then presents the requirements for a database project to both groups.  Those in group LESS are asked to specify the largest project risk, give an estimate of effort required to implement the project, give a success assessment of the project, and declare how competent the feel they are to achieve the project.  Those in group MORE were asked for similar outputs, with the exception of being asked to think of as many risks as possible.  They were also given a small risk assessment method, being asked to think about risk severity and probability, as well as receiving a checklist of risks to consider.

The results of this experiment are rather facinating.  The mean number of risks identified by both LESS and MORE were 1.0 and 4.1 respectively.  While the mean effort was estimated by LESS at 316 work-hours, while those in the MORE group had a mean estimate of 200 work-hours.  One would think if you identify more risks, you would estimate more effort to overcome them.

The requirements used in Experiment C were given to 4 independent software companies for actual implementation, and the mean effort required to implement the project was found to be 700 hours.

The net result of these experiments seem to suggest that software engineers are over-optimistic in their effort estimates, and over-confident in their assessments of estimation accuracy.  Is it any wonder so many software projects fail under traditional project management techniques?

If there are any lessons to be learned, perhaps we need to adapt our development processes to be resilient to the fact that the estimates we produce are so poor.



Tags: , , , , , , ,

3 responses to “Risky Business”

  1. Stephanie Moore-Fuller says :

    I don’t think that this is limited to software engineers, or even engineers in general. Years ago a friend told me that she had read a study comparing how long people *thought* it would take to do something, with how long it *actually* took. They studied many different types of tasks, both people who were familiar with the task they were about to do and those that were not.

    Their conclusion? It generally takes about 3 times as long to do something as you estimate it will take.

    Unfortunately I don’t have a cite for this study, but it was interesting enough to me that I’ve remembered her talking about it all these years.

  2. tboehm30 says :

    Throughout my career I have learned how bad I am at estimating my tasks. When I was doing programming as tech-support, I learned to double my estimate before saying them out loud. When I was consulting, I learned to double them again before writing them down. Now that I make the decisions for the consulting projects, I double them again before working with the team.

  3. jhop says :

    This ‘multiply by 2’ effect SHOULD have the result that any change to the underlying estimate is doubled. So small changes in the estimate result in large changes in the reported estimate. That doesn’t seem right!

    Also there are links between the time you are given to do a task, and the amount of time you spend on it. Effort expands to fill available time.

    Not that I claim to be a good estimator!

%d bloggers like this: