Overconfidence: Why You Suck At Making Development Time Estimates 297
Dan Milstein from Hut 8 Labs has written a lengthy post about why software developers often struggle to estimate the time required to implement their projects. Drawing on lessons from a book called Thinking Fast and Slow by Dan Kahneman, he explains how overconfidence frequently leads to underestimations of a project's complexity. Unfortunately, the nature of overconfidence makes it tough to compensate. Quoting:
"Specifically, in many, many situations, the following three things hold true: 1- 'Expert' predictions about some future event are so completely unreliable as to be basically meaningless 2- Nonetheless, the experts in question are extremely confident about the accuracy of their predictions. 3- And, best of all: absolutely nothing seems to be able to diminish the confidence that experts feel. The last one is truly remarkable: even if experts try to honestly face evidence of their own past failures, even if they deeply understand this flaw in human cognition they will still feel a deep sense of confidence in the accuracy of their predictions. As Kahneman explains it, after telling an amazing story about his own failing on this front: 'The confidence you will experience in your future judgments will not be diminished by what you just read, even if you believe every word.'"
Scotty knows (Score:5, Informative)
La Forge: The Captain wants this spectrographic analysis done by 1300 hours.
Scotty: Starfleet captains are like children. They want everything right now and they want it their way.
But the secret is to give them only what they need, not what they want.
La Forge: Yeah, well, I told the Captain I'd have this analysis done in an hour.
Scotty: How long will it really take?
La Forge: An hour!
Scotty: Oh, you didn't tell him how long it would *really* take, did ya?
La Forge: Well, of course I did.
Scotty: Oh, laddie. You've got a lot to learn if you want people to think of you as a miracle worker.
- TNG 6x04
Re:Is that really the problem? (Score:5, Informative)
He comes out with the following gems:
- "I believe your estimates are too high"
- "I've already committed to a delivery schedule with the CEO and Engineering Manager"
- "Well, we'll just have to challenge your assumption"
- "We'll just have to find ways to work smarter"
- "We'll just need to work extra hours then"
- "You're not showing enough committment", when asked to work on the weekend and holidays. This despite being with the same company for my entire working life
It's like I'm in a Dilbert nightmare now.
Re:Pi (Score:5, Informative)
I usually take whatever estimate I'm given and change it to the next largest unit and double it. Thus, an estimate of two hours become four days. This is still usually less than the actual time required. And don't even get me started on projecting when some task will be completed as opposed to how much effort will be required. The above alogorithm does a reasonable job at estimating effort actually requied but determining the calendar completion date is a whole different animal.
Cheers,
Dave
Because civil projects never go over budget... (Score:4, Informative)
Predicting a civil engineering project, like a bridge, is easy.....
I'm going to stop you there, because civil engineering projects are NOTORIOUS for going over budget. You might have heard of projects like the big dig [wikipedia.org]. Less well know, is that going over budget in less spectacular ways is apparently a fairly common occurrence. I was looking around for a report to link for you that I read awhile back talking about why civil construction projects so frequently go over budget, but alas, I cannot locate it.
Alas!
Re:I guess I'm not an expert then.... (Score:5, Informative)
That's why I always say "That will take approximately 270 hours of development work" rather than "That will take 2 months". Then you write down how your time is actually spent, and can document that after 2 months you've actually only had 20 hours to devote to whatever it was, so it's no surprise that you're a long way from finished.