Overconfidence: Why You Suck At Making Development Time Estimates 297
Dan Milstein from Hut 8 Labs has written a lengthy post about why software developers often struggle to estimate the time required to implement their projects. Drawing on lessons from a book called Thinking Fast and Slow by Dan Kahneman, he explains how overconfidence frequently leads to underestimations of a project's complexity. Unfortunately, the nature of overconfidence makes it tough to compensate. Quoting:
"Specifically, in many, many situations, the following three things hold true: 1- 'Expert' predictions about some future event are so completely unreliable as to be basically meaningless 2- Nonetheless, the experts in question are extremely confident about the accuracy of their predictions. 3- And, best of all: absolutely nothing seems to be able to diminish the confidence that experts feel. The last one is truly remarkable: even if experts try to honestly face evidence of their own past failures, even if they deeply understand this flaw in human cognition they will still feel a deep sense of confidence in the accuracy of their predictions. As Kahneman explains it, after telling an amazing story about his own failing on this front: 'The confidence you will experience in your future judgments will not be diminished by what you just read, even if you believe every word.'"
I sucked because I was pressureed to being sucky (Score:4, Insightful)
'nuff said.
We've all been under pressure to give our "best" estimates and then some.
Give a realistic estimate? Off to India!
Experiment (Score:5, Interesting)
I would love to see an experiment. Take two groups and give them the same job. Group one would be based on a typical American corporate structure with a Boss, Scheduler, budget person, middle management, supervisor, and finally people doing the work.
The other group would have the same number of people but only those that work. No schedule or budget just work until it's done. I wonder what the results would be?
Re:Experiment (Score:3)
That's easy. Universally accepted software, like say, Unity.
Re:Experiment (Score:3, Insightful)
Re:I sucked because I was pressureed to being suck (Score:3)
Exactly. If someone asks me for an impossible prediction, I will give them what they asked for with unyielding confidence. When faced with the inaccuracy of my prediction, I will continue, with confidence, in giving equally inaccurate predictions in the future.
My real algorithm is as follows:
Is it fun/interesting to do? If yes, feel the room and give an estimate that will keep the project from being killed. Else, give a long enough estimate that can withstand cross examination that hopefully will kill the project. Regardless of what I answer, management will cross examine my estimate using their own equally inaccurate measurements and assessments, if I deliver with anything less than absolute confidence I will be smashed. You see, the bullshit is layered upon the bullshit, then convoluted by management bullshit, into spectral bullshit that makes for a great power-point.
If you want an accurate assessment of how long something that has never been done will take, you're asking for the impossible. If you want an accurate assessment of how long something that has been done before many, many times will take, either a) you're not in technology in the US, we don't do "competition" here, or b) I'll tell you 75 years because I really don't want to do that anyway.
Re:I sucked because I was pressureed to being suck (Score:5, Funny)
dev: low 3 weeks, mid 5 weeks, high 9 weeks
management/sales:Great! I was hoping you would say around 2 weeks, because this product is being launched next week, so if we push it, we should be able to get it out the door by tomorrow approval!
Time management (Score:5, Interesting)
If you ask any experienced software developer about estimating when the project will be finally completed you will get a blank stare --- for the simple reason that there are always higher mountain to climb, more features to add, more bugs to be squashed, more optimizations to be made, and so on ...
I do not do time estimation --- I do the reverse
I set out a limit on time before I even begin a project
Within that time span I partitioned it into "exploration", "research", "coding", "debugging", "finishing touch" --- and I can terminate the entire project when any part of the partition takes too long, or produce too few result, or both
That's the way I've been using since the late 1970's --- it might not be the best way, but that's my way of accomplishing my projects --- or abandon it before it dragged out way too long
Re:I sucked because I was pressureed to being suck (Score:5, Insightful)
Manager: "We need an application that does X,Y, and Z. When can you have it done?"
Developer: "Well, can you tell me more?"
Manager: "No, time I have a manager's meeting in 5 minutes. Just give a pall park."
Developer:" Ok, umm 3 weeks."
Manager: "THAT LONG?"
Developer: "OK, 2 weeks? Maybe less?"
Manager: "OK"
Later, in the manager's meeting.
Manager:"My developer says he can get it done in less than 5 days."
Re:I sucked because I was pressureed to being suck (Score:3)
5 days later: But you promised!!! Now I'm on the hook for a demo to the VP of International Sales and Marketing!!!
Powerpoint! Those bastards don't know the difference. Just show some slides...
I guess I'm not an expert then.... (Score:5, Insightful)
Re:I guess I'm not an expert then.... (Score:5, Interesting)
I guess part of being an 'expert' is being dumb enough to buy your own crap. That's why they always seem so sure of everything. Meanwhile, folks like you and me hedge our bets, and people attribute that to not knowing enough, rather than knowing all too well what the real deal is.
I suspect that prior to being an 'expert', that person makes one wild guess that they nail bang on. After that, they just point back to the ONE TIME they were right, and that carries them for the next few years.
Re:I guess I'm not an expert then.... (Score:3)
I guess part of being an 'expert' is being dumb enough to buy your own crap. That's why they always seem so sure of everything. Meanwhile, folks like you and me hedge our bets, and people attribute that to not knowing enough, rather than knowing all too well what the real deal is.
I suspect that prior to being an 'expert', that person makes one wild guess that they nail bang on. After that, they just point back to the ONE TIME they were right, and that carries them for the next few years.
The other problem is that when you're regarded as being an expert and and 2 & 3 don't apply, giving an estimate that hedges for realities to happen doesn't satisfy management. You get accused of padding hours, being difficult, or playing favorites (if there are multiple approaches being evaluated). What's weird is that after this song and dance, they still expect you might run a week late...
Re:I guess I'm not an expert then.... (Score:5, Insightful)
I wish people would understand that project schedules should *only* be considered guesses and estimates. Take the time you think it will take, and then take a step back and ask yourself, "No really, how long will it take?" When you get a number, take another step back and ask yourself, "No, *really*, when a bunch of things go wrong and it takes longer than I expect, how long will it take?" And then treat that time frame as a best-case scenario.
Part of the problem is that many projects can not be set to a specific schedule. The real answer is usually "it depends". How long will it take to build a new website? Well it depends on what unexpected hurdles we run into. It depends on how many features you want to add after we begin. It depends on how many revisions we go through.
When people ask me to set a firm deadline, I'm always tempted to ask them, vaguely, "When we don't meet that deadline, what do you want me to sacrifice?" Any deadline can be met if you sacrifice enough of the project requirements. So if we're coming up on a deadline, would you rather I miss the deadline or that I sacrifice some of the requirements? That is, let's say you want a website running with features X, Y, and Z, and we have a deadline of June 1st. The question isn't whether I can meet the deadline of June 1st. The question is, on May 31st, when feature Z isn't ready (there will be some feature set "Z" that isn't ready), do you want to go ahead and launch the site anyway? Or is Z worth holding up the launch?
In other words: project managers should should focus on priorities rather than schedule. "Being on schedule" and "being within budget" are just two more features that need to be prioritized within the set of features that a project is trying to meet.
Re:I guess I'm not an expert then.... (Score:3)
Re:I guess I'm not an expert then.... (Score:3)
This is a bit presumptive. Sometimes the deadline matters most to the client, and sometimes completeness/correctness matters most. When you perform an estimate (which should always be a range), and the client has specified a deadline (a specific date), ask them this question:
"When the deadline comes, would you rather the project be incomplete but ready for delivery, or would you rather push back delivery in favor of complete and correct software?"
Communication with the customer is essential, and continual communication is necessary. The customer will not be happy if the due date comes and suddenly finds out the software is not ready. They may have planned testing, rollout, server or client updates, and many other dependencies based on the agreed-upon deadline. And they may have legal ground to sue you for failure to disclose the status of the project on a regular basis.
Re:I guess I'm not an expert then.... (Score:3)
Take the time you think it will take, and then take a step back and ask yourself, "No really, how long will it take?" When you get a number, take another step back and ask yourself, "No, *really*, when a bunch of things go wrong and it takes longer than I expect, how long will it take?" And then treat that time frame as a best-case scenario.
The thing is it's not *that*. First I take how long it should have taken and multiply it up to how long it's going to take. Then I factor in all the other things related to the project that I'm likely to get sucked into while working on it. Then I factor in all the other factors like staff meetings, client down, server down, network down, fire drill and whatnot. Try getting some experience data on how much time you get to spend doing what you're supposed to be doing, you might be surprised. Also if somebody asks you how long it'll take to put a roof on the house, always assume the walls and foundation will need work to not collapse unless you did it yourself. Mysteriously enough I never get to push my deadline despite it turning out to being a stick hut built on quick sand. Never assume that what you're told to do will be what you're doing, then most estimates will fail.
Re:I guess I'm not an expert then.... (Score:5, Insightful)
I know I suck at doing development estimates.
A struggle is getting people to even agree on what a development estimate is:
Me: "That will take 2 months of development work."
[two months of interruptions, putting out fires and "prioritization" later]
Other: "Why is this not done? You suck at development estimates."
Re:I guess I'm not an expert then.... (Score:2)
Re:I guess I'm not an expert then.... (Score:4, Insightful)
The first thing you did wrong is that you estimated 2 months, without taking any time to break down how you were going to spend each and every day of that two months. If you had done that, you would have realized you were falling behind schedule within the first week.
In my experience, any estimate that's longer than 1 day, and often even as little as half a day, generally should require breaking down, so that it is clear exactly what needs to be complete. You break the programming tasks down almost to an atomic level, so that every discrete function of the software is described, along with how long it will take to implement each one. Sometimes you don't know how long something will really take, but that's okay... the time it takes to estimate a project should be factored into the time it will take to complete it. Breaking things down at this level also gives you a clearer idea of the technical requirements to complete the job in the first place, which helps you design technical solutions as you make headway in the project. Further, it gives you a metric once you are partway through a project to determine based on how much of the project you've actually completed within a given time, whether you are even going to complete the project within budget, and if not, institute measures to minimize losses. In practice, you're not going to be right every time, or even necessarily close to being right, but when broken down to this level, the overestimates and underestimates should balance out reasonably well, with perhaps a tolerance of up to about 10 or 20%. If they don't, then there's something else fundamentally wrong with the project, and as a first guess, I'd suggest that it may be on account of unclear program requirements,
Re:I guess I'm not an expert then.... (Score:3)
Sometimes you don't know how long something will really take, but that's okay...
I agree with everything you said, but the point I was trying to make (poorly worded), is that time spent doing something other than development does not advance development. I always pad for the unexpected, but if you pull me off a project to do something else, then that project is not progressing. It sounds like a basic concept, but it escapes those who are not responsible for the actual development.
Re:I guess I'm not an expert then.... (Score:5, Informative)
That's why I always say "That will take approximately 270 hours of development work" rather than "That will take 2 months". Then you write down how your time is actually spent, and can document that after 2 months you've actually only had 20 hours to devote to whatever it was, so it's no surprise that you're a long way from finished.
Re:I guess I'm not an expert then.... (Score:5, Insightful)
I know I suck at doing development estimates.
A struggle is getting people to even agree on what a development estimate is:
Me: "That will take 2 months of development work."
[two months of interruptions, putting out fires and "prioritization" later]
Other: "Why is this not done? You suck at development estimates."
Then make sure you're not surprising them at the end of 2 months. If at the end of week 1 you go to them with "I go two days against the project this calendar week, we still have 38 more to go", they are in the groove for project time and calendar time isn't the same. And if they want them to be, they need to stop you from getting interrupted.
Communication. Verrrrrrry important.
Re:I guess I'm not an expert then.... (Score:3)
I've never been particularly confident about any estimate I give having a good chance of being accurate
I tell IT folk and non-IT folk the same thing: an IT estimate is the first point in time having a non-zero probability of being true.
They both appreciate the truth of the adage. Like somebody else said, multiply by pi. That takes into account the 'problem surface' around the vector.
External Dependencies (Score:2)
Predictions on the time it takes for me to do something can be off, but not by much. Most good predictions have contingency plans, etc...
In my experience, the biggest variability in estimates is the reliance on external dependencies. If I were the only person needed to work on something and I estimated 40 hrs of work, I would probably get it done in 30-45 hrs. But when that works requires someone else to do something at a critical point, even if it only takes 1 hr, the ability to acquire that resource in a timely manner ALWAYS messes with the time. Instead, the 30-45 hrs turns into 40-60 hrs. Amazingly, the "wait time" makes my time spent worse as well. I have to go through "ramp up" time again.
You can even schedule out that you will have the person for 1 hr a whole week ahead of time. But I have found it rare that you are able to acquire that resource remotely close to the time you scheduled.
Re:External Dependencies (Score:2)
Please have your ad blocker up before visiting the website.
http://tvtropes.org/pmwiki/pmwiki.php/Main/ScottyTime [tvtropes.org]
Not completely useless (Score:2)
Well, I at least have my wife trained to treat my time estimates as "no sooner than", and I don't have any trouble sticking to those commitments. Can't be that much harder to train your boss to have the same expectations.
Anyway, isn't most of Agile centered around coming up with time estimates formed from a consensus of team members who know you well?
I always follow Scotty's law (Score:5, Funny)
Re:I always follow Scotty's law (Score:2, Interesting)
I have a PM who actually does this. He takes everyones estimate by 'pi'. He says it works and has theories why it works. But just knows it does. In my exp he is right. Someone says 'it will take me 2 hours', it will take them about 6-8 hours.
Only with the dead easy tasks are people spot on. Anything else they are usually wildly guessing. Unless they have done it before (and even then...).
Re:I always follow Scotty's law (Score:5, Funny)
He takes everyones estimate by 'pi'....In my exp he is right.
If your PM takes somebody's imaginary estimate and multiplies it by pi and you exp it, your result will necessarily be complex, yet the error will be easy to bound with a circular range (even if with an initial wild guess)... Just say'n...
Re:I always follow Scotty's law (Score:2)
He takes everyones estimate by 'pi'....In my exp he is right.
If your PM takes somebody's imaginary estimate and multiplies it by pi and you exp it, your result will necessarily be complex, yet the error will be easy to bound with a circular range (even if with an initial wild guess)... Just say'n...
How dare you frame a humorous anecdote in statistics, and render it completely logical and un-funny.
Re:I always follow Scotty's law (Score:3)
Re:I always follow Scotty's law (Score:3)
Actually, doubling or tripling the estimate is USUALLY correct, the problem is that it's not correct if you apply it all at once. I've known managers that take any estimate and double it, but crucially, you don't allocate the effort all in one block.
If you need to code a widget, and it'll take you 3 days, realistically, that's just for the initial implementation. You can debug it, but that's no guarantee that it'll work as intended all the way until the end of the project. You probably have another 3 days of work to KEEP it working.
Overestimating is almost always the right thing to do, if you can get the people writing the schedule to understand that when you say six days, you mean three now and three later.
Re:I always follow Scotty's law (Score:2)
So you're saying that the real date of the events in Star Trek was actually the year 6795?
Re:I always follow Scotty's law (Score:3)
In not sure why anyone thinks this funny, because it's absolutely true.
No matter how much experience you have, there will *always* be that huge feature you initially thought would be a minor thing, there will *always* be those impossible-to-predict functionality hangups that take forever to solve and the client will *always* have "oh, yeah, and..." types of changes to the project requirements that completely alter the scope.
Scotty Principal... (Score:2)
Re:Scotty Principal... (Score:4, Insightful)
Re:Scotty Principal... (Score:2)
And when it invariably takes them 4 hours to implement the feature and their sales 'win' ends up costing them more than the value of the contract, natural selection works its way though and they can instead estimate how long it'll take to find a new job.
Re:Scotty Principal... (Score:2)
Testing (Score:3)
Is that really the problem? (Score:5, Insightful)
I often find another problem is management's refusal to accept the estimate of the developer. I am usually pretty good at estimation. Here is what usually transpires for me:
Manager: "How long will it take you?"
Developer: "2 months."
Manager: "You don't have 2 months. You only have 1 month. Redo your estimate. How long now?"
Developer: "2 months."
Manager: "You don't have 2 months. You only have 1 month. Redo your estimate. How long now?"
At this point I feel like saying:
Developer: "Why are you asking for my input? Just write down 1 month. And do you want me to tell you I will be 1 month late right now or in 1 month from now?"
Re:Is that really the problem? (Score:5, Interesting)
I was once invited to a meeting with the customer because my manager was sick. When people started talking schedule I casually mentioned the 18 months it would take to complete the software. The customer went ballistic. Apparently the schedule I gave my manager never made it to the customer.
I was never invited to a meeting with the customer again.
Re:Is that really the problem? (Score:2)
Re:Is that really the problem? (Score:5, Insightful)
Re:Is that really the problem? (Score:5, Interesting)
I've run into that. "If you want the lies told to the customers to be consistent, you must identify which lies you've already told them." Management didn't like my snark, but I received a briefing before future customer meetings.
Heh. I actually worked out a system with one of the sales guys I worked with. When he rubbed his eyebrow in a certain way it meant "I know I'm lying; please don't say anything that might undermine my lie."
In his case I was okay going along with it because he always had some (generally quite reasonable) backup plan that meant my team would never have to actually deliver on his lie. I was still uncomfortable, but he never burned us, and the customer always ended up happy.
Re:Is that really the problem? (Score:3)
Re:Is that really the problem? (Score:5, Informative)
He comes out with the following gems:
- "I believe your estimates are too high"
- "I've already committed to a delivery schedule with the CEO and Engineering Manager"
- "Well, we'll just have to challenge your assumption"
- "We'll just have to find ways to work smarter"
- "We'll just need to work extra hours then"
- "You're not showing enough committment", when asked to work on the weekend and holidays. This despite being with the same company for my entire working life
It's like I'm in a Dilbert nightmare now.
Re:Is that really the problem? (Score:5, Funny)
Indeed, it is a Dilbert nightmare.
This particular one, in fact. [dilbert.com]
Relevant quote: "Anything I don't understand is easy to do."
Re:Is that really the problem? (Score:3)
I'm deeply confident (Score:2)
That it'll take 2x-3x longer than it takes in my head. If there are no spec changes (i can dream, right?) or other surprises, maybe put that down to 1.5x.
When given a project, I'm sure most people will have a macro-level architecture thought up within minutes. It all seems so easy at that point! If you're lucky you get to spend a little more time in thought before being asked for a time estimate. If you're unlucky, well... in those cases I just multiply by 3. Underpromise, overdeliver and all that.
management (Score:2)
My team seems to do ok on the estimates. Then we get beaten into 1/2 that by management. Then in the end it takes twice as long as management expected. So the original estimate was good.
So we would be fine if only management did not try and squeeze it.
Management never accepts the "debug", "refactor" and "new feature" timelines, those are generally considered as "not needed". It just supposed to work perfectly and on the timeline they negotiated before consulting the people who would actually deliver it.
*sigh*
Proper scoping is EVERYTHING. (Score:3)
I agree. Fortunately for me at least, I happen to be in the happy world where management supports us in realistic timelines and realistic scoping.
Spanning almost seven years now and well over a hundred assorted projects we have been overdue on projects two times total. One of those was during the exceptional case of a co-worker getting in a car accident and breaking 13 ribs, the other was an exceptional case where very serious external forces caused the design to shift mid-development. In no case has it been due to poor estimation.
We have come to learn the development cycle for our small teams:
When I hear about other groups hitting 60% or later in their development cycles and still not getting feature complete, I pity them. They have made the mistakes the original article warned about, and were probably driven to that madness by the poor management you mention.
Actual article summary (Score:2, Insightful)
Blah, blah, blah. Bad estimates.
Blah, blah, blah. Oh noes! Waterfall!
Blah, blah, blah. Fixed by Agile!
Re:Actual article summary (Score:3)
Pretty good summary.
The solution seems to be "don't commit to a schedule longer than a sprint (even if that's only a week) and you won't be far off on the average".
Of course, this doesn't work so well with customers. A giant customer who is considering kicking your product out the door and replacing it with a competitor "if you don't get feature X in" wants to know when he can expect feature X. This is often easy for seemingly small projects (add a new style sheet), but not so much for "hard" (many tens of person years of development or more) which are relatively indivisible (a distributed system that only corrupts data in 20% of the cases for which code hasn't yet been written is about as useless as one that does so 100% of the time). In the hard projects, if you tell them:
their answer will be simple - "Thanks for the information, we are cancelling our maintenance contract and won't be using your system anymore. Please give me your vendor badge which we have just deactivated anyway."
Not that hard (Score:2)
But it's not really that hard to predict estimates where predictable and predict a reasonable time to determine if an area is predictable.
The RUP methodology is excellent for this.
1) You gather the feature set and identify the risk vs non risk portions of a project.
a) New technology.
b) Relying on develop of technology which doesn't even exist yet.
c) Performance.
2) You work on the risky items first. You do not start on the non-risk portions until the risks are mitigated.
3) Work in a time-boxed fashion. The time box can be 3 weeks or 5 weeks but deliver a working build each release. Note which features are not on track and drop them, adjust estimates, or even cancel the project.
And there is also baselining your coders. Over time, some will consistently be over cautious, under cautious, or on target. And by a consistent amount.
Let me put it this way...
How long would it take you to develop a sorting algorithm for a screen element?
How long would it take you to develop an import mechanism?
OTH, how long would it take to integrate your web site with an app using a new poorly documented library delivered last year?
I agree with the author that some things can't be estimated. But many things can.
The biggest problems I've seen are
1) Business decides the delivery date, features, and sometimes even the budget without consulting IT.
2) If a couple 70 hour weeks work to deliver in a crunch then always working 70 hours must be even better, right?
3) Business firing anyone that says a project is risky ("not drinking the koolaid").
4) This is a funny one.. They come and say, "How long will this take" and one person says 4 weeks and the other person says 2 weeks. So they consistently give it to the person who said 2 weeks. And then it takes them 4 weeks (sometimes longer).
It's not that hard (Score:2)
It's easy to manage time if you keep this simple law in mind:
The first 90% of the work will take up the first 90% of time, and the remaining 10% will take up the other 90% of time.
Scotty knows (Score:5, Informative)
La Forge: The Captain wants this spectrographic analysis done by 1300 hours.
Scotty: Starfleet captains are like children. They want everything right now and they want it their way.
But the secret is to give them only what they need, not what they want.
La Forge: Yeah, well, I told the Captain I'd have this analysis done in an hour.
Scotty: How long will it really take?
La Forge: An hour!
Scotty: Oh, you didn't tell him how long it would *really* take, did ya?
La Forge: Well, of course I did.
Scotty: Oh, laddie. You've got a lot to learn if you want people to think of you as a miracle worker.
- TNG 6x04
Either .. (Score:2)
For the most part, double your estimates and then adjust where it gets too costly and you know your players can perform fast than expected.
Scotty knew. (Score:2)
Lt. Commander Geordi La Forge: Look, Mr. Scott, I'd love to explain everything to you, but the Captain wants this spectrographic analysis done by 1300 hours.
[La Forge goes back to work; Scotty follows slowly]
Scotty: Do you mind a little advice? Starfleet captains are like children. They want everything right now and they want it their way. But the secret is to give them only what they need, not what they want.
Lt. Commander Geordi La Forge: Yeah, well, I told the Captain I'd have this analysis done in an hour.
Scotty: How long will it really take?
Lt. Commander Geordi La Forge: An hour!
Scotty: Oh, you didn't tell him how long it would *really* take, did ya?
Lt. Commander Geordi La Forge: Well, of course I did.
Scotty: Oh, laddie. You've got a lot to learn if you want people to think of you as a miracle worker.
It also helps you plan time for unforeseen setbacks.
So true (Score:5, Insightful)
I hate making estimates. I'm always, ALWAYS wrong. I always know I'm GOING to be wrong.
I've been trying to fix this for 12 years. I thought it was just inexperience talking, but I'm a grown-up programmer now. 'Senior', by some estimates. And yet I still have a hard time estimating the time of getting things up and running. I write one thing, and four things that I couldn't have anticipated crop up. This is particularly true in my industry (video games) where you're often working with an engine that's a few years old, and other people are in the middle of working on it, and specs are changing under everyone all the time. Things that look straightforward end up taking bad detours through networking components that nobody else understands because that part was written years ago and those programmers aren't around anymore.
Man, this story makes me feel a lot better about myself.
Re:So true (Score:2)
You need a "rule of thumb." I had the same problem until I decided that the minimum a project would take for basic functionality is 4 hours for each table in the database. If they need fancy ajax stuff or any eye-candy, then it goes up from there.
It's worked pretty well for the past few projects. I've even come in under on a couple.
Re:I've been trying to fix this for 12 years. (Score:2)
I know, insert prelim apology for sounding "arrogant" etc. Then let's thrash out a theory.
"I've been trying to fix this for 12 years." When something takes 12 years to get better at, there's hidden factors at play.
Suppose you try a thought experiment. Imagine one of your recent projects. So you get to the stage of the "estimate" (really some kind of pre-pre-pre estimate!) and imagine what you were thinking when you worked it out.
Then try to pin down at least a couple of the "oh my gawd" moments when the whole thing exploded. Clarify a little why that particular moment didn't work.
So as part of the thought experiment, the next time you get a project, make THREE estimates. (Feel free to add a couple of bonus ones). The first is private and not told to anyone. *Because you just throw an "insane" chunk of time on top of it*. Go wild! Three month project? Whee! Let's pretend it takes two years! And lo and behold, it came in at 10 months. Yay! You were "under your estimate!" That's your first private estimate - throwing so much time that it's designed to *not go over, with NO penalties*.
So then the second one should perhaps also be private - the one that made you *think* (wish?) it was three months. But that one will be too short, for all the reasons you said you've struggled in 12 years.
Then your third one is to build in contingencies for "nightmares" - "I don't know what it is yet but something awful will go wrong here."
Meaningless Summary/quote (Score:2)
Re:Meaningless Summary/quote (Score:2)
Hofstadter's Law (Score:5, Insightful)
Hofstadter's Law [wikipedia.org]:
It always takes longer than you expect, even when you take into account Hofstadter's Law.
Not overconfidence... Unrealistic expectations (Score:2)
In my previous life as a software architect for a small rural software development shop, I would try to give estimates and my bosses (all salesmen) would come back with, "I can't sell the client that!" Then when I missed the deadline or had to work weekends they were quick to blame me for giving an unrealistic deadline. My favorite line was always, "Give me an estimate, I won't hold you to it." Yeah. freakin. right.
Even better is when I would attempt to show them what other SUCCESSFUL development shops were doing. They would then give the excuse, "That's because their developers just sit and do nothing all day. Big shops have money to blow." Wrong. Big shops grow big because they know what they are doing. Eventually when they figured out my patience had run out, they dismissed this constructed advice as me just dissing the company.
I'm not sure why I stuck around with them for so long. I guess it was a sense of loyalty. Yet they tried destroying my marriage with their unrealistic estimates (and contracts that allowed clients to call me at home when-freakin-ever they freakin wanted. Guess as long as it didn't bother them or their time with their family, who cares, right?) Perhaps I just had a form of "stockholm syndrome".
My reasons why development estimates are hard (Score:5, Insightful)
People estimate based on how much time they think it -should- take, but you almost never estimate:
1. External factors which grow time
2. Feature/function clarification takes time
3. Outside resource turnaround takes time
4. QA may never be satisfied
5. We're moral and WE make a lot of mistakes along the way
6. Most likely, you don't know all the caveats of developing the piece of work until AFTER the development is over
7. General personal time spent elsewhere (meetings, consulting with co-workers)
Sadly, the best estimate for completion ends up being 1.5-2x longer than my original gut check, so as long as you pad out your estimates, you should be fine.
so called 'experts' (Score:2)
The so called 'experts' are just as much experts at estimating requirements and timing as they are 'expert architects'.
Here is a thread [slashdot.org] where I argue that J2EE is a crutch given to people labelled as 'architects', turning them into typists while removing any real architectural thought from their activities. If you read through the thread you'll see some AC objecting to that notion and he does not realise that he is arguing my points there when he talks about architects.
He is mistaking what 'enterprise' means, he believes it has to do with some technology, with some instrument, a tool or a set of tools. He does not realise that 'enterprise' really means an approach, a process, set of processes and standards that a company forces itself to adhere to, be it in implementation details or documentation process (all of which are important of-course), but enterprise does not mean just some solution provided by some vendor even if it has the word "enterprise" in its name!
So with that in mind, realise that what we actually have for architects are most of the time not architects at all, they are copy pastors, they are typists, they are managers probably, but they are not actually designers.
Those are the same people that would be considered 'experts', who managers turn to for time estimates. I don't remember myself underestimating projects at all or overestimating by more than a factor of 2, because I have the entire process of what it takes to build a project in mind and I break it down into all the little pieces, put a number that comes from past experience in front of that little piece, then the numbers are added up and there is some adjustment based on the team, the people that are going to be working on this.
AFAIC overestimating is not as a big problem as underestimating but if you are bidding on a job, then it does present a challenge. In case of bidding you are actually not truly estimating a project, you are just trying to get it before the other guys get it, and I think that's where the real problem comes in. Managing client expectations is a serious matter, you better be able to do that and I think the more you way overestimate or way underestimate the less likely the clients are to trust you in the future, so be true to yourself.
But again, how can somebody be true to themselves, when they don't even understand themselves what it is they are doing in the first place?
Level of Detail (Score:4, Interesting)
Back when Joel spent time on writing, Joel Spolsky of Joel on software had an interesting method for doing time estimates [joelonsoftware.com]. His point was to go into a deep level of detail. Instead of handwavy "code the GUI" the only way to really get anything remotely close to a real time is to estimate everything down to at least half day, if not lower granularity. It's not the "oh you feel the time better" as much as to think of EVERYthing you need. If you go to a lower level, you may remember that dialog box that you didn't think of at the 25,000 foot level.
It would be interesting to see if anyone ever used it to improve their estimates. Even he "disavows" it now, preferring the method in his software tool. But I like the "the world is a big place, are you sure you're thinking of everything" that the older method pushed you to.
Re:Level of Detail (Score:2)
My experience (Score:3)
Something my boss has us do when we estimating projects. She has a certainty factor that we set for each task, simple terms, which equate to a percentage in her calculations. The higher our certainty, the less risk that the task is underestimated. The lower the certainty, the larger the margin that the estimate needs to be factored.
Makes a huge difference in ballparking our estimates.
Estimates don't account for Risks and Unknowns (Score:3)
As per my blog post a couple of years ago at http://use-cases.org/2011/06/04/getting-good-estimates/ [use-cases.org] [use-cases.org] and http://use-cases.org/2011/06/22/updates-on-getting-good-estimates/ [use-cases.org] [use-cases.org]
Most good estimates have a range - and not a number, or a number with a confidence (both are interchangeable).
If an engineer says it will take two weeks - I push for a range or a confidence. If the range is weird (2-8 weeks), I push for the engineer to tighten their estimate through discussing or raising and discovering the unknowns or the risks that they are aware off. That sort of estimate would usually end up around 3-5 weeks which is a reasonable margin - and a lot better than than underestimating by 50%.
Same with estimates that are too narrow. "2 weeks +/- day". That implies a full level of understanding, no risk and no dependencies. Almost never happens. Work through the same risks/unknowns and the estimates usually become really bad - typically at least double of the "high confidence" estimate - similar to TFA.
There is a lot of reasonably applicable theory behind this (confidence intervals, cone of uncertainty, etc). But we don't necessary focus on mastery of our art...
Some overlooked bullet points (Score:2)
3 days for bitching, pissing and moaning.
3 days for dicking around on the interwebz
1 day lunch overages.
2 days for "zoning out"
3 days for witty banter.
Here's how it really works (Score:2)
Suit: "How long do you estimate development will take on this project?"
You: "My best estimate is 2 years, 3 months, as long as specs don't change."
Suit: "But the customer would like the product in a year"
Bean counter: We'll need 6 months to determine the task flow".
You: Then you'll need to add six months to the scehdule."
Suit: Okay, it's settled. We'll start tommorrow, the accountants will take the first six months to determine the charge numbers, and the programmers will have the job finished six months later.
Two days later.....
Suit drops by....p> Suit: "Hey, I just got off the line with the customer, and they changed all the specs. Don't worry though, I told them there wouln'd be any impact on the schedule. Oh, by the way, the accountants say they need another month. But I every confidence in you.
a year and a day later.........
Annoyed suits and accountants sitting around the meeting table...
Suit: Okay, now what the hell is the problem here?
You: We only had five months to complete the task, and specs and accounting time were all changed..
Suit interrupts: You told us you could have it finished, you aren't going to make it very far here - we need better estimates on your part!"
You: Sigh... Well, I think if we work everyone double shifts, we can get it out the door in anotther two months"
Bean Counter: We'll need a month to redistrubute gharges and funds."
You: "Wait! umm.."
Suit interrupting: Okay, that settles it. A month and a half out the door, just like you promised. And don't let me down, again. Just what to you programmers do with all your time anyway?
Estimate failure (Score:2)
During my last project, one component was estimated (by others) at 2 man months, and it ended up taking 6 full time developers a year to implement. The estimates were absolutely horrible. As much as it was the fault of the original estimate, management constantly rode the development team to get it done asap, which probably in the end did more harm than good.
This is doubly hard for consultants (Score:3)
When my customer comes to me and asks me to provide an estimate for a job, if I give them a conservative estimate, some of them may think that I am milking them with the extra hours. Specially if they get a competing estimate from an overly aggressive Indian company who is eager to sign the contract but has no clue on how to deliver.
I usually do not fret too much about customer feelings in a case like this. But during slow times I have little choice. Bottom line is, most of us would love to provide conservative estimates, but often times it is not as simple as that.
There was this company that wanted a project ... (Score:3)
... to be developed for them in 3 months. I estimated 10 months. So they decided to look around for another developer. A couple years later they came back and asked if I could do it in 6 months. I told them it would take 12 months, now.
Confidence (Score:2)
You can have confidence in your estimates and still be aware that that confidence is misplaced. One of the common things I keep saying to my manager is "Yes, I'm pretty sure we can finish this in 3 weeks. But I want to schedule it for 6 because always, always we spend half our time getting pulled off onto other things and I want to account for that now before we get in a bind.". I have confidence in my estimates, but I also have confidence in the statistical evidence of how reality varies from my estimates and I'm not prepared to ignore the latter.
As others have said, I also end up in arguments where people "up the chain" have already decided when they want something delivered and are pressuring me to make my estimates conform to the schedule they've already set. I don't consider this a problem with my estimates or my planning/scheduling, because I have no input into this or ability to control it. The problem lies with the people who're making promises without making sure those promises can be made good on, who then expect someone else to pull their chestnuts out of the fire. I can't do anything about that, because I can't order them to ask for estimates before setting delivery dates.
Uh no... (Score:5, Insightful)
My estimates suck because:
Project leader: Ok, so we need to know how long it will take for you to do X
Me: I'm not sure, that's an entirely new API, proprietary to the vendor, there's almost no documentation and their website has a support forum filled with questions and basically no replys to any of them.
Project leader: Well, we need a number.
Me: Why?
Project leader: I have to fill in this box here... see?
Me: Ok fine, 800 hrs
Project leader: Now hold on a minute, this wont take 800hrs
Me: It could, I have no idea. It's already taken the majority of at least one hour and I don't even know what language it's in.
Project leader: Fine, I'll put down 800hrs, but you're the one that's going to look silly.
POST PROJECT REVIEW ....
Project leader: I can see here your original estimate was 800hrs, and your actual billed time was 1265hrs. What causes led to you missing your estimate, and how can we avoid those in the future.
Me: Don't make estimates.
Project leader: Come on now, I need a real answer.
Me: Why?
Project leader: I have to fill in this box here... see?
Me:
Expert ISNT (Score:2)
Anyone who calls themselves an expert isn't qualified to call themselves an expert.
propose projects substantially already done (Score:2)
Estimation is a losing game. (Score:2)
Think about it.
Time is, for all practical purposes, linear. Your task will take a specific quantity of time to complete. You don't know that quantity of time in advance, because you don't control all factors, so you're guessing.
Now, what is the environment of your guess? You are trying to pick out a specific point in the future at which your task will be done.
Balance that against the infinite number of points of time in the unbounded future in which your task could actually be completed in.
1 estimated point against an infinite number of possible points. That's your odds of picking the CORRECT point in the future. 1 divided by infinity. Although it's not necessarily mathematically correct, it's a useful convention to reduce that expression to "0". And that is your precise odds of estimating the completion time correctly. Zero.
Re:Estimation is a losing game. (Score:2)
A very simple method that works (Score:2)
break the problem down into parts, carefully... (Score:2)
Break the problem down into parts. Carefully consider contingencies. Get creative - sleep on it. Think about it in the shower. What bullshit will crop up as I work on this project?
Using all your experience as to how long similar components took to implement in the past, plus how much longer it would have taken if worst case nuclear godzilla attack had occurred, compute time estimates for each component.
Add all the time estimates together.
Multiply by the Planck constant in Joule seconds / (pi^3 / e^2) * 10^34... approximately 1.58. It has something to do with brane theory and the double slit experiment. Trust me.
I dont think he has worked it out yet. (Score:2)
"The hardest part of solving a problem is understanding it" - ?
The reason its hard to estimate development time is because programming involves design, design is a creative task.
Nobody can predict how long it takes to be creative, its a universal unknown. Creative workers (such as graphic artists) often estimate the design phase by giving themselves a hard limit and then just choosing the best idea they could come up with.
Most programmers dont even acknowledge their work is a creative expression, so they are bad at estimating what a reasonable "hard limit" might be. But even so, im not sure the same method of 'choosing the best idea within a given time limit' is suitable to programming. Some things just have to meet certain objective benchmarks or there is no point continuing.
Best idea i can come up with is to allocate your self "design time" first, which wont be long enough. Then you should be able to get a reasonable estimate of implementation time.
All estimates are bullshit anyway, and here's why (Score:2)
Because by the time anyone who could come up with accurate estimate is asked for their opinion, the product has already been sold and the contracts signed.
Prioritise the order of development and get on with it. It'll be done when it's done, you'll get paid or you won't. Wasting time producing imaginary numbers has never fulfilled a contract, ever.
Here's how we estimate (Score:2)
Start by prototyping before the project is funded. This helps the business parties see what is possible and also gives the developers a much better understanding of functional requirements than any document ever could. Do UX testing sessions (test using your best developers - they find cool hacks, and business people, and anyone). Evolve the prototype, if even over a couple of weeks (I'm pretty good at rapid prototyping, two weeks is usually enough for 2-3 revisions resulting in a reasonably functional prototype for 5-10 screens). Prototyping and UX testing take experience, try them on a pilot project. Note, estimating hasn't begun yet. But do these things first, it's very important.
Then, once funded, start estimating. If the project is large (>1 person year), break it into logical functional components with a goal of at most 1 person year of "high" level estimated time (better estimates come next). Treat each of the functional components as a separate project. Large projects fail a lot, small projects are surprisingly successful (I don't have the article links handy at the moment, my anecdotal experience supports this as well).
Then, for each logical component, estimate. Estimating sucks, but I have seen the value in it, as has management where I work. It means sitting in a meeting room for multiple days per component. You will need at least one business representative, the development team, and a DBA, at minimum. Use the prototype to explore the functionality and system requirements. Start namespace/class definitions (helps a lot to categorize/define functionality boundaries). For each namespace/class, estimate the number and complexity of methods. Same for database stuff. Again, the prototype is a useful tool (never DB connected, generate data manually, Excel is a great code generator for such purposes).
Then put some time estimates on all of the tasks (think of it is as backlog, but for the project, not a sprint).
Continue estimating for each functional component (don't forget integration time between components, I pad these, it's always harder than you expect).
If you can't do this level of estimating then you do not understand software at a level required to make the project a success.
Then decide if the project should move forward. I've been involved in two hard estimating efforts (2-3 weeks each, huge projects, one that wasn't considered that big until we did detailed estimation) that resulted in management deciding against moving forward, it was obvious to us that this would be the result given the results of the estimation. Millions of dollars probably saved, large projects fail a lot.
As for daily work, we do two-week sprints, spending the first Monday doing nothing but backlog grooming (the crappiest day, but again, valuable). We strive to breakdown tasks to less than 8 hours, with 2-4 hours being the target (.5 hour tasks as very common, such as create Widget.Cache and Widget.Cache.Test projects). The sprint tasks are sacrosanct, they are the focus. If production support rears it head remove sprint tasks. We only schedule 50 hours of work time per 2 week sprint.
We do UX testing at the end of each sprint, enables us to be reactive to the results during the next sprint.
Yes, we are targeting agile methods and I'm not sure if they would scale too far. But, if a project is broken into functional components I don't see why it wouldn't scale, a large team would simply be composed of several smaller teams. Remember integration! It's usually a big part of the 20% of my 80/20 rule (where 80% of development takes 20% of the time, and the remaining 20% takes 80% of the time). Enhancement/support situations are something we haven't worked with much yet.
When starting a project, focus on "project killers" first. This includes things that are unknown, known to be hard or particularly complicated, or external integrations. Project killers are the things most likely to kill a project or directly impact the delivery date/budget. Do them first. Yes, it's usually plumbing, but one doesn't hang drywall before installing water pipes. If you fail at these the project can be ended somewhat gracefully.
For our 2 week sprints we usually land with under 10% of tasks unfinished (they get pushed to the next sprint). Overall we have generally been within 15% of our original high level estimates.
Management trusts us. Invaluable.
known unknowns (Score:2)
I too partially suck at estimates. Aside from the "unknown unknowns" which you can budget for but never predict I have found several rules of thumb have got my estimates from "fairyland guesses" to "accurate with withing a factor of 2". These are:
Thanks to all the other posters for their lists and suggestions. Time estimates are much much harder than development.
Re:I am confident thqt this is the (Score:4, Funny)
Re:I am confident thqt this is the (Score:2, Offtopic)
APK was arrested last night for disturbing the public and public indecency. Basically, his meds weren't adjusted (yeah, you probably noticed) and police officers don't like it when you masturbate in public, run around naked, and hurl your shit at them.
Well, there go about half the people on the Internet.
Re:Sounds like ... (Score:2)
Yeah, odd that being [www.ipcc.ch] right [arstechnica.com] keeps people feeling confident in their predictions...
Re:When The CEO Can Estimate Time To Profitability (Score:2)
Profitabily is 12 to 18 months out. Thats when the hockey stick curves up.
It was that way 3 months ago. It will be that way in 3 months.
Re:Looking waaay too deep in to the issue. (Score:2)
Re:Pi (Score:5, Informative)
I usually take whatever estimate I'm given and change it to the next largest unit and double it. Thus, an estimate of two hours become four days. This is still usually less than the actual time required. And don't even get me started on projecting when some task will be completed as opposed to how much effort will be required. The above alogorithm does a reasonable job at estimating effort actually requied but determining the calendar completion date is a whole different animal.
Cheers,
Dave
Re:Pi (Score:2)
Pi is funny, but I do triple it and it's about right. Unfortunately, it makes Management very unhappy.
The problem with estimation is something I've meditated on for a long time, and I blame it on several factors.
1. The users/managers saying "It's Simple! All You Have To Do Is...". Frequently accompanied by the pointed observation that "My 8-year old niece could do it."
2. The developers/designers saying "It's Simple! All I Have To Do Is...".
3. A refusal on the the part of management to accept an honest estimate. I used to tell students in the college lab that their printouts would come out faster if they grabbed the paper and pulled, and that's about what management tries to do. This works especially poorly when the honest estimates were already unrealistically optimistic. Give them a realistic number and they'll pressure you until you give them a number they like. And I guarantee it will be a lot smaller that the realistic one.
Where does the AYHTDI effect come from? In the case of the developers, some of it is ego, but some of it is ignoring reality. My suspicion is that development estimates are scaled on the unconscious premise of how long it would take to instruct a human to do something and computers are a lot stupider than humans when it comes to seeing what needs to be done and doing it on their own. Most humans, anyway. Well, some humans.
The other thing that contributes to false expectations is the Modelling Effect. In most fields, a model is visibly a flimsy representation of the final product that cannot possibly be mistaken for the final product itself. These days, most computer systems are all about the UI. A final-looking UI can be knocked together in a matter of hours often as not, but it's no more the final product than a mask is a human being. The difference being that it makes the application look like its finished long before it really is. Add the "instant gratification" frameworks that use can put together basic functions in an hour, even though the industrial-grade equivalent may take weeks, and there's all sort of room for disappointment.
Re:Too many factors. (Score:5, Insightful)
Actually, most of those things can be predicted. What is harder to predict is the creative aspect of development. Predicting a civil engineering project, like a bridge, is easy because engineers can compute the number of beams, the volume of concrete, the depths of the footings, etc, and they already have a good idea how construction people will behave, so they can add 5% for vacations, 20% for staffing difficulties, 10% for late trucks, etc. Predicting the creation of a new piece of software is less certain, because so many of those pieces are unknown. You make an initial survey of the requirements, and take an educated guess at what the solution might be, but you know that's never the final picture of the real solution.
If you're on a mature Agile team, you trot out your velocity, map your epics to some t-shirt sizes, and do some simplistic multiplication. But you also know to announce the estimates in terms of your team's delivery dates, and you don't overpromise. At this point, either management trusts your team's reputation and you boldly go forward, or you give them estimates with confidence levels around 20%, because you simply can't be more accurate than that.
If you're on a waterfall team, any software development estimate with an accuracy figure of higher than about 50% should be viewed with suspicion, and anyone claiming 90% or higher should be flagged as a true bullshit artist.
Because civil projects never go over budget... (Score:4, Informative)
Predicting a civil engineering project, like a bridge, is easy.....
I'm going to stop you there, because civil engineering projects are NOTORIOUS for going over budget. You might have heard of projects like the big dig [wikipedia.org]. Less well know, is that going over budget in less spectacular ways is apparently a fairly common occurrence. I was looking around for a report to link for you that I read awhile back talking about why civil construction projects so frequently go over budget, but alas, I cannot locate it.
Alas!
Re:Because civil projects never go over budget... (Score:3)
I'm not disagreeing, but I question whether this is something that happens more with really big projects which are very unique (Big Digs aren't exactly a regular occurrence, hence the name), rather than with your mundane, everyday civil engineering projects like a boring commercial building that's not much different from dozens of other commercial buildings in its area.
Re:Because civil projects never go over budget... (Score:3)
I'm not disagreeing, but I question whether this is something that happens more with really big projects which are very unique (Big Digs aren't exactly a regular occurrence, hence the name), rather than with your mundane, everyday civil engineering projects like a boring commercial building that's not much different from dozens of other commercial buildings in its area.
Well then, since you are using generic, cookie cutter building projects as supporting evidence for your argument, wouldn't it be an accurate comparison to look at how often web design firms go over budget when building generic 5 page websites for small businesses? I suspect that the numbers will be roughly equal, and the type of work would be similar. In contrast, an unusual project like the big dig would probably be comparable with someone like Microsoft writing a new OS from scratch.
My point is that software development isn't really that different than a lot of other engineering projects. Building things is hard.
Re:Too many factors. (Score:3)
Construction has very real material costs -the beams and concrete you talk about - so every component is drawn and specified. In software development the material costs are virtually zero so you might as well build it twice, once to understand it and once to productionize it, it's the same as writing a detailed spec and then coding it.