Are There Limits to Software Estimation? 225
Charles Connell submitted this analysis on software estimation, a topic which keeps coming up because it affects so many many programmers. Read this post about J.P. Lewis's earlier piece as well, if you'd like more background information.
Unless it's a simple project... (Score:5, Interesting)
2x+7 (Score:5, Interesting)
After gaining a few years of "real world" experience in software engineering (and I know that the very term real world experience is debatable
Accuracy, and doubling a double (Score:5, Interesting)
Here was his answer I have ALWAYS found it accuraye to +/- 10% so far on hundreds on small to massive projects.
1. Once you know all , or most of the forseeable estimates take that number. say 10 hours.
This number is an instinctual reaction to a perfect enviroment , a little experience, some ego on your part of what might be accomplishable in a vacum.
2 Take that Number ad double it.
This takes into account all the real world distractions. Events, etc.
3.Take that number and double it again. This takes into account unssen variables and events beond mortal control.
40 Hours.........
I use this on EVERY single estimate I provide, WHY ?? It works, its not too high not too low, just right.
I tell people this and they laugh, then I tell them that there are MANY legacy applications SSI, IRS, FBI, you name it that were qutoed by my father in this EXACT manner.
There is NO practical limit to estimation, As long as you have the information neccesary to determine what the job youre actually doing is.
Gut level formula (Score:1, Interesting)
Take your gut feeling about the development time required.
Multiply that by 20.
Divide the result by the number of years you've been developing similar projects.
it develops over time (Score:3, Interesting)
How is this possible? It has evolved over time, some of the same people who started this project 9 years ago are still here and they know the system very well. That knowledge, combined with good project management leads us to several categories. During a requirements phase, designers assign a complexity to the changes for a module, and based on the type an hours extimate is generated.
Now, Lewis is right, no algorithm can be developed to figure out the compleixty, but a human can, and the computer can figure out how many hours should be devoted to documentation, coding, and testing.
My overall point, as a software product matures...esitmates are easy to estimate and project dates are easier to meet. But you already knew that...
Who is the article for? (Score:2, Interesting)
What I would like to know is, how is this going to effect expectations from non-technical people in charge of projects that demand "accurate" estimates. I've had good and bad managers, maybe these kinds of articles will help make developers life a little less stressful and more flexible.
Programmers intuitively know this (Score:5, Interesting)
I'm glad there's finally a resource to help the folks who insist on accurate estimates understand why my response to the inevitable inane question is always a cynical "two weeks", regardless of the complexity of the problem.
How many angels can... (Score:2, Interesting)
You can prove philosophically the limits of mathematical methods,. but that doesn't make them useless. A formally-proved system, when put in contact with an informal world, may show itself to have limits, but it'll probably perform better than a system that's not been formally proven, and if it does fail, the reason for the failure will be glaringly obvious.
We build systems of ever-increasing complexity with tools that are constantly playing catchup. Does that mean we ignore the tools? I don't think so. Instead, we reflect and improve.
This is the wrong way round. (Score:4, Interesting)
What usually happens is you get told roughly what to build and the final date by which it needs to be ready. There then takes place a series of negotiations and compremises on the scope of work until everyone is "happy".
I suppose that doesn't really invalidate the point of the article at all, it's just an observation for those who think that estimation is the nice science that it is sometimes presented as being.
Re:Accuracy, and doubling a double (Score:2, Interesting)
> take four days.
Erm, right. Your programmer gets run over - you have to get someone else in to do it, you have to find them, interview them, get referees - now the project has taken over a week.
You may estimate 1 days worth of work - but in reality you have to try and plan for the unexpected, which is what project planning is all about.
Write a list of everything that "could" go wrong, and any bets someone, somewhere has had it happen to thier project. Just a quick 2 minutes thought and I come up with :
1) Personal Problems - Birth, death, illness
2) Transport problems - Train strikes in the UK for example
3) Electrical problesm - Power cuts, workmen cutting power lines
4) Servers dies - backups are off site and will take a day to recover
They may not all happen to the same project, but if they do you will wish you doubled that estimate again.
Re:Problems with scheduling (Score:2, Interesting)
...and...
Good people are hard to find, but the project manager can be reasonably expected to know who is working on the project and how competant they are, and plan accordingly.
...are DREAM statements in my experience. When contracts are bid, most (many?) are bid before the team has been completely assembled. The project I'm currently on I came in on as a sub contractor years after the start and 6 months before we delivered a release (which ended up being about 7-8 months late). Our staff has changed a bit even since I came on board. Hirings, firings, quitting, etc, change the way a project not only delivers, but works together. You get an annoying QA guy that rejects everything you want to implement, that's bad. You get an annoying QA guy that implements EVERYTHING everybody implements, that's just as bad.
One idealogy that contracts (government contracts especially) seem to embrace is "If we put more people on the project, it looks like we're trying to produce more work." AND THIS ACTUALLY PASSIFIES GOV. AGENCIES! My project was a good case in point, we added a ton of people around the same time I came on and the productivity...dropped, because not only did we have to do our work but the work of the other people (who aren't on the project anymore, I might add).
Let me rephrase/clarify...you bid X amount of time. (because to be honest, the people that place these bids ALWAYS underbid time...they're competing, after all, on who has the lowest cost/time/best design, and extensions are easy to get than contracts). You then have an internal deliverable schedule that is reduced by 80%. IME, people will always be late, but they will try to hit the mark, so if you attempt to finish at 80% time, you may actually make it by 100%.
Maybe if you had a reliable team ahead of time, could garantee they were all sticking around from the beginning to the end, had competent managers and especially designers and a time frame that was reasonable, you could apply an algorithm to find out what kind of schedule you could deliver within. And if you ever find that, please email me...I want in
--trb
My critique (Score:4, Interesting)
The glaring flaw of the paper is that the main argument can be applied equally to any human endeavor, not just to programming. The argument is essetially a rigorous version of the statement, "You can't (in general) know how hard (complex) it's going to be, until you do it". The author supports this by pointing out that the purpose of any program is equivalent to generating a string that is a complete, precise description of the problem. Complexity theory tells us you can't predict the length of that program (without a formal system bigger than the program).
But it's not hard to cast any problem into this form. Take baking a cake. The problem can be thought of as generating a precise description of how to turn some inputs into an output within the range of what we consider a cake. In a reductionist sense, that process is incredibly complex (much more than any computer program), involving gazillions of elementary parcticles and their interactions. But nonetheless it's pretty easy to estimate how long it will take to bake a cake.
Complexity theory shows us that complexity is indeed pervasive in general; but everyday experience shows us that it is usually encapsulated within simple abstractions. Most things we plan and do have relatively simple descriptions in terms of objects with those properties we are familiar, and things we have done countless times before. So while estimating complexity may not be possible in general, it is usually not very hard for the things we care about.
In order for the paper to be persuasive, Lewis must show that computer programming is, in practice, more complex than most other activities--that new problems can't be easy stated in terms of already solved problems. (He does begin to address this, but only as a side-note.) I think most practitioners would essentially agree (and I'm not going to argue this, unless someone challenges it). What does this mean for the relevance of complexity theory? It's a deep and difficult question, but I suspect that some insights can be drawn. In particular, I do believe that there are problems that can't be estimated without effectively solving them.
Regardless, there are more obvious, intuitive reasons that complex activities are difficult to estimate. One is that that humans vary wildly in their efficiency at complex tasks. We all know the experience of cracking nut after nut one day, and being stumped the next. Sometimes, to be sure, this is due to misestimation of difficulty, but just as surely it is often purely psychological. Another is that teams working on complex problems are prone to miscommunication and other group disfunctions. A third is simply that the flesh is weak--we often lack the discipline and concentation to plan our projects in sufficient detail.
And this list only considers the difficulties that derive from complexity. Software development faces a host of additional "accidental" challenges, such as bugs in third-party software, clients (and marketers) that change their minds, changing fashions in tools and methodologies, etc. In short, you don't need a fancy theory to conclude that predicting development time is quite hard!
Re:Unless it's a simple project... (Score:3, Interesting)
But you can go further with this! Any short-cycle iterative model will give you this sort of feedback every couple of weeks, rather than at the end of every version.
For more info on it, take a look at Rapid Development's [construx.com] discussion of evolutionary delivery. Or try a process that's built around feedback, like Coad's Feature Driven Development or, my personal favorite, Beck's Extreme Programming (XP) [jera.com] (dumb name, but a great process).
Art VS Engineering (Score:3, Interesting)
Programmers that I've worked with have almost always intuitively known this to be true, and non-programmers (in particular, product managers responsible for scheduling) have almost never understood this.
Those in the "Programming is an Art" camp tend to agree that there is no real way to estimate how long doing something new is going to take.
Those who think of programming as simply bulk engineering, repetetive, boring, or just "coding" tend to be frustrated by this seeming fact. It is almost irreconcilable with normal business practices to know how long a job will take until it is actually done. This makes it extremely difficult to make close-ended contracts, and to predict budgets.
Asking how long a particular software job will take is often equivalent to asking how long a research job will take.
Im sure the scientists would be amused if a suit walked down into R&D and asked them when they would be "done"
Re:Estimation isn't all that difficult (Score:2, Interesting)
First, detailed specs aren't available in most cases. Clients want a solution, and they don't want to be bothered with gathering requirments, reviewing proposals,etc. They believe that's your job, not theirs. Even if you get a buy-off on a spec, it will be done without much thought and will change once they see your solution.
Second, the estimates of the smaller tasks will be wrong. You will have to rely on people who are intimately familiar with the code to make the estimates, and some of them aren't good at estimation. If you try to do it all yourself it will take you years to even comprehend the existing source-code base.
Third, sickness, turnover, etc., will cause unexpected delays in many cases.
Fourth, unexpected problems---a bug in a vendor tool that has to be worked around, for example--- will cause further delays.