Software Defects - Do Late Bugs Really Cost More? 125
"If you're a software engineer, one of the concepts you've probably had driven into your head by the corporate trainers is that software defects cost logarithmically more to fix the later they are found in the software development life cycle (SDLC).
For example, if a defect is found in the requirements phase, it may cost $1 to fix. It is proffered that the same defect will cost $10 if found in design, $100 during coding, $1000 during testing.
All of this, to my knowledge, started by Barry Boehm in papers[1]. In these papers, Mr. Boehm indicates that defects found 'in the field' cost 50-200 times as much to correct as those corrected earlier.
That was 15 years ago, and as recently as 2001 Barry Boehm indicates that, at least for small non-critical systems, the ratio is more like 5:1 than 100:1[2].
[1] - Boehm, Barry W. and Philip N. Papaccio. 'Understanding and Controlling Software Costs,' IEEE Transactions on Software Engineering, v. 14, no. 10, October 1988, pp. 1462-1477
[2] - (Beohm, Barry and Victor R. Basili. 'Software Defect Reduction Top 10 List,' Computer, v. 34, no. 1, January 2001, pp 135-137.)"
Thigs they don't tell you ... (Score:5, Interesting)
At any stage, you can only find bugs that are introduced at or before that stage. So while fixing a requirements bug in the coding phase might be more expensive than fixing it during the requirements phase, fixing a coding bug during the requirements phase is a tricky operation that I'll leave as an exercise for the reader :-)
Of course, if you omit some of these phases completely, you won't introduce any bugs during them. That's why the JFDI(*) methodoloy is so popular.
(*)Just F*cking Do It
Re:Thigs they don't tell you ... (Score:2)
s/Thigs/Things/
s/methodoloy/methodology/
Oh, and I saw JFDI originally in an article in Computer Weekly, I think. It's also here [orangeandblack.co.uk]
Re:Thigs they don't tell you ... (Score:2, Insightful)
But then again, what methodology does work for those cases?
--
Real computer scientists don't program in assembler. They don't write in anything less portable than a number two pencil.
Re:Thigs they don't tell you ... (Score:1)
Oh, the whole project was 50 people * 3 years.
Re:Things they don't tell you ... (Score:1)
Get a phone call at 17:00 from a section that you have never heard of and get a project dumped in your lap to be ready by 10:00am, now that is a JFDI.
Re:Thigs they don't tell you ... (Score:4, Funny)
Not if you are a company like Microsoft or Sun and you let your customers do your bug hunting for you, for free.
Of course, that's also how bug hunting works for open source software, but with OSS, at least I don't pay anybody for the privilege of finding their bugs for them.
Re:Thigs they don't tell you ... (Score:2, Funny)
it's called 'Incremental Development'
Re:Thigs they don't tell you ... (Score:2, Insightful)
Re:Thigs they don't tell you ... (Score:2)
Re:Thigs they don't tell you ... (Score:2)
Yes they do. (Score:5, Informative)
There's plenty of proof out there. Even "ancient" but worthy texts like "The Mythical Man Month" discuss this one.
The size of the project and the nature of the bug really combine to drastically affect the outcome.
For me personally we have just spent about a year tracking down a particular set of bugs (probably not all nailed yet) which showed up post-live. When we were pre-live these would undoubtedly have been easier to fix, but something else that we could have done at that point would have been to improve our design, which would have nuked most of the bugs completely. Once we are in production however we have this forward/backward compatibility heuristic tying one hand behind our backs, and redesigning the thing gets much much bigger.
But that's just anecdotal, of course.
Also, bugs take $$$, who should pay? and ethics? (Score:5, Interesting)
Most recently I've been tracking down an error in our system. After nearly a month of trying various things, I found the problem of an error. In this case, two years ago the hardaware engineer building the FPGA and DSP programs didn't bother to fix the [relatively simple] design problem. Rather than give all communications the same format, a few commands differ substantially from all others (different responses in certain circumstances, for example).
The problem made it into the PC software that interfaces with the board. The problem is documented in several [maybe 20?] bugs of the software that works between the PC and the external device. The problem is documented in at least 50 bugs in a port of that PC software. It has been in production for several years, and implemented by external companies (which I feal sorry for, due to the complexity of the communications bug).
Now we're working on a completely new FPGA/DSP board to replace the earlier board. Design changes prevent us from directly implementing the bug in the new design, although otherwise the communication protocols are the same. Implementing the same malformed communications will mean breaking the simple straightforward design and carefully implementing a set of 'design exceptions' (read: 'bugs').
It would have taken one engineer an hour or so to fix this thing when they first saw it. It would have taken both teams a few days to fix it when writing the PC to DSP interface (~1 FTE month). It would have taken a few weeks to fix it when writing the port, requiring changes to the PC software and the DSP (~1 FTE year). If we choose to fix the error now, it will probably result 2+ FTE years of work to just fix everything, and more time for regression testing every old peice of software for this one bug. If we choose to leave it in, we will devote at least that much time in evaluating, implementing, and testing the old errors. Not to mention the continued maintenence work when the eventual bugs are found in the new board.
Now we're forced with a tough financial decision: do we spend a month or more carefully re-creating and testing the 'design exceptions', (probably 3-5 FTE years in total) or do we do it 'the right way' and break both our own and our customers' software? (again, several FTE years, but potentially loosing faith with the customers.)
This particular bug could have been prevented by about $50 of work. It has now cost the company tens of thousands of dollars, and will probably cost a few hundred thousand before all is said and done.
Now, lets throw some financial ethics into the $50 --> $5,000 --> $50,000 --> $500,000+ problem: The engineer was in a hurry to fix the problem before a company imposed deadline. Is that engineer responsible for the enormous financial cost? If so, how much? If not, why not? It can be argued that his negligence cause a half-million dollars in damages. It can be argued that the engineer was responsible for $50 but the team was responsible for allowing it to grow. It can be argued that this is a regular business cost due to falibility of engineers' designs.
This begs the question:
How responsible are any of us for the errors we introduce?
frob
Re:Also, bugs take $$$, who should pay? and ethics (Score:3)
If the engineer was rushed through the design and nobody had the time to check his designs, the company gets what it deserves.
However if the engineer skimped his work (for whatever reason) and the team failed to check his work, I think the team would share some kind of responsability.
If the engineer made the mistake and willfully let it in, he and the team should both partialy liable (he because he left an obvious flaw in and his team for not checking it)
This is also why I n
Re:Also, bugs take $$$, who should pay? and ethics (Score:2)
You say the protocol has exceptions instead of always being the same. Do you KNOW that the exceptions were put there to get around a bug? How do you KNOW that a fix for the bug existed - maybe that fix was the addition of the protocol exceptions because for technical reasons there wsa no other solution available to the engineer. Do you KNOW that the hardware engineer saw the bug - or e
Re:Also, bugs take $$$, who should pay? and ethics (Score:2)
Yes there are two sides, but in this case the other side does not fit any of your reasons. I kno
Re:Also, bugs take $$$, who should pay? and ethics (Score:1)
But perhaps something as important as command protocal the should have been designed before hand?
What software engineering should mean (Score:2)
That's an interesting idea that goes right to the heart of how software development is done today. Realistically, at present, a company will have to be accountable for the errors, because if it were all pinned on an individual developer, no-one would risk taking on the job.
In a better world, software engineering (currently a rather offensive term to real engineers, and on
Re:Also, bugs take $$$, who should pay? and ethics (Score:2)
"Begging the question" aka "circular reasoning", in argument, means that you assume that a statement which depends on the conclusion is true, and you use it as proof of your argument.
My rhetorical argument was based entirely on the premise that software engineers must bear responsibility for the errors they introduce, and therefore they are at fault for the errors. The belief that software engineers are responsible implies that they are at fault, the
Re:Also, bugs take $$$, who should pay? and ethics (Score:3)
"Begging the question" [infidels.org] is not a synonym for "circular reasoning" [infidels.org]. "Begging the question" simply means that your argument is based on questionable premises. "Circular reasoning" means specifically that you're using your conclusion as your premise. Circular reasoning may be begging the question, but begging the question is not necessarily circular reasoning.
In this case, I think you may be right, it's both circular reasoning and be
Re:Also, bugs take $$$, who should pay? and ethics (Score:1)
Other than the infidels.org site you presented, all other descriptions of the two that I have seen, they are synonymous.
And I agree with you and have done so with others in the past: it would be good if there were more real argument and debate on /. as opposed to just contradiction. [Homage to Monty Python goes here.]
frob
Re:Also, bugs take $$$, who should pay? and ethics (Score:2)
No it wouldn't!
Re:Also, bugs take $$$, who should pay? and ethics (Score:2)
[mental note: Ignore obvious trolls/nazi attacks like this in the future.] Um, yeah, that was exactly what I said, and it's exactly what I meant. That's why I said it was begging the question.
Re:Yes they do. (Score:2)
Re:Yes they (still) do -- but 15 times? (Score:3, Insightful)
Today most people download their software throught the Interne
Re:Yes they (still) do -- but 15 times? (Score:2)
Re:Yes they (still) do -- but 15 times? (Score:2)
But that's the cost of updating the software. What we're all missing is the cost of the bug. Not the cost of finding and fixing the bug, but the cost of the bug itself.
Let's say that you're working for a bank and their wire transfer software delvelops a bug during the end of month or (even worse) the end of quarter period. There may only be 5000 to 10000 transactions
Re:Yes they do. (Score:1)
I've not read the particular references sited and I've not seen anyone care to speculate on the nature of the cost vs time curve (i.e. exponential or logarythmic) but my take on this effect isn't so much about bugs in code as it is about errors throughout the entire SDLC.
Here is an example. Let's say that the architect for project X decided to use a file based ISAM instead of a relational database to persist the data. Think how costly the fix would be if this error wasn't discovered until the part of th
Trade off (Score:2, Interesting)
Nevertheless a trainer is correct in stressing the golden think-before-you-code rule - especially when instructing unexperienced coders.
--
Every program has two purposes -- one for which it was written and anoth
Re:Trade off (Score:2)
It is equally easy to have a formal review on your design as it is to have a formal review on your code.
While you "plan" software you should have a clue how you want to esnure that you have: a) what you want, and b) how you want it, in every single stage.
A standard way is:
Use Use Cases to capture requrements
Use Scenarios to refine Use Cases and lay the foundation for later test cases
Use Wa
Re:Trade off (Score:1)
I wonder if the b
Re:Trade off (Score:2)
Consider the code:
void foo(char *input) {
int a; char buf[25]; int b;
strncpy(buf, input, 25);
}
Let's consider the case of having an input string 26 bytes long including the NULL terminator. If strncpy has an off by one error it'll NULL terminate the string (
Bug != Defect (Score:2)
Re:Trade off (Score:1)
1) A lot of times, a "coding bug" is actually the result of an undocumented assumption in the design that conflicts with an undocumented assumption in the requirements. Code bugs can be prevented in the earlier phases of a project by providing as much detail as possible in requirements and design.
2)Also, a lot of times people call it a code bug because they are too lazy (or don't have the audit trail available) to determine at what point the bug was actually introd
Logarithmic (Score:4, Informative)
Re:Logarithmic (Score:1, Informative)
Re:Logarithmic (Score:2)
Logarithmic progressions are exponential.
No, they aren't. Think about it. For any n > 0, base > 1, log(n+1)-log(n) < log(n)-log(n-1), whereas exp(n+1)-exp(n) > exp(n)-exp(n-1).
The parent was correct; the examples were 10^n for n=0,1,2... This is exponential, not logrithmic.
-- MarkusQ
Re:Logarithmic (Score:1)
That said, such progressions are generally said to be logarithmic. It is just a convention. Like the fact that geometric and arithmetic series can be thought of as division and subtraction respectively, but we generally don't.
No. (Score:2)
Exponential up, logarithmic down. They are opposites, like plus and minus. If a progression is exponential, it is logarithmic, and vice versa.
No, they are inverses [2dcurves.com]. If the mapping from a the range to domain is exponential, the mapping from the domain to the range is logarithmic. This does not mean that there is some value A' for every value A such that A^B is the same as log-base-A'(B) for all B. The functions are not the same.
-- MarkusQ
Re:Logarithmic (Score:2)
No, if a progression is exponential then the reverse of that progression would be logarithmic.
You're confusing "opposite" with "inverse". The next step after +n vs. -n and then n vs. 1/n might be e^n vs. logn.
Look at it another way. Exponential series grow faster than any polynomial. Logarithmic ones grow slower than any polynomial one. The two cannot be the same: arguably the most fundamental properties they have are opposites.
More references... (Score:3, Informative)
There's also material in Watts Humphrey's book on the Personal Software Process [amazon.co.uk] (about as far from XP as you can get). That book is illustrated throughout with statistics about students who tried to complete the exercises in the book, including in Chapter 13, where there's a section on "The Costs of Finding and Fixing Defects.".
Re:More references... (Score:2)
Yes (Score:3, Interesting)
to
the cost of just building and shipping new code
that has yet to undergo testing or launch.
To give you an idea, managing the testing and upgrading over-the-air softare in mobile phones can become a new project in its own right with all the associated monitoring and overheads.
Fixing the bug of a pre-launch project can be a 1 minute job.
Re:Yes (Score:2, Informative)
Agreed. I've always thought Barry's estimates were on the money.
Comes down to what you consider a bug ? If you think a bug is a spelling mistake on a web page then 1:5 is probably not far off (but bad enough!!!) If a bug is bad design decision in a mass market product then 1:1000 might be a bit on the light side...
Don't look back! The lemmings are gaining on you!Re:Yes (Score:1)
Posit: It costs more because where one massive design problem has been built into the system, there are likely to be several (dozens?) more just as bad or worse. Fin
Re:Yes (Score:1)
Costs accumulate (Score:5, Insightful)
Never forget that complexity accumulates. Fixing the bug itself probably costs about the same at every stage, but other costs are introduced as the project moves along, and peak after the software has been deployed.
A bug found after deployment has costs associated with it that a bug found during coding does not:
The cost of finding and fixing the bug may be negligible compared to other costs.
Another aspect of the issue is the nature of the bugs you find late. In my experience, bugs that survive testing and deployment tend to be either bugs in requirements or pretty subtle bugs that slipped through testing, and both are more expensive than the type of bugs commonly detected early on during development.
Re:Costs accumulate (Score:4, Insightful)
I can't point you at any studies, but I think it's common sense. In anything but a fly-by-night shop, the later in the cycle you are, the larger the ripple-effect is from making any change.
If I can fix a bug in my code before it gets to QA, QA has never seen the bug. There's no bug in the bugtracking database, there's no need to review the bug at a weekly cross-functional bug triage meeting, and there's no need to write specific regression tests that specifically make sure that bug is fixed. There's also no need to perform those specific regression tests on every build that follows to make sure it's still fixed. There's no need to hold a meeting to justify the cost of fixing the bug versus the cost of simply leaving it in and documenting its presence and its workaround. Just there, I've saved a ton of time.
The costs explode even higher once the software is in the field. Once it's in the field, it hits support every time the bug is reported in the field (multiple times, usually, as of course level 1 support is usually going to blow off the report, tell them to reboot, or whatever the "filter out bogus complaints" method of the week is). Finally it might bubble up through support but only after it gets seen multiple times, costing us money every time. Then it gets argued about by who-knows-who (more time/money) until finally someone tells development it's a bug, and then we have to hold meetings and decide whether it's important enough to fix immediately, whether it should go in a service pack or just the next version, etc. We have to write up a technical bulletin and distribute it, that bulletin has to be reviewed by documentation, product management, QA, and who-knows-who else. Then QA has to specifically add test cases to make sure the fix is there in future versions, etc.
The costs explode. Again, in any sort of large corporate environment, a cost difference of 100:1 seems completely reasonable to me.
Re:Costs accumulate (Score:2)
I think the key point behind your post is that you're considering the overall costs to a large scale outfit, including all the supporting infrastructure the company has to create to handle bugs from QA and customers. Perhaps a lot of people here are only considering the time spent directly by the development and QA teams themselves.
OTOH, with a smaller scale outfit the cos
Re:Costs accumulate (Score:3, Insightful)
Re:Costs accumulate (Score:2)
That was a great post, but I do disagree with this one part:
I think one of the reasons fixing bugs later on is more painful is that the developers who were intimately familiar with the code when they wrote it will have moved on, and will take time to get back up to speed. Worse yet, the original developers may have left, and unless they were of the rare breed who both write good comments and leave compact but complete high level docume
Have you watched fight club? (Score:3, Interesting)
I would expect the same kind of factors come into play when the product is software instead of hardware. So why not try google [google.com]
Sometimes it costs less to pay a person to manually correct data that is incorrect due to a fault in the core of a product, sometimes it's cost less to do a re-write.
Alistair Cockburn (Score:1)
"The exponential cost curve is mostly in detecting and communicating the Mistake and naming the change that is to be made. XP cannot change that curve, and indeed, XP takes that increasing cost curve neatly into account. So the first lesson I get is that one should not
A recent example (Score:1)
Even disregarding the direct developer cost of finding and fixing the bugs, take a look at a recent example. Enter the Matrix seems to have been plagued with bugs (although I must admit that I haven't played it yet myself). It's "reputation" is now far from good, and so the bugs has caused a large loss in potential sales.
Now a hypothetical example. You work for a large company, and you're looking for some enterprise-level database software (HypotheticalDB). You get some that has been hyped, and spend a fa
Re:A recent example (Score:1)
Larry Ellison's Solution to Version 2.0 Problem (Score:5, Interesting)
Re:A recent example (Score:2)
Good point -- Backend bug fixin easier today (Score:3, Interesting)
Now we all know *poof* is not the case, and we all know that a well-factored system is about as hard to come by as nirvana (which means each fix requires ripping out a chunk of code), but the argument is still a valid one. Unless you have a huge system, where perhaps someone's "fixed" a bug by hack on top of hack ("Hrm, Bob's addFunction always returns a number one too low. Instead of bugging Bob, I'll just add one to the result in my function."), bugs today aren't like bugs in pre-object oriented days. If coders in the 80's had the debug tools and langauges we have today... Let's face it, it's much easier to create an Atari 2600 game today [io.com] than it was when you had to burn to an EPROM to test on hardware each time and print out your code to review it.
The bottom line is whether it's more cost-effective to prevent 99.44% of bugs up front than it is to fix the extra 10% that slip through. I believe the original post is simply suggesting that the cost of fixing on the backside is dropping considerably, especially compared to what the same results would've required decades ago, and that is, honestly, a good point.
(Remember, this isn't upgrading code -- might be awfully tough to make code that's slapped together change backends from, say, flat files to an RDBMS; this is just bug fixing to make what you've got work *now*. But XP tells us not to program thinking that far down the road anyhow [extremeprogramming.org], so future scalibility is another topic altogether.)
Re:Good point -- Backend bug fixin easier today (Score:2)
Why? I think bugs today are exactly like bugs in "pre-OO days". Modular code was a good idea then, and is still a good idea now, but I fail to see why OO has anything to do with it...
Re:Good point -- Backend bug fixin easier today (Score:2)
Well, it's a great theory. :o)
Re:Good point -- Backend bug fixin easier today (Score:1)
But often:
1. The interface(s) will have to change. Bad or poorly documented interfaces are a big problem.
2. Other parts actually relied on the bad unit behavior and compensated somehow. "Fixing" it may break other things that assumed bad behavior.
Hiding behind a bl
POP3 as an example. (Score:5, Insightful)
It will be as easy as getting the US to switch to the metric system or transition with the rest of the world to driving on the left side of the road. Both would be much cheaper if they were implimented in the beginning instead of attempting a transition later.
Re:POP3 as an example. (Score:1)
I was under the impression that the side of the road on which people drove was split fairly evenly between left / right? Britain, parts of the Commonwealth, and Japan on the left, the USA and mainland Europe on the right, and India straight down the middle except when there's a cow in the way.
Re:POP3 as an example. (Score:2)
That depends.. (Score:2)
If it's a bug where something is off by a dollar per 100 or so transactions, that is hella costly. Both to find if it's not consistent, customer support and all of the other efforts to fix
Re:That depends.. (Score:2)
Always remember the 90-90 rule. (Score:2, Funny)
Windows worms are cheap (Score:2)
I can't see how it would only cost 5 times as much to get millions of users to patch their system, account for their lost time, and write the patch.
Re:Windows worms are cheap (Score:2)
Re:Windows worms are cheap (Score:2)
Since a patch goes out to (hopefully) so many systems, it isn't
Re:Windows worms are cheap (Score:2)
Depend on the bug (Score:2, Insightful)
Re:Depend on the bug (Score:2)
And yet, every place I've worked has given only lip service to design reviews, refusing to spend the time to actually analyze the designs or fix flaws pointed out...
What type of bugs? (Score:4, Insightful)
Typos: Simple misspellings of words. Infrequent, easy to detect, easy to fix.
Writos: Incoherent sentences. More frequent, hard to detect, harder to fix.
Thinkos: Conceptually bonkers. Very frequent, subtle and hard to detect; almost impossible to fix.
Most 'late' bugs that I've seen in software projects belong in the last category - a lack of design or the failure to make a working mock-up leads to 'thinkos' which are only obvious when the application is nearly completed. These are expensive to fix.
Yes (Score:2)
If define "later phase of the project" as a point further down the line on a scale of code complexity, then it's obviously true. Rooting out a bug is much easier when the codebase is smaller and simpler than when it has grown into a huge complex behemoth.
Bug vs Design Flaw (Score:2)
Where we find ourselves paying a pre
examples from my former industry (Score:2)
Here's a similar project's repair estimates (pdf). [nasa.gov] Mind you, this product cost 1000x our product, but since they were at similar customer sites, the repair cost wouldn't be significantly different.
Service trip #1 = $413 million
Service trip #2 = $497 million
Service trip #3 = $547 million
Service trip #4 = $400 million
(note: these prices don't include airfare)
In fact, it would be far cheaper to just toss out our old hardware
Yes, 100:1 is outdated (Score:2)
Re:Yes, 100:1 is outdated (Score:2)
Depends on your context. The project I am currently working on doesn't have any possibility of firmware upgradability, short of replacing the main circuit board (since the firmware is in a mask-programmed surface-mount microcontroller). So not only will a re-mask be required (expensive in itself), but also my c
Use some common sense (Score:1, Interesting)
Would you buy a car from the same company again if your current car had a lot of recalls? Is it cheaper for the car company to fix a defect before the car is made, or perform
Re:Use some common sense (Score:2)
My current car has had two recalls in the three years I've owned it, yet that wouldn't stop me buying another car from the same manufacturer. What WILL stop me by buying another car from the same manufacturer is the difficulty I have had getting them to fix relativly minor problems under warranty.
Recall problems generally affect a large number of vehicles of the same type - so think of those as "design" problems. The
Contractors (Score:4, Insightful)
How do you fix a bug cheaply when the contract has ended and all the people working on it are gone? Enter training costs for new staff.
How about needing a whole new contract just for the bugs? Enter the immoble bureaucracy.
How about a year later, when, even if someone from the project is still around, it takes them a few days just to remember what they did 14 months ago? Enter seemingly wasted time.
Anecdotal evidence is viable evidence for the undeniable fact that late bug fixes are very expensive.
It's a fundamental XP issue (Score:2)
XP usenet post. In fact, in Kent Beck's book on eXtreme Programming, he discusses
the cost of changing software in the various stages of the development process,
and a recognition that this "10-100-1000" progression is not necessarily true is
a fundamental part of the XP philosophy.
For what it's worth, I believe there are many domains where that cost escalator
does not apply. If you have a well-designed application rolled out to a manageable
userbase with access to a helpdesk o
Show stopper bugs cost more with time. (Score:2)
Here's why: when it's early in the development cycle, the team working on the product is small and intimately familiar with it. Finding and fixing the bug takes less time and costs less money in idled staff-hours affected by the bug.
As the product grows, more developers are added, additional staff (let's pretend this is a commercial effort) comes in, sales, marketing, customer support, et cetra... Now, the same show-stopper bu
It's expensive when you have to trash stock... (Score:2, Interesting)
Fix bugs early; it's less expensive that way. =)
Exponential, not logarithmic (Score:2)
It certainly depends on the bug, but w
Forest for the trees (Score:2)
A bug found in the requirements phase is not the same as someone's misplaced semi-colon.
Say you were told to develop an inventory management style. You deliver a curses-based terminal app and the customer says "wait, I was expecting a web interface!"
That is a requirements-gathering bug that will require substantial work to correct after release!
Cost is a function if ignorance (Score:1)
"Corporate Trainers" are only mouthing conventional wisdom.
FWIW, many of the original studies are for software development projects for which (1) the system is unique (e.g., flight control); (2) has never been built before; and (3) "the right people" for the job exist but are too expensive (so you have to do with folks that are less skilled.)
With these constraints, the figures of 10x, 100x, 1000x help predict life-cycle cost given a larger economic constraint (i.e., scarcity of talent). Estimation is a
Scientific Expirement (Score:2)
Any takers?
Don't think of it in "dollars and cents" terms... (Score:3, Insightful)
The REAL cost of a bug while the project is being coded is in delays to your project, which could push you past deadline. The cost of a bug after the project rolls out is the embarassment of getting caught with your pants down, and of having the inconvenience of pulling people off of other work to fix it.
So in my opinion, bugs are "cheapest" to fix during the initial design and prototype phase, where you're probably not that close to your deadline and you have some wiggle room.
They're more "expensive" to fix when you're closer to a deadline and the delay screws you up (for example, find a bug during user acceptance testing and you've got to go back and code, then start the testing all over again).
They're most "expensive" to fix when you've rolled out the project, the users come to depend on it, and something goes wrong. This embarrasses you and makes your code look untrustworthy, and forces you to scramble to deal with the problem, rolling out a patch, etc, all while dealing with hot-under-the-collar users.
I think this three-level way of looking at it is a lot more useful than any kind of imaginary mathemagical flim-flam. Forget the numbers, worry about the egg on your face.
Some real-world numbers (Score:1)
Don't get any ideas (Score:2, Insightful)
BTW, if you only read one part of my post, read the last paragraph.
For small, non critical projects the difference is indeed smaller because the complexity is much more manageable. Let's say you're building a house for your dog and your design forgot to specify which way the door should be facing. I
Plastic biro or mont blanc (Score:1)
Maybe not just the lateness? (Score:2)
There are plenty of dinky little bugs that can get unearthed in the few days before release. These aren't necessarily more expensive to find or fix.
Really bad late bugs don't happen if you get your requirements straight, and test early and often.
Fixing one may introduce others (Score:1)
Again as mentioned in "The Mythical Man Month", bugs found during the later stages of development (and more so post deployment) are more costly, because:
- The cost of regression testing
- Other (perhaps subtle) bugs may well be introduced by fixing another
The book goes on to say that, post deployment, fixing bugs (and the introduction of new ones in the process) causes the system itself to atrophy. In other words become more unstable, not more stable.Gone over a waterfall lately? (Score:2)
It's not just the cost "to fix", but also... (Score:3, Insightful)
For example, if you get a requirement wrong and spend X developer-months designing and coding a subsystem around that requirement, the cost to fix it includes that already sunk cost plus the cost of reworking the design and code to make it conform to what the spec should have said.
Or consider the case where section II.3.iv of the spec conflicts utterly with the requirements detailed in section IV.2.iii. If you don't catch that early (and assuming its a large project, given the size of the specs), you'll have two different subproject teams off designing, coding and testing to cross purposes and you'll only discover the problem at integration time.
Sure, some requirements or design bugs are trivial to fix even after coding is almost complete (you got the color of some GUI feature wrong, say). Others aren't (you missed some key requirement that radically affects the way the data should be represented and you have to change all your data structures and database tables).
plan for bugs then. (Score:1)
There will be some points in your design that will have grave consequences if there are bug in it, it is up to the designers/programmers to identify those points and plan the repairs ahead.
How? I dont know really, I'm supprise
You can work it out and prove it for yourself (Score:3, Informative)
What it boils down to is that if I do something wrong, then at the minimum, the cost of correcting the mistake is:
cost of doing the wrong thing first + cost of changing it do the right thing - cost of doing the right thing first
As the cost of doing the wrong thing + the cost of changing it is always going to be larger than the cost of doing it right, you'll always end up with a positive number.
The rest is about momentum; the earlier the mistake was made in the cycle, the more subsequent decisions were made that are also wrong.
Note, however, this has nothing to do with the cost of adding new features later. Here, you've got nothing done wrong to start with, and the cost of changing it is equal to the cost of doing it right. What you lose is the opportunity cost, which can be iffy.
Bah, the question is meaningless (Score:3, Interesting)
With such huge range of differing costs for finding the bug before or after the shipping of your product, the "average cost" of bugs is meaningless.
I think that the only thing to remember is:
- bugs found late cost more to fix than bugs found earlier (any specific number is invalid)
- finding bugs early is difficult and can be expensive.
Of which you can deduce that:
- if late bugs can cost you very much (Ariane5 for exemple), you want to spend a lot of money on software testing|review at each level.
- otherwise if tests can cost more than the fix (a small number of internal users with a non-critical software), then maybe you can use the clients as testers, but it must be managed well (tell the users, be in close contact with the users, don't let them wait the fixes too much).
True, but... (Score:2)
In the above example, you're talking about a bug that is an error in getting the requirements down correctly and letting it live all the way out into the field. Such a bug would indeed be quite costly to repair! Most likely it would require a new version of the product. This type of bug might be that the prod