The Economics of Perfect Software 201
An anonymous reader writes "This article takes the interesting perspective that leaving bugs in software is good — little ones, at least. This quote is particularly insightful: 'How do you know whether a bug is big or little? Think about who's going to hit it, and how mad they'll be when they do. If a user who goes through three levels of menus, opens an advanced configuration window, checks three checkboxes, and hits the 'A' key gets a weird error message for his trouble, that's a little bug. It's buried deep, and when the user hits it, he says 'huh,' clicks a button, and then goes on his merry way. If your program crashes on launch for a common setup, though, that's a big bug. Lots of people will hit it, and they will all be pissed. ... The cost of fixing all the bugs in your program and then being sure you fixed them all is way too high compared to the cost of having a few users hit some bugs they won't care about."
How is this news? (Score:5, Insightful)
Lets look at it another way (Score:3, Insightful)
You cant afford perfect software.
Oh Please (Score:5, Insightful)
* It's created by people
* The programmers, testers, etc can never find all the bugs
* It's gotta get out the door so you can pay your programmers, testers, etc
* All of the above
There is a matter of pride with individual programmers, small groups, and most open source programming projects. We'd all love to be able to crow about shipping a bug free project/product. We'd do it if we could
Well, sure, if you have precognition (Score:5, Insightful)
Well, sure, except that this assumes that you are PERFECT in your ability to predict the effect of a bug. And if you're not, that bug that you think will only happen in some situation that's vastly improbably will, in fact, actually hit at exactly the WORST possible time, because maybe that key sequence gets used in some extremely important operation that you hadn't realized your software was going to be used for. Or maybe that bug is ALSO triggered by some different sequence that you weren't quite prescient enough to realize would be common.
probably good idea; definitely bad example (Score:4, Insightful)
When I'm trying very hard to make a program do what I want it to, the more hoops I have to jump through for every iteration of trying to make it work, the madder I get. So, the fact that the software has dark corners that you can get to like this is already a major strike against it...
Dangerous approach (Score:5, Insightful)
This sounds a lot like the "Good Enough Software" policies that certain companies used to have. It's true that having a few minor bugs is acceptable; in fact, it's pretty much inevitable. But you should expect to have bugs that you haven't even noticed at the time you ship your software. If you start ignoring the bugs you do know about, you increase the total number considerably. From there, it's a slippery slope. Pretty soon, you'll be answering bug reports with a chorus of "Minor, leave it," and your code will be riddled with tons of minor bugs. Your users will go from "huh" to "what?" to "dammit" to "f*ck this!" Your software will look sloppy and unprofessional, and even if none of the bugs are showstoppers, your customers will be looking for alternatives.
Re:Oh Please (Score:5, Insightful)
Exactly. Most coders have to use the code they write. Leaving bugs is not something you do if you know they are there. Sure some people do that, but not the ones that want to get it right - at least for their own use. This is why some open source software is not infinitely usable - it did what the original author needed, and was not intended particularly to solve everyone's silly problems. The example of the 'A' key is probably something that was going to work soon, but the need for it never materialized and someone forgot to disable the code that checks for the letter 'A' - and since it was not implemented, nobody checked it.
The first time I had code in production, running 24/7 I was kind of upset that I didn't need to look at it everyday, then amazed that it was working so well. Then, months later when I did have to go look at why it seemed to be not working right, I learned how important documentation and comments are. At this point I stopped being amazed that it worked well, but that it worked at all. Bugs are not left on purpose, only when there is not time or resources to fix them.
Maybe. (Score:4, Insightful)
I think it also depends on how much it costs to remove the bugs. Often times, the small bugs that people are more likely to accept or tolerate are also very easy to fix. When the cost of fixing the bug is very low, it should be a no-brainer to do it. If on the other hand the bug is expensive to fix -- requiring a complete redesign and re-coding, then if the bug is not severe it may well be better to leave it in.
I'm not sure I buy the argument (Score:3, Insightful)
Software is not so different from other engineering disciplines that we cannot learn from the best practices of adjacent fields. Although no human endeavor -- other than being Pope -- is infallible, we do see disciplined attempts to measure, manage and improve quality levels. The point is not to reduce the quality level to zero, but to know what your quality level is and be able to hit your mark. What level of quality is needed will depend on the market, your customers, the competition, the frequency of releases, whether the software is easily updated or burned into firmware and devices, etc.
In the end, the best way to create higher quality software is not to find and fix more bugs. It is to use a disciplined approach to programming that introduces fewer bugs in the first place. If you focus on defect detection and removal, then you are fighting the wrong battle. You can't win doing it that way. Quality software comes from a disciplined process at all stages, from requirements to testing, and not just as an endgame activity to sway as many bugs as you can.
If they need it, and it's buggy, it's annoying (Score:4, Insightful)
Is going to be one extremely unhappy user because they had a reason for going that deep into the sub-cellars of the software to get at that obscure feature ... and it is likely that they are the sport of user who is a bit more tech-savvy and ready to bitch than Joe Average user.
Re:Oh Please (Score:5, Insightful)
even as individuals with all of our time at our discretion we would not waste the rest of lives trying to get the bugs out of something as complex as modern software, so NO we won't do it SOMEDAY either.
We may not ever accomplish it, but that doesn't mean we won't stop trying. Bugs slip through no matter how much we test. One reason is because no matter how we may try to break it we can never imagine the utterly creative stupidity that a user will cast upon our software.
And as far as Skynet is concerned, it will be written by humans or by a system written by humans, so it's gonna have bugs, too (unless it was written by Deep Thought, but that's another story entirely).
sometimes crash == possible exploit (Score:5, Insightful)
It isn't always easy to judge the severity of bugs. Exploits often grow from a black hat figuring out how to crash a program. Some of the greatest exploits started with the smallest of footholds.
Re:How is this news? (Score:1, Insightful)
I leave in all kinds of weird bugs that only I can fix, just to guarantee my future job. I'm betting big software companies relying on people buying the new versions of their products do the same. After all, there's no reason for upgrading perfect software, is there?
Re:Oh Please (Score:1, Insightful)
Re:If they need it, and it's buggy, it's annoying (Score:3, Insightful)
How about we pretend that the user actually wants to use the feature to avoid analogies along the lines of "It's like saying a car is acceptably broken if the engine explodes when you turn on the radio since we can assume the driver had no intention of actually using the radio, that's just an obscure extra feature that no one uses..."...
Re:Oh Please (Score:5, Insightful)
Leaving bugs is not something you do if you know they are there.
Depends on how much of the code you have to refactor to get rid of it, how many users are annoyed at that particular bug in the previous release, and how many new users you'd get by implementing the most asked-for feature instead.
You probably won't fix the bug where your text editor crashes if the user tries to open a file larger than 32 Gb first, if they've been also nagging you for global replace functionality.
mediocrity (Score:5, Insightful)
s/nation/project/
Re:Oh Please (Score:3, Insightful)
Most coders have to use the code they write. Leaving bugs is not something you do if you know they are there. Sure some people do that, but not the ones that want to get it right
This happens all the time. Have you never read release notes that have a list of "known bugs"? Generally that's open source software. The closed source software has the exact same list, it just exists on the software makers private network.
Sure, if you KNOW how to fix the bug and it's easy most people will just fix it. The other cases it depends on the economics. Open source software isn't any different than closed source in this respect. Resources are always finite whether they're money or time.
No bugs isn't a perfect product. (Score:5, Insightful)
Even if it's bug free, that doesn't mean the software is designed to be easy or intuitive to use. There is often more money to be made from deliberate obfuscation. Every piece of Enterprise software I've used (particularly SAP) seems to follow this principle. And the huge aftermarket of expert consultants charging $150/hr for training and troubleshooting, seems to affirm it. Keep in mind those consultants are usually teaching the corporate experts, I'm not even touching the surface of end user training, which I'm experienced at being responsible for.
The real money comes from support packages and direct-line support levels (ie: silver, gold, platinum and such to maintain a 24h call centre). Let's ask a simple hypothetical: what if Windows was bulletproof and bug free? How would Microsoft make any money? If nobody had reason to fix a problem within 30 minutes like their job depends on it? Or even not upgrade to the next version because the last one works better? With XP and Vista we already know the answer to that.
Re:How is this news? (Score:4, Insightful)
The first problem is that it doesn't guarantee anything. There are very few software companies that close down or get rid of people because of small quirky bugs. You're not going to lose your job simply because you write perfect code either...
The second problem is that if you do keep your job, it's only to maintain the same piece of code. Aren't there more exciting things to do than to fix the same bugs you left in some spaghetti code that only you can understand? Isn't it better to write good solid readable code and move on to more interesting problems than off-by-one bugs you left on purpose?
And the third problem is that you can never justify that to your boss: "You've got to keep me because the code is terrible and only I can maintain it". The easiest thing would be to cut the losses and send you free, only to bring someone who can write some code that isn't an expensive liability.
Sadly, you're not the only person that writes bad code for "job security". So I hope your post gets a "Score: 5 Funny" and not a "Score: 2 Insightful"...
Re:How is this news? (Score:5, Insightful)
What if that little UI glitch gives remote root?
Then you're an idiot who didn't separate your software properly into layers and should learn, or get out.
I understand your example, but the ultimate fear you're expressing about "the unknown" isn't warranted. Software is complex and can lead to unexpected problems. But worrying about every single minor bug possibly being a major one is just silly if you understand how the system interacts with itself. That's why good software is developed in layers. If you've really got a system where a UI glitch leads to a root exploit, time to throw away the whole application and start over.
Re:Oh Please (Score:3, Insightful)
I doubt that that will happen. Anything made by a human has a flaw.
Yeah, but it's one of those infinite-number-of-monkeys-for-an-infinite-amount-of-time things. Enough programmers writing enough code for enough time will, more or less by accident, write a perfect piece of code.
Re:You sure about that? (Score:2, Insightful)
Programs run just fine, or so some hope, with compiler warnings. Despite that, parent says his shop considers them to be bugs.
Oh to work in such a shop, where I don't have to sift through 74 warnings every build because someone decides that catch (Exception e) is fine even if you never take so much as a passing glance at what got put into variable e. Hey, dev manager, how about you just turn the fsckin' things off if no one cares?
Re:Oh Please (Score:1, Insightful)
3(-2x + 1) = -6x + 3
Re:Oh Please (Score:3, Insightful)
Re:Oh Please (Score:3, Insightful)
That's one factor; but another larger factor is that there are often several orders of magnitude more users of your software than there are testers. Even if the testers were able to come up with all the "utterly creative stupidity" as the users, there wouldn't be enough hours in the day for them to test all those cases.