We Really Don't Know Jack About Maintenance 260
davecb writes "The ACM has been kind enough to print Paul Stachour's and my 'jack' article about Software Maintenance. Paul first pointed out back in 1984 that we and our managers were being foolish — when we were still running Unix V7 — and if anything it's been getting worse. Turns out maintenance has been a 'solved problem in computer science' since at least then, and we're just beginning to rediscover it."
Grrr (Score:2, Insightful)
"Software maintenance" has absolutely nothing to do with computer science. I wish people would stop calling business programming computer science. Computer science work gets done at universities and research institutions, not at Initech.
Re:Grrr (Score:4, Informative)
How do you design software that is able to be maintained? Many of the techniques for software maintenance are designed by these institutions, so saying "Software Maintenance" is not computer science is a bit far fetched. Writing maintainable software isn't only in the Initech domain.
And really if you are excluding software maintenance from the field of computer science, you pretty much have to exclude every other software technique. Techniques for writing maintainable code go hand in hand with every other development technique.
Re: (Score:2)
Development methodologies have nothing to do with computer science either.
Re: (Score:2)
Sorry for the ignorance, but is that true in university courses as well?
I have a degree in mathematics (with a few CS courses), and a technical diploma in Info. Systems. My IS courses had a very strong emphasis on methodologies and writing maintainable code. I've always assumed a CS degree would have the same, along with the CS theory.
If not, that explains some of the code I've seen written by some very bright people. Some of it (n
Re: (Score:2, Informative)
As a mathematician, I would have to disagree... computer program architecture is clearly in the domain of computer science, and properties of program architectures can be modeled algebraically. Indeed, this is why modern academic researchers work with data types that are initial algebras.
Moreover, every computer program and function is a monad (or, equivalently, comonad). People make a living from ignoring that trivial fact and re-inventing ways to make monadic structure apparent, via OO programming.
Re: (Score:2)
Programming language theory is computer science. I'll even concede that programming with a FP language is very computer science-y. Techniques for writing maintainable code is not.
Re: (Score:2)
You are absolutely right about maintainability. Code is written for two main purposes:
1. For a machine to execute or interpret
2. For a person to read
The programmer's intent should be plainly clear in every piece of code. Preferably without the need for comment. In cases where the code doesn't make its intent clear, a comment can be helpful - but preferably the code should be clear without it.
That is, without a doubt, more art than science.
I am disappointed, sometimes, by the lack of basic computer scienc
Re: (Score:3, Insightful)
Programming language theory is computer science. I'll even concede that programming with a FP language is very computer science-y. Techniques for writing maintainable code is not.
Many activities carried out every day for performing software engineering is not computer science. But good software engineering, the discipline itself, that is deeply root in computer science. Just like coding. Coding is not computer science, but good coding practices are rooted computer science.
Principles and notions such as cohesion, coupling, modularity, comprehensiveness, patterns, complexity management, architecture, they all originate in computer science.
I think you are right that software (and s
That's mighty elitist of you (Score:4, Insightful)
"Software maintenance" has absolutely nothing to do with computer science.
Actually, it does.
I have a real Computer Science degree, so I know what computer science is about. And while a lot of corporate programming is more drudgery and form assembly than anything, there are a lot of applications of computer science in the real world - from scalability issues in large systems, to proper use of encryption.
Furthermore the supposedly boring area of "Software maintenance" has a ton of research potential focused around the optimal path to producing correct code. Do code review help? How to team dynamics factor in to code quality? Does Test First really improve code? What even defines code quality? There are programs within companies that experiment with different methods to improve code output, and those experiments are even more valid than ones performed at research institutions since they work on a real-world code base solving a real problem using real people. In fact I would go so far as to say and research being conducted outside of a company on aspect of code quality improvement is pretty much worthless, which is why it's important for researchers to partner with real companies for some studies.
Re: (Score:3, Informative)
I have a real Computer Science degree, so I know what computer science is about...
Except what you go on to describe is software engineering, not computer science. I'm not picking gnats either; the distinction is very real, and I would hope it wouldn't be lost on someone with a "real Computer Science degree."
Re: (Score:2, Insightful)
You're using the new definition of CS not it's traditional meaning.
The combination of academic elitism, a societal trend toward specialization, and a CS tendency to sub-divide complex problems have combined to split the traditional, interdisciplinary Computer Science into multiple sub-fields in the last decade.
Traditionally we had Computer Engineers to design and build hardware and Computer Scientists to design and build software to make use of those devices. (The business world added Computer Information S
Re: (Score:2)
You mean, as science advances fields will get more and more specialized and specialization won't stop?
Re: (Score:2)
I concur that the definitions of Computer Science and Computer Engineering are changing, but what bothers me is the lack of understanding of many foundations of computer science to be found in CS graduates.
I've been interviewing a lot over the past couple of years. I mostly look for people who demonstrate good problem solving, intelligence, some knowledge of the subject, and the ability to enter into technical discussions.
But it's disheartening when many graduates haven't done algorithms, data structures,
Re: (Score:2)
Re: (Score:3)
Except what you go on to describe is software engineering, not computer science.
True the latter part of my post is less pure science. But that does not mean it is not valuable too, which is why I went on to mention it even though somewhat tangental, and is as worth of research as more "pure" computer science issues would be.
Re: (Score:2)
A Turing machine cannot solve the problem of software maintenance. You cannot model software maintenance as a finite state machine. There is no algorithmic solution. There is no space-time trade-off that you can make improve the situation.
It is not a problem to be solved by computing. It is outside the realm of Computer Science, and clearly in the lap of Software Engineering.
Re: (Score:2)
A Turing machine cannot solve the problem of software maintenance.
That is true but does not mean it's not a valid sub-field with equal importance. The same kinds of issues exist in "real" engineering disciplines where a plan must go from conception to reality involving many different people.
Re: (Score:2)
I completely agree with you, it is a very real engineering problem that requires serious academic examination. This is a good candidate for something like that Software Engineering Institute [cmu.edu]. The issue is one of writing software and managing a code base. It is a logistics problem. Maybe it can only be solved by a new language or method of source control or verification. In that area, yes, it's open to computer science to explore. But in general, it's a code architecture, testing, development methodolo
Re: (Score:2)
Computer Science is not limited to things that can execute inside a turing machine. A large body of computer science is made up of correctness proofs - which, strangely, weren't written by programs running on a universal turing machine, for one example...
What is and what is not in the realm of Computer Science is not defined by what can run inside a turning machine.
I also doubt that the turing machine will be the end of Computer Science.
Re: (Score:2, Offtopic)
I beg to differ, in my computer science classes at the University of Missouri Rolla, they taught us software maintenance to go along with our programming classes. It is for more than just business software or application software it can be used for system software as well as operating systems, firmware, database applications, and everything else except the kitchen sink (but we are working on that one, he heh heh:).
If you don't do maintenance for your software, it is a lot like not doing maintenance for your
Re: (Score:2)
You got it dude.
The problem is management is flat out fucking stupid
and only worried about their stock options.
Fucking stock options have screwed everything up for
over 20 years now.
The dumb ass pointy hairs can not recognize a
good programmer because they are push-button.
Worthless.
Proper software development is properly done correctly
up front, with *PLANS* for maintenance.
Re: (Score:2)
Warning: This message may contain cynicism that some people may find offensive. Viewer discretion is advised.
But they were so blind to see that the work I did while it was an expense, it saved then millions of dollars in lost productivity, lost CPU time, cost savings in not needing to buy faster processors and more memory as my code ran tight and compact in lower ones, and days if not weeks of downtime and a dysfunctional system and software that takes them years to fix by hiring high priced contractors to do the work I would have done for my small salary.
It seems like it takes an inordinate amount of effort to sell doing maintenance/cleanup to management types, and I don't know enough about psychology or the "business" mindset to understand why that is. Let us know if you figure out the secret to being a high-priced contractor while still having a conscience and some integrity. I suspect that the most essential component of success as a high-pai
Re: (Score:2)
Don't kid yourself, genius.
Efficient software maintenance is directly related to the Halting Problem. I'm sure you could figure out a thesis about it. Or at least a proof.
Besides, "business programming" involves many things, some of which includes problems (for example, my field, aerospace) that are ridiculously complex, certainly at least as much as a good 80% of any CS degree or thesis.
Re: (Score:2)
It first categorizes different ways of maintaining software, with no real rationale for choosing those categories, then goes on to describe a versioning system that somehow m
Re:Grrr (Score:4, Insightful)
""Software maintenance" has absolutely nothing to do with computer science. "
And that attitude is EXACTLY why computing still sucks.
If it involves computers, and it's an interesting unsolved problem - and guess what, a LOT of real-world problems turn out to be extremely interesting and extremely unsolved, full of all sorts of corner cases and exceptions to previously supposed 'laws' - then it's computer science.
System administration is like artificial intelligence. It's a human-centric job involving a lot of common sense, and it can't always be readily automated because the rules keep changing. That ought to raise warning flags. .Like vision or language recognition, a lot of scientists seem to think administration is so easy that only dumb people bother to investigate it. So they don't. But that's the opposite of the truth. Humans are still stuck doing rote maintenance precisely BECAUSE it's so tricky to do that computers can't yet do it. Which means it's very, very interesting.
A sensible approach to computer science would treat all phases of the development and deployment of information, computing and communication systems as computing systems in themselves. Yes, that involves crossing disciplinary boundaries. Perhaps that's more cybernetics or systems thinking or linguistics or sociology or psychology than computer science as it's currently narrowly defined itself to be. But in that case, computer science has defined itself out of the interesting part of the game - the impact and use of computing in the real world and what parts of this can and can't be predicted and automated - which would be a very sad thing.
Re: (Score:3, Insightful)
""Software maintenance" has absolutely nothing to do with computer science. "
And that attitude is EXACTLY why computing still sucks.
If it involves computers, and it's an interesting unsolved problem - and guess what, a LOT of real-world problems turn out to be extremely interesting and extremely unsolved, full of all sorts of corner cases and exceptions to previously supposed 'laws' - then it's computer science.
No true. Software Engineering solves a lot of unsolved problems without having to do computer science. Computer Science is about theory, not implementation. Computer Science deals with the Universal Turing Machine and treats all computers like it. Computer Science programs tend to not discuss how to do things with efficiency or with limited resources.
Computer Engineering programs tend to talk a little about Computer Programming; but mostly focus on the engineering behind computer hardware (e.g. PCI board
Re: (Score:2)
Computer science work gets done at universities and research institutions, not at Initech.
That's because computer science isn't good for the company.
Re: (Score:2)
"Everyone knows maintenance is boring" (Score:5, Insightful)
Taking code and cutting its size by half, fixing up all the screwed-up inconsistent formatting, while adding functionality and reducing bug counts, is a pleasure.
It's all in the mindset. It's only boring if you limit yourself to the boring parts.
Re:"Everyone knows maintenance is boring" (Score:5, Insightful)
What company do you work for where it is even remotely possible to accomplish all that and still meet your deadlines? In my world, you always wish you could totally refactor the code like that, but there's never enough time in the schedule for that. So instead, you "make it work" as best as you can, and that's about it. Maybe, if you're lucky, you can find enough time to do at least a partial re-write of the worst parts of the code, or tidy up an interface or two. But cutting the code size by half? Yeah right. I wish.
Re:"Everyone knows maintenance is boring" (Score:5, Insightful)
You don't tell them you're doing that ... you just do it. It's quicker anyway. Start by cleaning up the formatting and moving stuff into headers, some righteous macros to make the code more readable, etc., as a first step in "getting familiar with the code. After all, you don't want to break something while fixing a problem. You'll probably notice some code that looks like it's either "copy-n-paste" or dead. Keep it in mind.
You'll find potential for simplifications as you go along, and you'll also probably find other bugs and stupidities, like parameters that are never used (that happens a lot as code gets modded). Code that doesn't smell right. Signed ints that are unsigned elsewhere. Potential buffer overruns. Unsanitized unput. In-line code that really should be in a separate function call both for clarity and for re-use. Look through the bug list - if the bug (or potential bug) you found is there, great - mark it as fixed. If not, log it, then mark it as fixed. Either way, your fixed bug count goes up - and if you can show that you've fixed bugs before they're found, that's just a bonus.
Now start a bit more higher-level fixing. Nothing too wild, but if you have to add a bit of functionality, use that as an excuse to simplify the code that it touches. You'll probably confirm some of the dead code from the first step. Remove it. You'll probably confirm cases of multiple instances of copy-and-paste - refactor them as well into something a bit more same. Now you've removed a few more bugs, shrunk the file size, and added some functionality.
Now that you've done that, review the code again, and look for more instances of copy-and-paste. Code that was written in a hurry will have LOTS of it, so you'll definitely be able to give it some more w8loss treatments. And since you didn't bitch and moan TOO much about making the mods, and they actually work, they'll probably be happy to let you graze on the bug-fix list. There's always bugs, nobody wants to touch them, and as you clean up the code, you'll see it start to shrink.
A 50% reduction on crappy spaghetti code is reasonable under those circumstances.
At that point, you own the code. You're in the loop for changes that might break it. When you say, "okay, but it's going to take x time" they're more likely to actually listen. Consider it office politics for nerds.
Re:"Everyone knows maintenance is boring" (Score:5, Insightful)
That's great, until your huge refactoring introduces a new bug. And it will, you can't expect to refactor a decent sized body of code without breaking something. When it does, it doesn't matter that the code is 100 times more readable and extensible, or that you fixed ten other bugs while doing it, your ears will get hot. Because you broke something which means more work for someone else and that the count of outstanding bugs was just incremented.
This happens in the places where each bug report has to be annotated with about two dozen different abbrevations and numeric codes so that the correct departments and bug examiners will get to look at it. When it takes two hours to file bugs and each bug has to be examined by four persons, you can forget about filing bugs about "potential problems." Bug reports are for the issues customers have complained about and nothing else.
What about the next bug report that comes on? If it can even remotely be associated with the new code you wrote, then it will be. Because yours is the only new code in what is otherwise a totally legacy code base with hundreds of band aids. Expect to take the blame.
And then you have the original author of the body of code you just killed/refactored. He wont be so amused about you rewriting in one week what he spent four months writing. Expect to hear nitpicks and complaints from his side. You won't be able to ask him anything about the code anymore because "you destroyed it."
So yes, while refactoring code makes it better, you wont make many friends while doing it and you are taking a huge personal risk. Especially if you are doing it secretly.
Re:"Everyone knows maintenance is boring" (Score:4, Interesting)
This is why you refactor the code alongside the original. Fix bugs in parallel. When there's a new bug, you roll back to the old, non-refactored code and see if it goes away. If so, you stay on the old, crappy code until you can fix the new, cleaned up code. Then you swap back to the new code. You repeat this testing phase until switching on the new code doesn't introduce any critical regressions, then continue living on the new code for a few months before nuking the old code.
Yes, this is slower than doing development on a single version of the code, but it's a short-term penalty until the code is cleaner and more flexible, at which point you then make up for the period of slowness in the minds of your bosses by proceeding to successfully add twelve new features in a single week---you know, the ones that have been punted for six years because they were too risky to make in the current fragile code base.
The trick is to alternate between rewrites and enhancements. Don't stay focused on rewrites too long or you'll seem unproductive. Don't ignore it for too long or you'll find yourself blocked and eventually become unproductive. The only way to survive is to stay somewhere in the middle.
Re: (Score:3, Insightful)
That's great, until your huge refactoring introduces a new bug.
You can always make sure you've got some legitimate reason to have done the refactoring/redesign. I got away with rewriting a section of code that the owner of the company had cordoned off with comments like "DO NOT TOUCH, YOU CAN'T MAKE THIS ANY FASTER" because I made it run about 10 times faster. (Honestly, how can you ignore that kind of comment if you like programming?)
I checked in a bug fix (with unrelated speed improvements included) that introduced another bug, but also dropped application CPU usage
Re: (Score:3, Interesting)
"... some righteous macros to make the code more readable..."
OMG, somebody fire this jerk. In most cases adding macros MAY make the code more readable for you, but the next poor guy who comes along can no longer look at that section of code to see what's going on, but must now combine the code he's reading with a macro that he can only assume was implemented properly in the first place.
And if by "righteous" you mean some complex and convoluted code boiled down into a single line... then you should not only
Re: (Score:2)
It only works if the project is small enough that a sole person or a small team (max 3 persons) can take ownership of the whole code base.
If you do the cleanup you suggested on just part of the code, you're only introducing "yet another style of coding" to the code base: congratulations, you've just increased the complexity of the code base and made it less stable.
I've seen this kind of code where multiple people at different times have tried to "enforce their vision" in the code they controlled: without ex
Re:"Everyone knows maintenance is boring" (Score:4, Insightful)
Then you have a process and management problem, not a coding problem.
One person should have the authority to approve or disapprove.
It's like when you make a complaint, and they say "I'm sorry, I can't do that." The proper response is "Then why am I wasting both of our time talking to you? Please pass me to someone who can."
Strictly a management problem.
Re:"Everyone knows maintenance is boring" (Score:4, Insightful)
Re: (Score:2)
I don't know what the GP had in mind, but maybe it's a significant bug (e.g., showstopper) on a production system?
Re: (Score:2)
That is not maintenance. (Score:5, Insightful)
Taking code and cutting its size by half, fixing up all the screwed-up inconsistent formatting, while adding functionality and reducing bug counts, is a pleasure.
Yes it is.
But that is not maintenance, as practiced by any rational company. That is development or (more specifically) optimization.
Maintenance is about solving a problem code is having, with the absolute smallest number of changes possible. Even new features can fall under this heading when software is in true "maintenance mode" in order to avoid a lot of excess testing.
I actually don't mind it, as it is a different sort of challenge to take a code base you know little about and introduce a working change with as little code as possible. But it's not as glamorous as fresh, raw coding.
Re:That is not maintenance. (Score:4, Insightful)
I believe that no developer can ever become a good or great developer without having done significant maintenance on someone else's code - your really only learn the true value of "coding for maintainability" (i.e. basics like commenting complex code, avoiding copy and paste, fail-fast code and such) when you're landed with fixing/improving code that was done without any such concerns.
That said, software maintenance is often a frustrating, painful process in anything but the freshest of code (and sometimes even is code "just out of the over").
It's especially unpleasant when one is faced with fixing/changing software where some or all of the design or the code were done by (often long gone) less experienced software developers.
The bad news is that as you become more experienced you see more and more "glaring problems" with the design/coding on any system you have to fix/improve, which can be infuriating at times (for example, when the design and development of an important sub-system was given to a team composed wholly of average/below-average developers with 4 years of less of experience and you are tasked with fixing it)
The good news is that the outsourcing wave has caused a bubble in IT in India, resulting in loads of people joining IT for the money (most of which should never have done so) so there is a whole lot of really, really bad code out there which will be providing freelancers (like me) with highly-paid maintenance work for years to come.
Re: (Score:2)
Re:"Everyone knows maintenance is boring" (Score:5, Insightful)
More likely, the code was written and documented badly in the first place, so it's not a matter of taking it and cutting it down into something more elegant. Instead, it's about trying to decipher exactly what's going on, then trying to figure out how to make it at least superficially better while trying to keep within the same crappy framework. This has to all be done within budget, or the work done will go to waste when the project's scrapped, and there won't be a budget for maintenance again.
Maintenance isn't a rewrite from scratch to do the same thing, as much as we'd like it to be. Production systems have to be kept up. Migration to new code has to be both backwards compatible and produce enough visible results to make it worthwhile. It can't be too big of a change, nor too little. It's hardly any wonder software developers don't get to, and managers don't want to do it.
Re: (Score:2)
You're there anyway - why not clean it up a bit. Every file you touch, clean it up (because people in a hurry always leave a mess). It'll help decipher what's going on, so you're less likely to fubar something.
Do that often enough, and the code becomes more workable, and you'll begin to see duplication that can be remedied with no impact, as well as find a few errors. Check the bug list - if you've found the source of a bug, tell the powers that be that, while doing the mods, you found the source to bug
Re:"Everyone knows maintenance is boring" (Score:4, Insightful)
But if you have 20 or 30 branches for deployed systems and your fix needs to be imported into most of them then your clean up is viewed as a total PITA because now it is not clear why you made all those changes.
Re: (Score:2)
"Maintenance isn't a rewrite from scratch to do the same thing, as much as we'd like it to be."
Meaning that you can't figure out how the previous developer did what he did, and if you rewrite it from scratch at least YOU will understand it.
Unfortunately, that doesn't help the next developer, who's now totally unable to read and parse your brilliant semi-logical code...
And wants to rewrite IT from scratch.
Re: (Score:2)
Well, I once realized, that all code basically is "throwaway code". Meaning, that you only can do proper evolution of a software project, if you do a complete rewrite from time to time.
That is, because the basic usage and purpose of the program changes with time. New, better paradigms get discovered, etc. And they don't fit with the old basic architecture.
If you just endlessly patch them on top, you get something like Windows ME, Internet Explorer, or MS Office.
So you make clear cuts: You stay with the old
Not only PHB (Score:3, Interesting)
Re: (Score:3, Insightful)
Taking code and cutting its size by half, fixing up all the screwed-up inconsistent formatting, while adding functionality and reducing bug counts, is a pleasure.
Clearly you've worked only on code that needs a bit of tidy up. Try reading code that hasn't been documented well if at all, and looks like it's been run through a code obfuscator (ie not just poor formatting), then realising that not only is the task the code is suppose to accomplish not documented, but the people who wrote the original code left
Different maintenance.. (Score:2)
He is talking about the need to version protocols to simplify future change, you're talking about code refactoring which usually don't change the interface..
I've needed worked on a project where we used 'versionned protocols' though :-(
Move the functions to Business (Score:3, Insightful)
No, I'm serious. It's the latest Trend (tm).
Cut the IT production support budget, cut the IT staff, move the functions that used to be IT Prod support to business and let their budget handle it.
If you can't see the humour in seeing a business person make direct live updates to a database table in Production (IT doesn't have Production access.. but Business does .. [go figure]) then you probably can't see that SEV1 sailing in from over the horizon to make your day just that little bit more special.
As for business editing files directly in Production because the cost of having IT do it (process - backup file, edit file, copy file to prod with due authorisation, verify the change) can just be avoided.
After all, we don't really need to pay for Production Support and System Maintenance and Documentation. The system works without these things, doesn't it?
What could possibly go wrong?
Re: (Score:2)
Are you looking for a consultant? I think I can free a few ... years?
Maybe I'm just young and foolish... (Score:2, Funny)
Important forgotten steps (Score:5, Insightful)
Re: (Score:3, Insightful)
See, I've always looked at meetings primarily as a way for project managers who have no technical work to do on the project to feel like they're doing something.
In fact, this purpose explains the following relationship:
Number of meetings per week = Stress level of project manager / Time remaining before project deadline
So early on, you can expect a kickoff and a couple of weekly status meetings. As the deadline begins to loom, suddenly you're looking at 2-3 meetings per day per developer, with many meetings
Re: (Score:2)
As the deadline begins to loom, suddenly you're looking at 2-3 meetings per day per developer, with many meetings getting called at the last minute.
Wouldn't that hurt the problem by taking developers off getting the project done to talk about what needs to be done?
Re: (Score:2)
You must still be in college. Your reason and sense isn't welcome out here.
Re: (Score:2)
Re:Important forgotten steps (Score:4, Insightful)
Dipshit PHBs would be buying them left and right!
Re: (Score:2)
And nobody to take charge and responsibility for it all.
A project done well needs a strong leader who can manage both the needs of the users, the coders, and upper management. Only then can things happen. Without somebody like that, all you'll get out of it is an office tug-of-war between all of the interests. It's why leadership and direction are such important qualities in management, and why companies try to hire and keep the best.
In fact, it's why the CEO having a vision of where the company's going, an
Good luck (Score:2)
A lot of admins are pretty wary of throwing the latest and greatest on their boxes for the simple reason that it may, or shall I say, will break things. Its no fun to throw a service pack on to find it has nuked your installation, or upgrading your mail server to find out that your configuration isn't global to the new version. Now that sort of thing happens. I'm just saying there are all kinds of little issues (or huge ones) that can arrive. At least with older software you already know the faults and are
Numeric function calls (Score:3, Funny)
FreeBSD 1 and FreeBSD 2 had slightly different semantics for some system calls, but FreeBSD 2 changed the system call numbers, so it was possible to modify the FreeBSD 1.1.5.1 kernel to run the binary Netscape for FreeBSD 2.x by implementing the new API for one call in the old kernel. Alas, I can't find the patch now, which is embarrassing because I was the one hosting it... about 15 years ago.
six sigma (Score:5, Funny)
Re: (Score:2)
Whenever someone fixes a bug that creates a new bug, then it's a waste of everyone time and effort.
Bah, you're doing it wrong. We used to say in my old work that every bug fix created TWO new bugs :) . That also ensured job stability :P .
Different Kinds of Companies (Score:5, Interesting)
Decades ago, companies which developed technology were...technology companies. With real engineers, and highly technically skilled management. Today, companies with business-oriented management and zero technology background own and develop systems. They often do it poorly, with insufficiently empowered engineering teams, and insufficiently skilled engineers.
So today we've got a lot of Java and .Net shops filled with junior-level programmers and no disciplined, experienced systems engineers. Is it a surprise that when MS brought programming to the masses that the masses failed to learn engineering?
Re:Different Kinds of Companies (Score:5, Interesting)
When I worked at Analogic Corporation [www.analogic.com] in the mid-eighties, the then owner/president/ceo Bernie Gordon refused to grant anyone working in software the title "Engineer". We were all "just" programmers. Until we could show him specs and tolerances and statistical failure rates of our designs we were nothing but a bunch of untrustworthy programmers. And you know what? He was right: we still are.
Re: (Score:2)
When I worked at Analogic Corporation [www.analogic.com] in the mid-eighties, the then owner/president/ceo Bernie Gordon refused to grant anyone working in software the title "Engineer".
In Uruguay (my country), the title "Engineer" is regulated (just like Lawyer, MD, etc..). You don't get a title of "Engineer" unless you have actually studied Engineering.
And to get a degree in Software Engineering you DO have to study engineering - including ALL the math, statistics, physics and chemistry that other branches of engineering have, which means only 0.0333% of each generation graduates, and I'm not kidding - I started that career with other 1500 students, only 50 of which graduated as Engine
Re: (Score:2)
No, but whether MS started that earlier. It would not have mattered a bit whether MS brought programming to the masses. Someone would have. Actually, Borland was way ahead of MS with C-Builder and Delphi when it comes to easily accessible RAD tools.
The core problem is that the need for more software was there, when the amount of good programmers remained more or less stable, or at the very least didn't explode at the same rate as the demand. Think back 25 years, to the time before RAD tools, and notice how
Re: (Score:2)
Re: (Score:2)
Re: (Score:3, Interesting)
Delete the word 'software' from TFA title (Score:2)
and the sentiment is no less true.
My observation is that we shaven monkeys that make up the human race are fundamentally incapable of maintenance in any sense. Rather than maintain something in as-new functional condition (maintenance) so it does not fail, we choose to either fix it (fixenance) or replace it (buyenance) when it does.
A factory that makes plastic widgets is just as likely to make these same mistakes in relation to their machinery.
Just my $0.02
err!
jak.
Re: (Score:2)
The word "software" isn't in the title...
we're just beginning to rediscover it." (Score:3, Interesting)
In this age of 20yo CEOs and single-quarter companies it's hardly suprising that most software is no better than a rigged demo.
Just make it shiny enough for someone to buy the company and then let their support staff of MS trained monkeys deal with it
Then we have the "artists" (in both the software and hardware field) who have survived for twenty years without "all that sh1t". Course, like the CEO, they've gone on to their next challenge long before the chickens come home to roost. And it's not their fault everyone else is incompetent, is it?
I continue to be amazed, on a weekly basis, by the complete lack of experience shown by the actions and products of very large companies.
Oh, I reject the claim that
The author has obviously never maintained hardware: it has bugs, patches, upgrades just like any other part of your system.
Re: (Score:2)
Typical modern CACM article (Score:2)
Long, wordy, buzz-word heavy article with a little bit of interesting content buried deep inside. I wish I hadn't bothered to read it.
In case you haven't, but are thinking you might: you can run machines that are never down, even when software is being updated, if you use a few tricks. I knew most of the one's they mentioned already, and use them on my company website, which is far from downtime-proof, but has a 3-year uptime so far: call my software maintenence status "fairly sturdy".
If you're interested
Re: (Score:3, Insightful)
Sure there's going to be bugs and exploits, and it can take a long time to port code, but a lot of it is self inflicted because of changes to the objectives or a failure to ensure that the code was properly engineered, commented and audited.
Properly formatted and written code should b
Re: (Score:2)
Modular design can really allow for fewer security holes, along with making everything more maintainable for developers. For example, if the people who make the OGG codecs fix security holes an audio editor doesn't have to rewrite their program to use the new codecs. The problem really lies with bits of incompatibility technology, Flash is a great example of it, while Firefox, Chrome and Safari all have good track records on secu
Re: (Score:2)
It's even possible to update/upgrade on-the-fly while the program is running. This is not a problem for client application as it's no issue to shutdown Open Office or Firefox to update, but a server program busy serving hundreds of clients that depend up on it.
With Shared Libraries in C/C++ and classes, etc in other languages one can create a architecture design of empty skeleton to hold all code as individual loadable modules that can be loaded/unloaded anytime. So you can load v2 of a module, tell the ap
Re: (Score:3, Funny)
You mean you don't remember Flash for ENIAC?
Re: (Score:2)
Hundreds of millions have been born and died in that time period.
Carpe Diem!
Re: (Score:2)
Actually, the population of the Earth has at least doubled in the past 60 years, so that's billions that have been born and died without having a 64bit version of flash.
Re:Wait a second... (Score:4, Insightful)
Coding is important, but what is also important is program design and architecture. This isn't thought of that much as a smaller utility that "scratches an itch" just needs to be coded.
However, larger projects, and projects with any security needs at all [1] need to have a lot of time put in to design security before a line of code is written. For example, a Web browser should see if it can run under a lower privilege context without preventing the user from downloading files. A Web browser plugin needs to assume any code it gets from a website is likely tainted, treat the code as suspect, and run it in a sandbox (CPU/filesystem/RAM limits, preferably enforced by the OS, and even better, able to have the code run in a limited context other than the user.)
Compartmentalization is important, so instance of a program cannot interfere with another. This way, something that mucks with a cache directory only affects that instance, and cannot modify another. Same with a thread of execution needing protection from others that might get code injected into them.
Separation of code modules that are vital to security is critical. This way, extreme code review can be focused on the security critical modules, while other modules (such as the one that renders stuff) can be reviewed, but it wouldn't have the extreme focus as the parser of incoming code.
This stuff is vitally important, and can't just be bolted on without a complete rewrite. It has to be part of the architecture from the first line of code on. However, there are two benefits from having this integrated into a core program design. The first is that the security critical code is in modules that can be thoroughly scrutinized and audited. The second is that the program will be far more resistant to attack, and if an attack is done, it can be fixed far faster than a product that has no security in its core design.
[1]: One would be surprised at what programs need security. Anything touching the Internet like a browser plugin, mail filter, or even a MUD server are just as important to have security factored in as a SUID root utility.
Re: (Score:2)
I'd guess it's that the PHB's, in all their imperial expertise, do not understand the need for abstract things such as commenting and modularization and whatnot, and instead opt to be hardasses about the one thing they DO know...numbers.
Faced with the bosses breathing down their necks to shit code out as fast as they can, is it any wonder that they are sloppy? Any programmer in an environment like that who DARED to suggest that they needed good design would probably get fired.
Re: (Score:2)
I run 64-bit flash, you insensitive clod!
Meme aside, I am. And, before it came out, I was running the 32-bit flash on Linux/AMD64 just fine, even with 64-bit browsers. At least under firefox. Under konqueror, konqueror would crash sometimes, though I've not had that crash in a while now - probably fixed in KDE 4.3.3.
FYI: nspluginwrapper [gibix.net] is what you need to run 32-bit plugins with 64-bit firefox (and konqueror, and, presumably, chromium).
Theory vs practice (Score:5, Insightful)
Doesn't modular programming solve this problem?
That is one of those answers that sound great in theory but in practice suffer from all the same problems from which all the other answers suffer.
Too much upfront cost is a contract-killer, so compromises in designing for scalability must be made.
Furthermore, today you have ideas about how the software will be used, and tomorrow those ideas will be changed, and you will discover that the dividing lines you have set up between your modules are not optimally efficient. This will result in simple-seeming enhancements being prohibitively expensive....unless you start crossing your module boundaries directly. The pressure to do this is too intense to resist, and you wind up violating the modularity of your design in the name of getting it out the door in time.
Sooner or later your system will get complaints that it is too slow, and you will need to open further holes through your module boundaries in order to optimize performance within acceptable delivery deadlines. You will warn that these are quick fixes which must be refactored properly after their release, but the opportunity to do this never comes.
As such changes form layers around your ever-growing onion, the core modules will be come too precious to change. Every little tweak you make to a core module will conflict with assumptions made by other modules, and will cause surprising bugs that are hard to detect in QA testing (since test case count grows exponentially with each new feature). This will result in more demand to fix problems without adjusting the core modules, or by making minimal adjustments to them, which will wind up forcing you to further compromise the modularity of your design.
As the complexity of your system increases, the cost of each new feature also increases, which will displease management and prompt them to say "we used to be able to do this sort of thing in a few hours...now it takes weeks!" They will continue to make unreasonable demands of their senior staff until they push said staff right out the door, leaving the ongoing maintenance of this (now very un-)modular system to the junior level programmers who never shared in the original vision of the system, and hence have no sense for where the module boundaries should be, and just fold to managerial pressure to hack in changes as quickly as possible.
Eventually the slow turnaround times and general bugginess of the system will drive clients to start looking for alternatives. Some brand-new ones will be available (some of which written by the disgruntled senior staff, in fact). Thus begins the end of the product, and of the company if this was their flagship.
Re: (Score:2)
Re: (Score:2)
All video games are like this to some extent, except perhaps the longest-lived MMOs (which have probably evolved beyond that simply out of necessity.) At least, all the ones I've perused the source code for, which is a lot.
I blame hard deadlines and crunch time, but I'm not sure if that's the real cause, or if it's just a different class of programmer writing video games. The excuse *used* to be optimization, but I don't think that applies nearly as much anymore.
Re:Wait a second... (Score:4, Interesting)
I blame hard deadlines and crunch time, but I'm not sure if that's the real cause, or if it's just a different class of programmer writing video games. The excuse *used* to be optimization, but I don't think that applies nearly as much anymore.
I think the game dev industry is growing up. Part of it has to do with the fact that a mature codebase is pretty valuable nowadays, and modern games are so complicated, those who don't take good software development processes seriously end up with a technical disaster, and it tends to show. It's very rare for a game to be written completely from scratch these days. When I was started, it wasn't all that unusual to start over for each new successive generation of games. Some of the base libraries I'm working on are nearly a decade old - about when the company I currently work for was founded. They're well-tested and highly optimized. We'd be insane to toss these out and start over.
You also have to differentiate between engine code and game code. By it's nature, game code is essentially a one-off project - essentially single-purpose code. Keep in mind that unlike commercial projects, if the requirements for version 2 of the game change, the code can be rewritten. There's no need to preserve old features or functionality unless the new game requires it as well, as each version of the game is essentially a new fork. To some, it looks sloppy to see hard-coded gameplay features, but it makes a bit more sense when you think about it in that light. MMOs do have it a little harder in this aspect, being so long in continuous development, so we have to think about things like preserving file format compatibility and such, similar to other commercial apps.
Engine code is typically treated quite a bit differently. It has to be much more robust than typical game code. A number of games will be using it, so the code has to be both modular and highly optimized, as well as being fairly general purpose. Obviously, that's a fairly difficult combination, so developers typically put a lot more effort into engine code, and tend to be more protective of it.
Oh, and good heavens, yes, we still have to optimize the hell out of our code (oddly enough, I've been doing that all last week). It just happens more often at the architectural and algorithmic level rather than at the functional level like it used to. It's true that we can get away with a lot more, but... we're also doing much more complicated things.
Re: (Score:3, Interesting)
No modular system survives version +0.1.
You have a perfectly modeled modular system, and you really think you thought of everything. Only to notice that someone comes up with some problem you didn't take into account. So what do you do? Redesign the whole modular system, unravel all the finished modules and rework them to fit the new interfaces you have to design? Nah, we can handle a single nonstandard data handoff...
Do I have to go on?
Re: (Score:3, Insightful)
So what do you do? Redesign the whole modular system, unravel all the finished modules and rework them to fit the new interfaces you have to design?
If you have to do that to accommodate a typical change in requirements, then your system was never really modular at all, was it?
Modules represent, among other things, units of change.
Re:Wait a second... (Score:4, Funny)
Doesn't modular programming solve this problem?
Theory, I'd like to introduce you to practice. You two are very different, you should have lots of things to talk about.
Re:Wait a second... (Score:4, Insightful)
Scope Creep
Re: (Score:3, Insightful)
Re: (Score:3, Funny)
Tell me about it... There's so much feeping creaturism going on in my project that I have to make a SAN check to go to work every morning (if I fail, I go).
Re:Wait a second... (Score:4, Insightful)
Oh, modularity is undoubtedly a good thing. There's a word for non-modular code: crap.
The problem is all the things you *can't* solve with modularity. Having chosen the wrong architecture because you had the wrong conception of the problem isn't fixed by modularity. You can swap out modules all day long and it won't help, because the problem is how the modules fit together. Then there are the interdependencies that exist outside your architectural conceptualizations; that pretty much includes most security issues. Your "modularity" is an abstraction you impose on the physical reality of computation. The black hats peek under the covers and find all the couplings you can "safely ignore".
Then there are the things your module depends on: library versions, frameworks, operating systems, databases etc. The answer to this is that most "modules" aren't really very modular because they're coupled to these things; I think of these couplings as "vertical" couplings as opposed to "horizontal" couplings with other modules I create. The best answer is to de-couple modules from those things too, to isolate dependencies in a very small number of interface modules. But that's a lot of work, and a lot of times you've got to do it with programmers who have something like "Struts for the Ignorant Programmer" open on their desk, giving them concrete, cut-and-paste examples of how to *couple* their code to some version of a framework. Those programmers are so far from the ivory tower they're living in the treacle well.
That's the problem of craft in a nutshell. You don't have unlimited time, but pure expediency can waste more of your time than anything else. You can start chopping down a tree sooner if you don't bother to sharpen your ax, and if the boss judges progress by ax cuts you'll probably end up doing it that way.
Re: (Score:2)
So they change something and if they can't see the bug anymore they go cvs commit right then and there.
Yes. Bug trackers should have statuses like "Developer in denial" for situations like that. (Mozilla's bug tracker has a "WORKSFORME" status which is used far too much.)
Re: (Score:2)
And that means the validation people run through their test book and create maybe 1000 or 2000 bug reports. These bug reports bounce around through the development teams acquiring cruft along the way. When the bug eventually stops bouncing somebody might have a go at fixing it. So they change something and if they can't see the bug anymore they go cvs commit right then and there. At the same time the other 1999 bugs are bouncing around, looking for a home.
At the end of the (as we call it) bug fixing process the original software doesn't exist in its original form. It appears to pass most of the test sbut you would not be advised to lean against it or anything because something else might break.
software maintenance =/= bug fixing.
Re: (Score:2)
Wasn't the claim, Watson.
Strong typing would make it impossible to send old format data to the API -- which would then force you to create an API extension. Couldn't just subsume the functionality.
Yes, weak typing along with versioning will work. "Strong typing" (ala C++ implementations "add the parameter descriptors in detail to the argument") won't, simply because the same "type" isn't being passed.
Yes, this can be misused to add 1 to "1". Which, in itself, isn't such a bad thing. After all, my Rosetta St