We Really Don't Know Jack About Maintenance 260
davecb writes "The ACM has been kind enough to print Paul Stachour's and my 'jack' article about Software Maintenance. Paul first pointed out back in 1984 that we and our managers were being foolish — when we were still running Unix V7 — and if anything it's been getting worse. Turns out maintenance has been a 'solved problem in computer science' since at least then, and we're just beginning to rediscover it."
Grrr (Score:2, Insightful)
"Software maintenance" has absolutely nothing to do with computer science. I wish people would stop calling business programming computer science. Computer science work gets done at universities and research institutions, not at Initech.
"Everyone knows maintenance is boring" (Score:5, Insightful)
Taking code and cutting its size by half, fixing up all the screwed-up inconsistent formatting, while adding functionality and reducing bug counts, is a pleasure.
It's all in the mindset. It's only boring if you limit yourself to the boring parts.
Move the functions to Business (Score:3, Insightful)
No, I'm serious. It's the latest Trend (tm).
Cut the IT production support budget, cut the IT staff, move the functions that used to be IT Prod support to business and let their budget handle it.
If you can't see the humour in seeing a business person make direct live updates to a database table in Production (IT doesn't have Production access.. but Business does .. [go figure]) then you probably can't see that SEV1 sailing in from over the horizon to make your day just that little bit more special.
As for business editing files directly in Production because the cost of having IT do it (process - backup file, edit file, copy file to prod with due authorisation, verify the change) can just be avoided.
After all, we don't really need to pay for Production Support and System Maintenance and Documentation. The system works without these things, doesn't it?
What could possibly go wrong?
Re:Wait a second... (Score:3, Insightful)
Sure there's going to be bugs and exploits, and it can take a long time to port code, but a lot of it is self inflicted because of changes to the objectives or a failure to ensure that the code was properly engineered, commented and audited.
Properly formatted and written code should be relatively easy to maintain or at least not a total nightmare.
Important forgotten steps (Score:5, Insightful)
Re:"Everyone knows maintenance is boring" (Score:5, Insightful)
What company do you work for where it is even remotely possible to accomplish all that and still meet your deadlines? In my world, you always wish you could totally refactor the code like that, but there's never enough time in the schedule for that. So instead, you "make it work" as best as you can, and that's about it. Maybe, if you're lucky, you can find enough time to do at least a partial re-write of the worst parts of the code, or tidy up an interface or two. But cutting the code size by half? Yeah right. I wish.
That's mighty elitist of you (Score:4, Insightful)
"Software maintenance" has absolutely nothing to do with computer science.
Actually, it does.
I have a real Computer Science degree, so I know what computer science is about. And while a lot of corporate programming is more drudgery and form assembly than anything, there are a lot of applications of computer science in the real world - from scalability issues in large systems, to proper use of encryption.
Furthermore the supposedly boring area of "Software maintenance" has a ton of research potential focused around the optimal path to producing correct code. Do code review help? How to team dynamics factor in to code quality? Does Test First really improve code? What even defines code quality? There are programs within companies that experiment with different methods to improve code output, and those experiments are even more valid than ones performed at research institutions since they work on a real-world code base solving a real problem using real people. In fact I would go so far as to say and research being conducted outside of a company on aspect of code quality improvement is pretty much worthless, which is why it's important for researchers to partner with real companies for some studies.
Re:Important forgotten steps (Score:3, Insightful)
See, I've always looked at meetings primarily as a way for project managers who have no technical work to do on the project to feel like they're doing something.
In fact, this purpose explains the following relationship:
Number of meetings per week = Stress level of project manager / Time remaining before project deadline
So early on, you can expect a kickoff and a couple of weekly status meetings. As the deadline begins to loom, suddenly you're looking at 2-3 meetings per day per developer, with many meetings getting called at the last minute. It makes the PM feel like they're doing what they can to make things happen quickly (even if it doesn't), and keeps them from having to face the reality that the project will be late, they can't do anything to prevent that, and they will be held responsible.
That is not maintenance. (Score:5, Insightful)
Taking code and cutting its size by half, fixing up all the screwed-up inconsistent formatting, while adding functionality and reducing bug counts, is a pleasure.
Yes it is.
But that is not maintenance, as practiced by any rational company. That is development or (more specifically) optimization.
Maintenance is about solving a problem code is having, with the absolute smallest number of changes possible. Even new features can fall under this heading when software is in true "maintenance mode" in order to avoid a lot of excess testing.
I actually don't mind it, as it is a different sort of challenge to take a code base you know little about and introduce a working change with as little code as possible. But it's not as glamorous as fresh, raw coding.
Re:Important forgotten steps (Score:4, Insightful)
Dipshit PHBs would be buying them left and right!
Re:"Everyone knows maintenance is boring" (Score:5, Insightful)
More likely, the code was written and documented badly in the first place, so it's not a matter of taking it and cutting it down into something more elegant. Instead, it's about trying to decipher exactly what's going on, then trying to figure out how to make it at least superficially better while trying to keep within the same crappy framework. This has to all be done within budget, or the work done will go to waste when the project's scrapped, and there won't be a budget for maintenance again.
Maintenance isn't a rewrite from scratch to do the same thing, as much as we'd like it to be. Production systems have to be kept up. Migration to new code has to be both backwards compatible and produce enough visible results to make it worthwhile. It can't be too big of a change, nor too little. It's hardly any wonder software developers don't get to, and managers don't want to do it.
Re:"Everyone knows maintenance is boring" (Score:5, Insightful)
You don't tell them you're doing that ... you just do it. It's quicker anyway. Start by cleaning up the formatting and moving stuff into headers, some righteous macros to make the code more readable, etc., as a first step in "getting familiar with the code. After all, you don't want to break something while fixing a problem. You'll probably notice some code that looks like it's either "copy-n-paste" or dead. Keep it in mind.
You'll find potential for simplifications as you go along, and you'll also probably find other bugs and stupidities, like parameters that are never used (that happens a lot as code gets modded). Code that doesn't smell right. Signed ints that are unsigned elsewhere. Potential buffer overruns. Unsanitized unput. In-line code that really should be in a separate function call both for clarity and for re-use. Look through the bug list - if the bug (or potential bug) you found is there, great - mark it as fixed. If not, log it, then mark it as fixed. Either way, your fixed bug count goes up - and if you can show that you've fixed bugs before they're found, that's just a bonus.
Now start a bit more higher-level fixing. Nothing too wild, but if you have to add a bit of functionality, use that as an excuse to simplify the code that it touches. You'll probably confirm some of the dead code from the first step. Remove it. You'll probably confirm cases of multiple instances of copy-and-paste - refactor them as well into something a bit more same. Now you've removed a few more bugs, shrunk the file size, and added some functionality.
Now that you've done that, review the code again, and look for more instances of copy-and-paste. Code that was written in a hurry will have LOTS of it, so you'll definitely be able to give it some more w8loss treatments. And since you didn't bitch and moan TOO much about making the mods, and they actually work, they'll probably be happy to let you graze on the bug-fix list. There's always bugs, nobody wants to touch them, and as you clean up the code, you'll see it start to shrink.
A 50% reduction on crappy spaghetti code is reasonable under those circumstances.
At that point, you own the code. You're in the loop for changes that might break it. When you say, "okay, but it's going to take x time" they're more likely to actually listen. Consider it office politics for nerds.
Re:"Everyone knows maintenance is boring" (Score:4, Insightful)
Re:Wait a second... (Score:4, Insightful)
Coding is important, but what is also important is program design and architecture. This isn't thought of that much as a smaller utility that "scratches an itch" just needs to be coded.
However, larger projects, and projects with any security needs at all [1] need to have a lot of time put in to design security before a line of code is written. For example, a Web browser should see if it can run under a lower privilege context without preventing the user from downloading files. A Web browser plugin needs to assume any code it gets from a website is likely tainted, treat the code as suspect, and run it in a sandbox (CPU/filesystem/RAM limits, preferably enforced by the OS, and even better, able to have the code run in a limited context other than the user.)
Compartmentalization is important, so instance of a program cannot interfere with another. This way, something that mucks with a cache directory only affects that instance, and cannot modify another. Same with a thread of execution needing protection from others that might get code injected into them.
Separation of code modules that are vital to security is critical. This way, extreme code review can be focused on the security critical modules, while other modules (such as the one that renders stuff) can be reviewed, but it wouldn't have the extreme focus as the parser of incoming code.
This stuff is vitally important, and can't just be bolted on without a complete rewrite. It has to be part of the architecture from the first line of code on. However, there are two benefits from having this integrated into a core program design. The first is that the security critical code is in modules that can be thoroughly scrutinized and audited. The second is that the program will be far more resistant to attack, and if an attack is done, it can be fixed far faster than a product that has no security in its core design.
[1]: One would be surprised at what programs need security. Anything touching the Internet like a browser plugin, mail filter, or even a MUD server are just as important to have security factored in as a SUID root utility.
Theory vs practice (Score:5, Insightful)
Doesn't modular programming solve this problem?
That is one of those answers that sound great in theory but in practice suffer from all the same problems from which all the other answers suffer.
Too much upfront cost is a contract-killer, so compromises in designing for scalability must be made.
Furthermore, today you have ideas about how the software will be used, and tomorrow those ideas will be changed, and you will discover that the dividing lines you have set up between your modules are not optimally efficient. This will result in simple-seeming enhancements being prohibitively expensive....unless you start crossing your module boundaries directly. The pressure to do this is too intense to resist, and you wind up violating the modularity of your design in the name of getting it out the door in time.
Sooner or later your system will get complaints that it is too slow, and you will need to open further holes through your module boundaries in order to optimize performance within acceptable delivery deadlines. You will warn that these are quick fixes which must be refactored properly after their release, but the opportunity to do this never comes.
As such changes form layers around your ever-growing onion, the core modules will be come too precious to change. Every little tweak you make to a core module will conflict with assumptions made by other modules, and will cause surprising bugs that are hard to detect in QA testing (since test case count grows exponentially with each new feature). This will result in more demand to fix problems without adjusting the core modules, or by making minimal adjustments to them, which will wind up forcing you to further compromise the modularity of your design.
As the complexity of your system increases, the cost of each new feature also increases, which will displease management and prompt them to say "we used to be able to do this sort of thing in a few hours...now it takes weeks!" They will continue to make unreasonable demands of their senior staff until they push said staff right out the door, leaving the ongoing maintenance of this (now very un-)modular system to the junior level programmers who never shared in the original vision of the system, and hence have no sense for where the module boundaries should be, and just fold to managerial pressure to hack in changes as quickly as possible.
Eventually the slow turnaround times and general bugginess of the system will drive clients to start looking for alternatives. Some brand-new ones will be available (some of which written by the disgruntled senior staff, in fact). Thus begins the end of the product, and of the company if this was their flagship.
Re:"Everyone knows maintenance is boring" (Score:4, Insightful)
But if you have 20 or 30 branches for deployed systems and your fix needs to be imported into most of them then your clean up is viewed as a total PITA because now it is not clear why you made all those changes.
Re:That's mighty elitist of you (Score:2, Insightful)
You're using the new definition of CS not it's traditional meaning.
The combination of academic elitism, a societal trend toward specialization, and a CS tendency to sub-divide complex problems have combined to split the traditional, interdisciplinary Computer Science into multiple sub-fields in the last decade.
Traditionally we had Computer Engineers to design and build hardware and Computer Scientists to design and build software to make use of those devices. (The business world added Computer Information Systems specialists who could translate the jargon of the other two into something comprehensible to ordinary folks.)
The old CS is a combination of mathematics, science, and engineering all rolled together. The interdisciplinary nature provided great energy, but perhaps trying to hold such an unwieldy mess together was futile. Now we have the "pure CS" crowd restricting itself to the mathematical foundations with occasional small steps into science. The engineering aspects are claimed by the new "Software Engineering/Infomatics" field and the "Information Technology" area provides a nice holder for those "practical" issues of actually using technology.
Hopefully after another decade folks will see the folly in over-specialization.
Re:Wait a second... (Score:4, Insightful)
Scope Creep
Re:Grrr (Score:4, Insightful)
""Software maintenance" has absolutely nothing to do with computer science. "
And that attitude is EXACTLY why computing still sucks.
If it involves computers, and it's an interesting unsolved problem - and guess what, a LOT of real-world problems turn out to be extremely interesting and extremely unsolved, full of all sorts of corner cases and exceptions to previously supposed 'laws' - then it's computer science.
System administration is like artificial intelligence. It's a human-centric job involving a lot of common sense, and it can't always be readily automated because the rules keep changing. That ought to raise warning flags. .Like vision or language recognition, a lot of scientists seem to think administration is so easy that only dumb people bother to investigate it. So they don't. But that's the opposite of the truth. Humans are still stuck doing rote maintenance precisely BECAUSE it's so tricky to do that computers can't yet do it. Which means it's very, very interesting.
A sensible approach to computer science would treat all phases of the development and deployment of information, computing and communication systems as computing systems in themselves. Yes, that involves crossing disciplinary boundaries. Perhaps that's more cybernetics or systems thinking or linguistics or sociology or psychology than computer science as it's currently narrowly defined itself to be. But in that case, computer science has defined itself out of the interesting part of the game - the impact and use of computing in the real world and what parts of this can and can't be predicted and automated - which would be a very sad thing.
Re:"Everyone knows maintenance is boring" (Score:5, Insightful)
That's great, until your huge refactoring introduces a new bug. And it will, you can't expect to refactor a decent sized body of code without breaking something. When it does, it doesn't matter that the code is 100 times more readable and extensible, or that you fixed ten other bugs while doing it, your ears will get hot. Because you broke something which means more work for someone else and that the count of outstanding bugs was just incremented.
This happens in the places where each bug report has to be annotated with about two dozen different abbrevations and numeric codes so that the correct departments and bug examiners will get to look at it. When it takes two hours to file bugs and each bug has to be examined by four persons, you can forget about filing bugs about "potential problems." Bug reports are for the issues customers have complained about and nothing else.
What about the next bug report that comes on? If it can even remotely be associated with the new code you wrote, then it will be. Because yours is the only new code in what is otherwise a totally legacy code base with hundreds of band aids. Expect to take the blame.
And then you have the original author of the body of code you just killed/refactored. He wont be so amused about you rewriting in one week what he spent four months writing. Expect to hear nitpicks and complaints from his side. You won't be able to ask him anything about the code anymore because "you destroyed it."
So yes, while refactoring code makes it better, you wont make many friends while doing it and you are taking a huge personal risk. Especially if you are doing it secretly.
Re:Wait a second... (Score:3, Insightful)
Re:"Everyone knows maintenance is boring" (Score:1, Insightful)
That's an astonishingly naive response. Here's several reasons why:
1. You casually dismiss the "no time" issue by claiming that refactoring is actually faster than just making it work. That may be true in some cases, but when you're talking about half a million lines (not exaggerating) of legacy code, it's not that simple.
2. There are other developers to consider, especially when you're talking about such a large code-base. You can't just go off and decide to undertake a major overhaul of the code without coordinating with the other developers. Good luck getting them on board with your big plans, when they are already working 60 hours a week trying to get their actual assignments done. Undertaking such a major refactoring effort would undoubtedly pay big dividends in the long run, but in the short term it's going to spell disaster.
3. Your whole argument is built on a heap of assumptions. You're just pulling numbers out of thin air, and assuming they'll work in all cases. You're also assuming that management isn't going to notice what you're up to. Yeah, I know all about the pointy-haired boss stereotype, but my bosses aren't so conveniently stupid.
Believe me, I really wish refactoring was just a simple matter of having the will and the know-how, but in a lot of real-world situations, that's just pure fantasy.
Re:"Everyone knows maintenance is boring" (Score:3, Insightful)
Taking code and cutting its size by half, fixing up all the screwed-up inconsistent formatting, while adding functionality and reducing bug counts, is a pleasure.
Clearly you've worked only on code that needs a bit of tidy up. Try reading code that hasn't been documented well if at all, and looks like it's been run through a code obfuscator (ie not just poor formatting), then realising that not only is the task the code is suppose to accomplish not documented, but the people who wrote the original code left while you were still in primary school. Oh by the way it's part of a poorly integrated but critical system and if any part doesn't work exactly the way the old part did, you're in trouble because the whole system falls apart.
The trouble is even a gifted coder is likely to miss something, and those that review will have a shallower not deeper understanding so while they MIGHT pick up something important don't hold your breath. Realistically your only hope in many cases is to start fresh with a new analysis and design and replace your critical system big bang style (while trying not to go grey worrying over that approaching day).
It's all in the mindset. It's only boring if you limit yourself to the boring parts.
Most of the time, I'd kill for boring. I don't come to work for excitement.
Re:"Everyone knows maintenance is boring" (Score:3, Insightful)
That's great, until your huge refactoring introduces a new bug.
You can always make sure you've got some legitimate reason to have done the refactoring/redesign. I got away with rewriting a section of code that the owner of the company had cordoned off with comments like "DO NOT TOUCH, YOU CAN'T MAKE THIS ANY FASTER" because I made it run about 10 times faster. (Honestly, how can you ignore that kind of comment if you like programming?)
I checked in a bug fix (with unrelated speed improvements included) that introduced another bug, but also dropped application CPU usage from 75-100% to under 10%. The owner was initially pissed, of course, but I fixed the bug I introduced in under an hour, and showed that it was still a lot faster in addition to working correctly, and in the end the only reprimand I got was, "well, don't do that again without asking."
For the next few years I kept replacing bad code if I had to touch it, and, no, I didn't always ask first. I was more careful about testing my changes and (since the standard development model at this place was "seat of your pants" anyway) I rarely had any issues because of it.
Refactoring bad code is only scary if you're afraid to take responsibility for the code you put out.
Re:That is not maintenance. (Score:4, Insightful)
I believe that no developer can ever become a good or great developer without having done significant maintenance on someone else's code - your really only learn the true value of "coding for maintainability" (i.e. basics like commenting complex code, avoiding copy and paste, fail-fast code and such) when you're landed with fixing/improving code that was done without any such concerns.
That said, software maintenance is often a frustrating, painful process in anything but the freshest of code (and sometimes even is code "just out of the over").
It's especially unpleasant when one is faced with fixing/changing software where some or all of the design or the code were done by (often long gone) less experienced software developers.
The bad news is that as you become more experienced you see more and more "glaring problems" with the design/coding on any system you have to fix/improve, which can be infuriating at times (for example, when the design and development of an important sub-system was given to a team composed wholly of average/below-average developers with 4 years of less of experience and you are tasked with fixing it)
The good news is that the outsourcing wave has caused a bubble in IT in India, resulting in loads of people joining IT for the money (most of which should never have done so) so there is a whole lot of really, really bad code out there which will be providing freelancers (like me) with highly-paid maintenance work for years to come.
Re:Grrr (Score:3, Insightful)
Programming language theory is computer science. I'll even concede that programming with a FP language is very computer science-y. Techniques for writing maintainable code is not.
Many activities carried out every day for performing software engineering is not computer science. But good software engineering, the discipline itself, that is deeply root in computer science. Just like coding. Coding is not computer science, but good coding practices are rooted computer science.
Principles and notions such as cohesion, coupling, modularity, comprehensiveness, patterns, complexity management, architecture, they all originate in computer science.
I think you are right that software (and systems) engineering is computer science. But you might want to be precise as of why. Software Engineering is to Computer Science and Mathematics what the "physical" engineering disciplines are to Physics, Chemistry and Mathematics. The former cannot exist without the later.
Re:Wait a second... (Score:4, Insightful)
Oh, modularity is undoubtedly a good thing. There's a word for non-modular code: crap.
The problem is all the things you *can't* solve with modularity. Having chosen the wrong architecture because you had the wrong conception of the problem isn't fixed by modularity. You can swap out modules all day long and it won't help, because the problem is how the modules fit together. Then there are the interdependencies that exist outside your architectural conceptualizations; that pretty much includes most security issues. Your "modularity" is an abstraction you impose on the physical reality of computation. The black hats peek under the covers and find all the couplings you can "safely ignore".
Then there are the things your module depends on: library versions, frameworks, operating systems, databases etc. The answer to this is that most "modules" aren't really very modular because they're coupled to these things; I think of these couplings as "vertical" couplings as opposed to "horizontal" couplings with other modules I create. The best answer is to de-couple modules from those things too, to isolate dependencies in a very small number of interface modules. But that's a lot of work, and a lot of times you've got to do it with programmers who have something like "Struts for the Ignorant Programmer" open on their desk, giving them concrete, cut-and-paste examples of how to *couple* their code to some version of a framework. Those programmers are so far from the ivory tower they're living in the treacle well.
That's the problem of craft in a nutshell. You don't have unlimited time, but pure expediency can waste more of your time than anything else. You can start chopping down a tree sooner if you don't bother to sharpen your ax, and if the boss judges progress by ax cuts you'll probably end up doing it that way.
Re:"Everyone knows maintenance is boring" (Score:4, Insightful)
Then you have a process and management problem, not a coding problem.
One person should have the authority to approve or disapprove.
It's like when you make a complaint, and they say "I'm sorry, I can't do that." The proper response is "Then why am I wasting both of our time talking to you? Please pass me to someone who can."
Strictly a management problem.
Re:Wait a second... (Score:3, Insightful)
So what do you do? Redesign the whole modular system, unravel all the finished modules and rework them to fit the new interfaces you have to design?
If you have to do that to accommodate a typical change in requirements, then your system was never really modular at all, was it?
Modules represent, among other things, units of change.
Re:Grrr (Score:3, Insightful)
""Software maintenance" has absolutely nothing to do with computer science. "
And that attitude is EXACTLY why computing still sucks.
If it involves computers, and it's an interesting unsolved problem - and guess what, a LOT of real-world problems turn out to be extremely interesting and extremely unsolved, full of all sorts of corner cases and exceptions to previously supposed 'laws' - then it's computer science.
No true. Software Engineering solves a lot of unsolved problems without having to do computer science. Computer Science is about theory, not implementation. Computer Science deals with the Universal Turing Machine and treats all computers like it. Computer Science programs tend to not discuss how to do things with efficiency or with limited resources.
Computer Engineering programs tend to talk a little about Computer Programming; but mostly focus on the engineering behind computer hardware (e.g. PCI boards, processors, etc.).
Software Engineering is about producing products that resolve unresolved problems using standard practices. It's about architecting, designing, and otherwise engineering a product to meet that problem and overcome it. It's done with computers, solves interesting unsolved problems, and has a lot of corner cases and exceptions - far more than you find in most any Computer Science theory since the problems Software Engineering addresses are typically for limited memory/processor/etc, making the challenge that much more.
And no, this is not a new definition of Computer Science, or Computer Engineering. It's just that so many Computer Scientists fail to understand the differences between Science and Engineering.