How To Deal With 200k Lines of Spaghetti Code 236
An anonymous reader writes "An article at Ars recaps a discussion from Stack Exchange about a software engineer who had the misfortune to inherit 200k lines of 'spaghetti code' cobbled together over the course of 10-20 years. A lengthy and detailed response walks through how best to proceed at development triage in the face of limited time and developer-power. From the article: 'Rigidity is (often) good. This is a controversial opinion, as rigidity is often seen as a force working against you. It's true for some phases of some projects. But once you see it as a structural support, a framework that takes away the guesswork, it greatly reduces the amount of wasted time and effort. Make it work for you, not against you. Rigidity = Process / Procedure. Software development needs good processes and procedures for exactly the same reasons that chemical plants or factories have manuals, procedures, drills, and emergency guidelines: preventing bad outcomes, increasing predictability, maximizing productivity... Rigidity comes in moderation, though!'"
"Cobbled together over 10-20 years . . . ?" (Score:5, Insightful)
Wouldn't that be Linux? It seems to work fine for me.
If something has become spaghetti over 10-20 years, then no one cared that it became spaghetti over 10-20 years. And it will still be spaghetti over the next 10-20 years. Fixing something like this requires a commitment from management, which means money. If the management of the project aren't convinced that cleaning up the development process is worth the initial investment for the long term, then they choose to deal with the constantly higher costs forever.
Something like this makes me think that this is one of those problems that get pushed off for someone else to deal with later. And the next person perpetuates this, by doing the same.
Re:"Cobbled together over 10-20 years . . . ?" (Score:2, Insightful)
The really frightening tought is that there are many 40 year old first year CS students.
1. Lose the attitude (Score:5, Insightful)
Well step 1 would be to lose the attitude.
It's code, it may be in an obsolete language, it may be not to the best industry standards, but its code and it's got enough knowledge in it, that nobody wants to throw it away, and they hired you to maintain it.
Step 2, I don't know why you would define a process before you understand the code you are to apply the process too?
Seriously, wtf is all the process stuff about, you're the sole programmer, any rules you set are rods to break your back when you first hit a piece of code you have to break the rules.
Step 3, you serve them. If you want to port it to a more modern maintainable language, choose one that's easy for THEM to transition to. They've got the knowledge that drives the company, not you, you are the cleaner here. If the phone rings your turn off your vacuum and let them do their job, Mr Cleaner. Nobody gives a fork if the cleaner has best industry procedure for cleaning an office.
Step 4, break it down. tiny bit by tiny bit, port to a new CLEAN structure, bit by bit. They wrote it, they can identify the core stuff.
Step 5, once you've ported it, along comes an engineer with a code written to the old language and old methods. Again, that's fine, put away the process manual, these are the experts, if that's the language he can communicate to you in, it's fine, you can understand it, you can port it, you can help him speak the modern lingo. Don't quote your processes to him, you're just the cleaner.
As for this:
"Software development needs good processes and procedures for exactly the same reasons that chemical plants or factories have manuals"
That's someone who *implements* things, typically a bolt together module manager. He is not someone who creates *new* things. Because news things don't come with manuals. You don't know the rules of how they work till the problems needed to make them work are solved. One assembles Microsoft IIS blocks, the other works for Google on image processing. Which are you?
Re:"Cobbled together over 10-20 years . . . ?" (Score:4, Insightful)
would fixing it to not be spaghetti be any better though? would fixing it to be really non-confusing depend on changing real world processes of clerks etc?
spaghetti turned into OO is going to be spaghetti too, just with a lot more of sauce and different bits which look almost the same but turn out to be tomato or minced meat upon closer inspection and when you take plunge into the platter and eat something you don't really know what you ate or how it got there, what I'm trying to get at is that if it's spaghetti because of already re-using lots of the code in lots of places then the rewrite could end up being 2 million lines (commercial rewrites of sw for gov. have a habit of ending up like this..).
at least he can probably feel good about it not being knights capital trading system.
but he's asking how to deal with it, with good development tools(that have good find/locate) and good memory. start exploring what the sw really does and guessing where you might be asked to do modifications.
Don't touch it (Score:5, Insightful)
200,000 lines ISN'T THAT BIG (rewrite)... apk (Score:0, Insightful)
Try projects of "enterprise class" that span into the MILLIONS of lines of code.
* I'm serious here...
Yes, there's a BIG difference between walking into a system with 200k lines (this is REWRITE territory still imo, & it's a professional one with over 17++ yrs. & around 30 projects of the MILLIONS OF LINES sized code for information systems)
vs.
One with millions of lines in it (not so simple to just rewrite).
APK
P.S.=> There's a time to rewrite, and a time not to, and when you're sub 500,000 lines, a rewrite IS possible and if it is as "bad" as this article *tries* to make it out as, then you rewrite when it's THAT small (& yes, 200k lines is TINY)...
I've done my assessments here based on doing smaller stuff from:
---
1.) The shareware/freeware level (most of mine here maxed out at around 5,000 - 10,000 lines of code tops)
2.) Commercially sold code in utilities I am part of in THAT market (usually midrange size 10,000 lines - 100,000 lines)
3.) Finally, and yes, "Enterprise Class/Industrial Strength/Mission Critical" information systems (these had millions in front end code alone, toss on SQL Stored Procs & more? You get the picture!)
---
The latter's where I've made the largest part of my livelyhood professionally since 1994 ( these definitely can often be quite large into that MILLIONS of lines of code range, with MANY "moving parts" in libs, main code, SQL stored procs, and tables/devices on DB's galore + more)...apk
Re:Rewrite it (Score:5, Insightful)
200k isn't something you're going to rewrite in a couple of weeks. I think the absolute maximum you could get (for one very skilled person) would be about 5k-10k per week. Rewriting would take on the order of half a year.
Re:"Cobbled together over 10-20 years . . . ?" (Score:3, Insightful)
When I was taking my 2nd degree in Computer Science years ago, there was quite a few mature students. They are actually doing well in class in spite of other distractions in life such as family.
There was nothing wrong with people deciding to learn new things or switch career later in life. Are you biased or just like to discriminate people for their age?
I would think that matured people are more suited for computer science as they are not as much distracted by the new "Shiny" new thing that comes along and constantly have to redo things not because they don't work, but they are not the "latest and greatest".
Re:...no (Score:5, Insightful)
I disagree. For 200K lines of code, You immediately start a new project to produce the next major release of said code.
200,000 lines of code is a large project, but very do-able for even with a small team or one person. Although you could go in an attempt to tighten up code in smaller chunks, the very fact that this code was written over the course of many years, probably by many authors and styles, means it probably follows no standard to general layout, declarations, etc. (hence the spaghetti).
I would simply support what's there with only a break-fix policy, and immediately start documenting all aspects of it's functionality to rebuild it from the bottom up. The very fact that this code would have so many styles would mean most of it would have to be re-written and documented anyway.
Document the functionality, re-implement with standard code to guidelines, include any feature enhancements that may exist, release new version.
Inexact Results (Score:5, Insightful)
Rewrite it from scratch using the spaghetti code version to run correctness tests to verify you haven't changed the behaviour.
And just how are you supposed to write "tests for correctness" when the very concept of what is "correct" is embedded in the code?
Any such tests would embody your own notions of what is correct based on your understanding of a codebase that cannot be understood.
Furthermore, Doom is quite a different thing. You have an end result that can be somewhat different and it doesn't matter - it could render textures such that they appear rather different but if you find it visually OK then it's fine.
No such luck with business software which usually has extremely rigid and exactly output, often output other systems are depending on being just so. There is no room for alteration of behavior, yet as I said no-where exclaims all of the features of the output you cannot possibly understand....
I agree with a few responses that the only way to proceed is to re-write tiny parts, that at most affect one other system in the company - with the explicit buy-in from those other groups something may change, and the understanding you may have to back out your changes wholesale if you cause too much disruption.
Can't get buy in to proceed? Then quit or work with the code as is.
Re:...no (Score:4, Insightful)
Re:Don't touch it (Score:5, Insightful)
Beware of the second system effect.
http://en.wikipedia.org/wiki/Second-system_effect [wikipedia.org]
Rewriting code can kill you in the short term.
http://www.joelonsoftware.com/articles/fog0000000069.html [joelonsoftware.com]
Or help you in the long term.
http://notes-on-haskell.blogspot.com/2007/08/rewriting-software.html [blogspot.com]
I recall another similar article about a rewrite of MS Office, and what a mistake it was...
Re:1. Lose the attitude (Score:3, Insightful)
unfortunately, I think most devs (especially the kind to complain about someone else's "crufty" code) will spend months rewriting, refactoring and introducing today's "best practices" like IoC and Dependency Injection and come up with 300kloc of even worse spaghetti code, that now has extra bugs to be fixed too.
A bit like how a discussion on stack overflow ended up discussed on ArsTechnica for some (probably advertising-related purpose) and now has come to Slashdot for further adoption. 4chan, you're next.
Good luck with that (Score:2, Insightful)
Your employer probably isn't interested in spending the time and money on a re-write. Nor are the clients going to be interested in waiting that long for new features, either.
You will be made to figure it out and add features, or you will be shown the door.
Re:Don't put up with it. (Score:5, Insightful)