When Rewriting an App Actually Makes Sense 289
vlangber writes "Joel Spolsky wrote a famous blog post back in 2000 called 'Things You Should Never Do, Part I,' where he wrote the following: '[T]he single worst strategic mistake that any software company can make: They decided to rewrite the code from scratch.' Here is a story about a software company that decided to rewrite their application from scratch, and their experiences from that process."
Here's my short list (Score:4, Funny)
5. When it is written in Visual Basic. Always.
4. When I'm getting paid by the hour and it is written in Visual Basic. Always
3. When it was written in a mid-90s WYSIWIG bastard child of a mid-80s interpreted language.
2. When it uses a thousand "IF-THEN-ELSE" when it means to use regular expressions
1. When it is written in Visual basic.
Re: (Score:2, Insightful)
For me readability > length of code.
I always find it funny when developers are struggling to fit their code on as few lines as possible. Like the forgot how to scroll or something.
Still, if it was written in Visual Basic, I'd let you take care of it.
Re: (Score:2, Informative)
Do you really think that a thousand if-else statements is more readable than a regex statement. If it takes you one hundred lines of code to test for one condition, then the code becomes less readable, even if each piece makes more sense. RegEx is optimized to test for formatting of a string. This is what it was meant for.
Pseudo-code:
'If Email Address
If (String.RegExMatch("^([a-zA-Z0-9_\-\.]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([a-zA-Z0-9\-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?)$")) Then
'Do Stuff
El
What, no plus sign? (Score:3, Informative)
Re: (Score:2)
Sorry about that. I just used that as an example and did a quick search for a code. I agree that I am frustrated when a program doesn't validate, but the entry is to specification. The poster after you posted a link to a proper email regex string.
Re: (Score:2)
You are aware that parsing an email address is not the same as saying pino@example.com is equivalent to pino+whatever@example.com ... the + is something done by the server, nothing to do with the standard.
Granted, most email validators don't follow the RFC. I still go with "the only way to validate an email address is to send an email to the address and monitor for a valid response" (meaning user interaction, bounceback, etc).
Re:Here's my short list (Score:5, Informative)
dude, THIS is the regex to validate an email address: http://www.ex-parrot.com/pdw/Mail-RFC822-Address.html [ex-parrot.com]
Re: (Score:2)
For those who don't want to follow the link: (Score:2)
My $.02 (Score:3, Insightful)
I'm not a real programmer. I used to play with BASIC back 25 years ago or so and I've fooled around with writing C code for microcontrollers a bit lately but my skills are very limited to say the least. None the less, I was able to parse that regular expression without too much trouble. That leads me to believe that it can't be all that hard for someone who codes for a living to understand it. Still, it would be helpful to spend a minute or two with some helpful commenting.
Re:Here's my short list (Score:5, Insightful)
Regarding regexes, I've gone both ways: I've had my developers remove regexes where they were trying to use them, and I've had them add regexes where they were trying too hard to avoid them. Regexes aren't the answer, they're a single tool. You need to use them right. And the /x modifier in Perl helps a lot, allowing you to put useful whitespace and comments in the code (and a regex is code as much as any other language).
As for "as few lines as possible" - you need to do it right. When you span your code over 3 pages, that's not readable anymore. I harp on this with my developers all the time: SCOPE. Scope applies to variables, comments, and functions. The less I need to look at to understand what you're doing, the more likely it is I'm going to understand it. A huge 3-page if-elseif-elseif-else is going to be something I'm not going to understand, as I'll have forgotten all the conditionals by the time I get to the end to know really what scenarios are left - sequential access. A concise regex, on the other hand, is something that I can skim over just by moving my eyes - random access. These concerns aren't just valid for storage media (tape vs DVD/HD). Of course, 40 characters of regex special characters with no whitespace (either horizontal or vertical, preferably both) is generally going to overwhelm most readers, and is going stupid in the other direction.
Yes, readability trumps length of code. But sometimes that means to use a regex (or two or three - why make one big cryptic one when multiple simpler regexes can do the job?). And sometimes, that means avoiding them when what you really want to do is better done by another piece of code.
My favourite new-to-regex example recently has been someone trying to pull apart colon-delimited text with a regex. Woops - there are better language constructs for tokenisation, whether that's strtok in C or it's split in Perl (or better, Text::CSV_XS). Got rid of that regex in a hurry.
Re:Here's my short list (Score:5, Interesting)
My supervisor is (rightfully) wary of regexes, due to their reputation for unreadability. My favorite solution (in Java): Make a bunch of string constants to match various elements, then assemble those strings together into more strings, which are finally assembled into a single real regex. Sure, it takes about 15 lines for a moderately-complicated regex, but readability is superior to using 15 lines of indexof and substr.
And now, to just bring up a topic that I have a personal interest in:
Nobody knows how to write parsers anymore. I've never seen a recent university CS curriculum that covers parsing with respect to different parser constructs for different languages. Sure, the students learn to load up the chosen XML parsing library and pull in XML, and they learn to take text from stdin, but there's seldom any emphasis on what to do with more screwy formats. Maybe, just maybe, they might get exposed to different languages through a compilers class, but generally not how to process different languages outside of lex/yacc. This bothers me greatly.
Re: (Score:3, Interesting)
Nobody knows how to write parsers anymore. I've never seen a recent university CS curriculum that covers parsing with respect to different parser constructs for different languages. Sure, the students learn to load up the chosen XML parsing library and pull in XML, and they learn to take text from stdin, but there's seldom any emphasis on what to do with more screwy formats. Maybe, just maybe, they might get exposed to different languages through a compilers class, but generally not how to process different languages outside of lex/yacc. This bothers me greatly.
Warning: Long rambling post follows. To summarize, I haven't found any courses that covered anything close to parsing, except maybe invoking XML parsing libraries and stuff.
Before I took any programming courses, I figured out how to create an XML parser in javascript. I had a dream of making a game, which needed server-side storage, but I never got around to finishing it. Quite an experiment, though.
After that I went on to help decode Outpost 2 [wikipedia.org]'s compiled map format. (maps were DLL files - oh the design ch
Re:Here's my short list (Score:5, Insightful)
You should think about investing in learning tools. Regex is a well documented, well understood, capable feature of all modern languages. How many weeks will you spend debugging / refining your thousands of if/then/else when you could trust years of testing that has been done on regex engines? Your statement reminds me of novices who avoid ?: in favor of if/else because it's 'cryptic'.
Re: (Score:3, Insightful)
Your statement reminds me of novices who avoid ?: in favor of if/else because it's 'cryptic'.
It's not about being cryptic. I use if/else and I've been cutting code for decades. if/else and ?/: do the same thing, while one is easier to read while you're scanning through lines of code (it's just English after all). That the novices can understand what's been written is a bonus.
Re:Here's my short list (Score:4, Insightful)
I've massaged if-else code into regexes before. Having several if-elses for a piece of data is rickety and (in my opinion) would take more work to rework than a regular expression if the data changed.
Re: (Score:2)
Bingo! You stole my thunder. Mod parent up Insightful. The solution to the infamous regex problem is to comment them. Your style guide should even have something like, "any regex longer than N characters should have a comment next to it". Developers who live, breathe and eat regex should be mindful that it might look like line noise to the rest of us, or that we might be able to parse it in 15 minutes whereas we can read your comment in 2 seconds.
Re: (Score:2)
What if there is a misplaced character in a 50 character (or more) regex? If it only broke in some obscure cases, even with a comment saying "this matches an email address", how the hell would you debug it?
Re: (Score:3, Insightful)
Re: (Score:3, Insightful)
You can step through a large block of code bit by bit with the debugger. You can't do that with a regex.
I think this post really illustrates the gulf between those who fear regex and those who do not.
A lot of coders today simply do not know how to code (or know how to and avoid doing so), they know how to set breakpoints in a debugger looking for large duplo-blocks of code to comment out or alter in predictable ways until the code outputs what they expect from it.
Now if we are lucky, if we are very lucky the coders who do this are forced to at least write regression tests for their changes. But at the foundat
Re: (Score:2)
Using a RegEx could simultaneously decrease development time and reduce the number of bugs - everything the code do is located on a couple of lines of codes.
Conversely, this might make it very hard/impossible to understand and change by someone that isn't a wizard in regular expressions (or might contain hard to identify bugs). Also, moving from ASCII to Unicode might be difficult.
Regular expressions and pattern matching (see Erlang) are very powerful technologies
Re:Here's my short list (Score:5, Funny)
This is why I am currently rewriting everything from scratch in .NET at my company.
Re: (Score:3, Informative)
That has to be the +5 funniest thing I've read in a month. Thank you, I just about fell out of my chair.
Re: (Score:2)
3. When it was written in a mid-90s WYSIWIG bastard child of a mid-80s interpreted language.
In what universe is a language developed in 1964 and first marketed by Microsoft in 1975 a "mid-80s interpreted language"?
Re: (Score:2)
WYSIWIG
For the noobs, thats What-You-See-Is-What-I-Got, normally after some software that was promised to deliver ABC gets XYZ instead, after paying M thousand dollars, and is in the process of being reported to upper Management.
Re: (Score:2)
6. When it was a product of outsourcing to a foreign country. God, maintaining that shit is the bane of my existence. At least they've upgraded to VB 2005.
Sometimes to move forward (Score:4, Interesting)
Re:Sometimes to move forward (Score:5, Interesting)
you must stop looking backward.
And loose all your users. They are in your 'Backwards" direction.
I have filed or triaged well over 1000 bugs for KDE since KDE 4 came out. Forget about the missing features from KDE 3 to KDE 4, they don't care about breaking compatibility from KDE 4.n to KDE 4.n+1. Even Kaddressbook, one of the core KDE apps, had severe functionality loss from KDE 4.3 to KDE 4.4. Amarok, Koffice, and other lose severe core functionality when their "new versions" were actually "technology previews" with dot-oh version numbers.
Re: (Score:2, Interesting)
The new Amarok just blows period. They could have actually done mockups and put it to a vote using a poll on their site, produced two or three demo UIs with only basic functionality implemented, put it to another vote and gone with that. In the end, the devs really just wanted to reinvent the UI because table views are so 2001.
The result is slow as a pig, ugly and more confusing then the old one.
Hell, I hate it so much that I now just use a Konsole instance running MPlayer instead. mplayer album/*/*.flac FT
Re: (Score:2, Insightful)
This comment is disturbing in many ways.
1) There was and is a KDE player with the table paradigm: juk. It works very well. In the 1.x days, I would use it for tagging, and amarok for the listening, now, I do everything in Amarok, because the tagging works now so much better than t used to, whether in the tagging dialogue or in the playlist.
2) Amarok was always about meta-data around the music: similar tracks/recommended artists, WP articles, lyrics, moodbars and whatnot. In the 1.x days it was so, in the 2.
Re: (Score:3, Insightful)
Basically, I am questioning the common wisdom the KDE4 and Amarok 2 are failures. I question them mostly because whatever reproach there might be usually is stale. Stale as in: "this was fixed a year ago, now".
But hey, questioning common wisdom on slashdot is good for burning karma. Remember kids: thrashing free projects based on false impressions you got a year ago is fine. Defending them with actual evidence goes against truthiness, and we can't let a fact-based discussion take place.
But this is something
Ugh (Score:5, Interesting)
Drupal does indeed brutalize your database (see second link). So looking forward to D7 to clean this up. That alone was sufficient justification to rewrite the application :p
Unfortunately the author goes on to display his ignorance before this is all over: There are also other examples of total rewrites that have been successful. Apple's transition to OS X is a very good example. The classic Mac OS was an operating system with some good ideas, but the core architecture had a lot of problems. Did Apple make a bad decision to start from scratch with OS X? I don't think so. They brought over and improved the best bits from OS 9, and merged it with the solid Darwin core (much like we did). Uh, no, you are totally and completely fucked here. They started with NeXTStep, last updated in 1995; it was itself based on an older version of BSD (4.3?) and an older version of Mach. OSX is not a complete rewrite of anything. Its legacy stretches back into the 1980s, and so does its code.
Re: (Score:2)
Re: (Score:3, Insightful)
The original rewrite of MacOS was Copland, which used up Apples development resources for five years before it was cancelled.
http://en.wikipedia.org/wiki/Copland_(operating_system) [wikipedia.org]
Re:Ugh (Score:4, Insightful)
The original rewrite of MacOS was Copland, which used up Apples development resources for five years before it was cancelled.
Yeah, I remember that, it's what convinced me to stop being a Mac user. I had the INIT or CDEV or whatever that made your MacOS look like Copland. Then they cancelled it and brought out another shitty MacOS, and I ran like hell and never looked back. All the years from System 7 through System 9 were sad, abusive times to be an Apple customer. They kept bringing out inadequate hardware at astounding prices and almost spent all their cachet. Another year or two and they wouldn't have had enough fans left to float OSX.
Re: (Score:2)
Not to mention:
They brought over and improved the best bits from OS 9,
Apple *threw out* the best bits of OS 9. Right into a dumpster. Oh, they half-heartedly pretended to care for a short while (re-adding some features like Labels in Finder), but it was obvious they did not. They certainly didn't "improve" Labels when they *finally* got around to porting it. And many OS 9 Finder features they never bothered to port in the first place.
Not to mention abandoning the entire philosophy OS 9 was based around-- a
Re:Ugh (Score:5, Interesting)
But Apple did rebuild their GUI shell, application APIs, etc. from scratch. Sure the underlying OS is based on legacy code stretching back into, really, the 1960s (BSD itself was, after all, based on AT&T's Unix), but the GUI and application APIs were totally new.
That is totally and completely false [wikipedia.org]. "Cocoa [wikipedia.org] is one of Apple Inc.'s native object-oriented application program environments for the Mac OS X operating system. It is one of five major APIs available for Mac OS X; the others are Carbon, POSIX (for the BSD environment), X11 and Java. [...] Cocoa is the continuation of several frameworks (primarily the App Kit and Foundation Kit) from the NeXTSTEP and OPENSTEP programming environments developed by NeXT in the 1980s and 1990s." "Carbon descends from the Toolbox [wikipedia.org], and as such, is composed of "Managers". Each Manager is a functionally-related API, defining sets of data structures and functions to manipulate them. Managers are often interdependent or layered.
Newer parts of Carbon tend to be much more object-oriented in their conception, most of them based on Core Foundation. Some Managers, such as the HIView Manager (a superset of the Control Manager), are implemented in C++, but Carbon remains a C API." And I should HARDLY need to touch on POSIX, X11, or Java, the former two of which were already functions of NeXTStep, and the latter of which was not invented at Apple.
There is NOTHING in OSX which can reasonably be believed to have been invented (in any sense of the word) for OSX! Period, the end. ALL portions of OSX contain legacy code, from the microkernel to the presentation layer. They didn't go from Display Postscript to Display PDF because it was easy, they did it because they couldn't just throw over Display Postscript and be able to utilize existing GUI code. They didn't throw out the GUI and replace it, they replaced the engine under it, then made it work on that engine. The appearance of the GUI is still customizable, as it was under NeXTStep, by mangling the images it's made of.
Re: (Score:3, Insightful)
When I cook biscuits from scratch, I don't start with baking mix, or leftovers. I start with flour, and eggs, and water, and milk, and baking powder,
All of which are components you didn't create yourself. You didn't, for instance start out with flour seeds, chickens, cows, hydrogen, oxygen, an acre of farmland and whatever you'd need to create baking powder.
The point of my analogy is of course that much like in cooking, everyone in software development starts with some ready made component. Is taking Next
There are actually a few good reasons (Score:5, Interesting)
The problem is that companies usually rewrite for all the wrong reasons.
A good reason would be the emerge of a new technology that supports your problem much better, to the point where redoing your code from scratch means easier maintainance later. Usually this goes hand in hand with an old technology (the one you used so far) getting abandoned by its maker. A good example would be how I had to maintain a client/server app written in VB6 using DCOM. Not some quick hack, a full blown client/server solution it was never meant to be, that also has to communicate with a SAP interface and a few more nifty things that cried out "please rewrite me". The overhead to maintain it soon turned from insane to nuts and even the tinyest change required a team of four people to work for weeks.
Unfortunately, the reasons why something gets rewritten are usually different. Like at my next gig. A pretty well designed and fairly solid piece of code was thrown out when the original creator was fired and someone with an ego that required its own office took over and insisted we use "his" collection of libraries instead of that "old" libraries. We trashed about 2 manyears of labour down the drain to end up with what we had before.
Re: (Score:3, Interesting)
Often the problem is that there are things that should be rewritten (either whole apps or specific sections of code) but no-one is willing to green-light such a rewrite even though the rewrite will take less time than bolting hack after hack on top of the old system.
Re: (Score:2)
Often the problem is we developers are wildly optimistic in our estimates of how long such a venture might actually take. Especially if we are talking about code we dont know much about. Until we really get to know the problem domain, it all sounds so easy :-)
In my opinion, what Joel really is against is a wholesale "drop everything you are working on and rewrite the entire app". From hard-won experience, he
Re:There are actually a few good reasons (Score:4, Informative)
I quite like the way that the article claims that it shows an exception to Spolsky's rule, but actually isn't at all: they claim to have started off as a successful CMS company with "big name" clients, embarked on a rewrite that took them off the market for two years and ended up as a tiny player with "more than two hundred web projects [...] in Norway".
As far as I can tell this is a project that went exactly the way Spolsky predicted: they had a decent product, they embarked on a rewrite that took longer than they expected and they lost the market by doing it.
Re: (Score:2, Informative)
There are times for a rewrite and port. (Score:4, Insightful)
I've seen rewrites/ports go quite well. Systems that were originally on mainframes and needed or wanted to be moved to cheaper hardware for cost - if it was the proper thing to do (Sometimes you really need the metal).
Another rewrite that went well was a bunch of code that over the decades became so convoluted to be a maintenance nightmare - modify one thing or add on functionality and then break a shit load of other things.
Just do these basica things and it'll work out.
Go back to the specs and start there.
Talk to the stake holders - yep, there will be creep but also feature reduction because there are things that they never used or because it doesn't make sense anymore.
Plan, plan, plan. No cowboy programming and hacking out shit. And document everything.
It can work.
Re: (Score:2)
Just do these basica [wikipedia.org] things and it'll work out.
Fixed that for you.
The exceptions Joel should have included (Score:5, Interesting)
Re: (Score:2, Insightful)
The other problem is with scale.
When a company starts a new project or an entrepreneur starts a small business it's sensible to build something quite small and straightforward. It might take off, but it might not, so don't spend more on the tech than you really need.
So, something gets produced that's VERY hacky. And for a short term solution and from a business perspective, that's quite bright.
The problem is that scale it up a little and it starts falling apart. It becomes hard to maintain because the funda
Only 80000 lines... (Score:2)
... and most of them run in a controlled environment (server-side). So lots of Joel's advice is not going to apply.
Then they have the balls of comparing their move to the transition from Mac OS 9 to Mac OS X!
In the end, while their story is interesting, it adds really little value to the "Rewrite/No Rewrite" debate.
Simple rule... (Score:2)
rewrite ONLY if it means it yields less lines of code to maintain with the same functionality. Use whatever language necessary to keep the length of code to a minimum.
If the code doesn;t look as if it has been refactored even once... junk it.
Re: (Score:2)
rewrite ONLY if it means it yields less lines of code to maintain with the same functionality.
I have to register a huge protest to this idea that "less lines of code" == "more maintainable". Perhaps on a grand scale across several orders of magnitude this is true, but on the small scale of say a factor of 2 or 3 it's most definitely false.
I don't know that there's any one single simple rule that correlates well with maintainability. Maintainability has something to do with complexity, and quality which a
One typical problem (Score:2)
Rewriting an application can work if the developers know what they're doing. However, these rewrites are often side projects. If considerably more resources are spent to extend the original application, the rewrite will be behind until it gets cancelled. On the other hand, a rewrite is always a risk and betting your company on it is insane. In most cases it's safer to refactor the original implementation, even if it's more work than starting from scratch.
Here's a good example (Score:2)
When your "enterprise" ERP application is written in VB6 with a roll your own alternative to ADO, half your business logic is in stored procedures that can be 5,000 lines long and 30% to 40% of your code was written to spec B.
5 points if you can name that product
Re: (Score:2)
Don't pay so much attention to Joel Spolsky. (Score:5, Interesting)
Look. Spolsky runs a dinky little software development firm that sells a little project management program. And it's still a dinky little software development firm after a decade. It's not like this guy runs a company that does anything critical, like avionics software, or anything really big and tightly integrated like Facebook, or financially significant like Chase's banking system, or leading-edge robotics like Boston Dynamics, or cutting-edge manufacturing like HyperMill. No, they just do Windows and Mac desktop apps. That's trailing edge technology at this point.
Some of the better shops don't hesitate to rewrite. Some have systems which intercommunicate by message passing and are designed to be rewritten in sections. (Facebook, works that way internally.) The bigger shops may have one team on the new version and a maintenance team on the old one; a small firm may not be staffed for that.
Re:Don't pay so much attention to Joel Spolsky. (Score:4, Interesting)
Yes, I never understood why so many pay attention to Joel's inflammatory rants.
I don't even want to start on his company's product (Fogbugz). Seriously, ASP/VBScript translated to PHP? And then inventing a new programming language just for a web app with ability to output in several other languages? Ugh.
Re: (Score:3, Insightful)
Yes, I never understood why so many pay attention to Joel's inflammatory rants.
Partly because he's a fairly good writer, and partly because he seems to offer simple solutions to the larger problems of software, and partly because he's right at least some of the time.
I DO think he's just dead wrong on this issue though. In my experience starting from scratch is something that should be very carefully considered, but rejecting it outright on general principles is wrong. I really don't care much about his bo
Re: (Score:2)
Not too familiar with Bernstein
Bernstein is a bit of a blow-hard that thinks the major component of software has to be is SECURE. He wrote some small securely designed applications like DNS, and then goes on to challenge everyone else about how THEIR product isn't secure while HIS is. He likes to ignore the fact that real life often intervenes, requiring software to be more complex than you'd really like it to be.
I've never heard of the 37 signals people. I've used basecamp though. It's OK, but it seems
Re:Don't pay so much attention to Joel Spolsky. (Score:4, Informative)
Dinky company, perhaps, but quite successful at a personal level. In less than ten years, Joel took his company from zero to seven million dollars per year by my accounting.
http://joelonsoftware.com/articles/BionicOffice.html
$700 per employee in the original office
http://joelonsoftware.com/items/2008/12/29.html [joelonsoftware.com]
"built for 18 employees" = about $12,600/month
http://www.inc.com/magazine/20080601/how-hard-could-it-be-adventures-in-office-space.html [inc.com]
"When we moved into our current offices, our rent had been equal to 15 percent of revenue, which was high. But the company grew, and today our rent is only about 2 percent of revenue."
So revenue was $84,000/mo ($1,008,000/yr) and is now about $7,500,000/year.
So he's not a complete waste of space. And he may not be God but that doesn't mean he's never right and/or never worth listening to. READ THE F ARTICLE about rewrites--plenty of Slashdotters (you included) have been here long enough to know that at least, his example about Netscape/Mozilla is 100% accurate. They lost YEARS because they chose to rewrite everything.
And judging by the comments here, I think a lot of people are reading the title and thinking he's saying "never make any changes." That is 10000% NOT what he is saying. He's saying "never throw away 100% of your code and start over from scratch." If you actually read his original article (I know, I'm new here) you'll see a lot of really good points.
Joel isn't God, but he isn't just some stumbling moron either. There IS a continuum between those two extremes, you know.
Similar Experience (Score:2)
We faced the same challenge at our company recently.
My company publishes a CMS for specialized niches.
The product started around 2002, and we developed it without real direction, since we added functionalities as we needed them.
The first years were great, since we were the only ones to release such a tool, but some concurrents emerged after a few years, proposing a much cheaper alternative.
As we built the project without long-term view, adding new functionalities took more and more time, since the codebase
Re: (Score:2)
When you release it you will find the following things guaranteed
- Customers have forgotten to mention 20% of the scenarios they need which is present in your current release.
- 20% more have been misunderstood [jacobsen.no] by you.
- Your new
Old crappy code (Score:2)
That old crappy code has undecipherable bug fixes for 100's of obscure difficult to reproduce bugs. Many of them which could be hit only under really strange customer usage scenarios & which tooks days to reproduce, days to fix & many more days to fix whatever the fixes broke.
So unless you original software fully automated testsuites covering every functionality & you have added a bug fix verification test for each bug which has been fixed since then, do not rewrite something from scratch.
Re: (Score:2)
That old crappy code has undecipherable bug fixes for 100's of obscure difficult to reproduce bugs. Many of them which could be hit only under really strange customer usage scenarios & which tooks days to reproduce, days to fix & many more days to fix whatever the fixes broke.
Sounds like code that was just poorly written in the first place, and then endlessly patched and re-patched to try to "get it right this time".
Listen, software development is about software developers and the development proces
Re: (Score:2)
Do you have experience rewriting huge enterprise software which started small, got lot of features added over the years & now has 5-10 teams of developers working fulltime on it? This kind of software is a disaster to rewrite. Yeah, if you want to rewrite something mid-size like a browser, it would probably not be as bad.
If you want rewrite huge software, do it in small chunks across different releases after adding a lot of automated test suites & bug fix verification tests.
when rewriting, go wider, not narrower (Score:2)
Re: (Score:2)
DotNET was they're ONLY option... never even considered that it was, never considered anything else... never considered. If a mistake was made, that was it... and again, it may not make any difference because the product may still be successful even with lock in. But there is no doubt they're
*their *their man I am sloppy today
Rewrite if existing s/w congeals (Score:3, Insightful)
Often a rushed, under-resourced or under-skilled s/w project will congeal into a large, brittle, solid clot,
which is not extendable without breaking things in mysterious prohibitive to fix ways.
Congealment comes from insufficient or ill-conceived architecture, and/or rushed or ignorant ill-fitting extensions or mods or "fixes",
combined with insufficient continuous re-factoring.
This code may be worth keeping on expensive life-support if there are many existing customers depending on it,
but make no mistake. Your codebase is already dead, even if its heart still beats.
So then, if you still need software with similar but slightly updated or extended functionality, you should rewrite,
and in doing so, make sure you get good architecture, take sufficient time to build each part or layer, evaluate the quality of
all third party libraries or frameworks used (on the "volleyball" principle that the weakest member of the team drops the ball
and determines the team's i.e. the system's quality), use continuous refactoring, with technical-risk based work prioritization
(biggest risks dealt with first, always), document the classes and methods
sufficiently, and include unit tests and/or invariants and pre-postconditions, so that there is a lower probability that
further extensions will start congealing into brittle, excess complexity.
If you can succeed at maintaining that discipline without going bankrupt, then it will have been worth it, because the value
of your new software capital asset will be much greater than previously.
Of course you should have done it right the first time, (and should have had management enlightened enough to let you,)
because it would have been much cheaper to do it carefully once, than the punishing expense of the original crappy
development and maintenance plus the rewrite. There IS a valid argument that by the time you let your s/w congeal into
a complex, brittle clot, you are already too late, and you should pull the plug, shed a tear, and walk away.
This is a very simple question (Score:2, Insightful)
Whether to do The Big Rewrite always boils down to one very simple question: do the expected gains outweigh the expected losses?
Usually, the argument against doing a rewrite boils down to two key points:
Those are certainly valid concerns, and IME it is often true t
Rewriting works sometimes, without a doubt (Score:2, Informative)
I worked for a company that rewrote the same application three times in different technology. 2 time using MS .net tech - aspx, .net desktop, and 3rd with PHP.
The first incarnation was a disaster from a performance/scalability point of view, but we did learn the (surprisingly complex) business requirements very well.
The second incarnation was good, developed quickly, but missed the target market -- nobody wanted a desktop app
The third performed much better than either predecessor, was simpler to maintain, a
Joel contradicts the IEEE (Score:3, Informative)
Joel's position contradicts a paper I read years ago in an IEEE software journal that basically said you needed to plan on rewriting your application about every 7 years or have it collapse on you. The logic in the paper was based on two things I've found to be true in the real world. First, the world changes. Individually it's small changes, but looking at it on the half-decade-to-decade scale it can add up to huge differences in what's needed in the software. Second, software isn't infinitely extensible/adaptable. Any software has a basic architecture and world-view, and a limit beyond which it can't be pushed without an exponential increase in the time and effort needed to successfully make the changes. The two combine to mean that at some point it simply becomes technically infeasible to extend and adapt an existing system. The requirements have changed too much and you're having to fight the system trying to make it do, not just what it wasn't designed to do, but what it was actively designed not to do.
Now, business doesn't like this. It doesn't make sense from a business perspective, and it'd be much better to simply keep adapting and extending what's already there. But that ignores the fact that something must be technically feasible before you can even ask whether it makes business sense. If you've got the best idea in the world that'll make the business tons of money while giving you a virtual monopoly in the field and reducing costs by 99%, that basically is from a business standpoint the absolutely ideal thing to do, but it requires the manufacture of say room-temperature superconducting wire by the mile, then it just ain't gonna happen. How desirable it is from a business perspective doesn't matter because it just isn't technically possible at this point in time.
I also liken it to building a 20-story office tower. It's tempting to start with a simple one-story building and slowing add to it until you've got what you want, but the foundation of a one-story building just isn't going to be able to support a 20-story tower. You might be able to get 2 or 3 stories out of it, but at some point you're going to have to tear the whole building down and re-do the very foundations themselves to support the greater weight.
Re: (Score:2)
Re: (Score:2, Informative)
I'm the author of the blog post. The Joel article is 10 years old, but not the one I wrote(the the second link).
Vidar Langberget
Re:And why? (Score:4, Insightful)
We do some small custom web apps for clients, and even a few that have the potential to grow into bigger (but not big) products. While rolling out one of them this week, to the first of three clients who have ordered it, I'm already designing a complete re-write of the core of this product. What started out as a small helper app for one client has turned into a PITA to scale up and out for these other clients. Clearly it's better to fix it now than to keep patching & splicing in order to make it work for them
Re: (Score:2)
Einstein said something about making the same mistake and expecting different results ...
No he didn't. That was Rita Mae Brown in her book Sudden Death [wikiquote.org], but that never sounds as dramatic or important as claiming it was Einstein.
Re: (Score:2)
Don't worry about refuting a zealot, the two technologies have pretty much nothing in common anyway.
Not that I'm a fan of either approach, exactly, but this was just a cheap jab at Microsoft based on prejudice. You should be expecting that around here by now, your user id is low enough.
Re: (Score:2)
Don't worry about refuting a zealot, the two technologies have pretty much nothing in common anyway.
I don't think the difference in technologies is actually relevant to his point. If company X is known to make unreliable products for reasons relating to corporate culture, then "use tech A from company X" and "use tech B from company X" are essentially the same mistake even if tech A and tech B are automobiles and recombinant gene therapies. I think that was more of his point. Having used Visual Studio fairly extensively (in the pre-.Net days) I'm also inclined not to use ASP or .Net based on my experie
Re: (Score:2)
but this was just a cheap jab at Microsoft based on prejudice.
Prejudice is judgment without facts. I'd say this was more of a cheap jab based on decades of experience.
Re: (Score:3, Informative)
I am about half way through the article in the second link, and it is really interesting, and informative. :>
Maybe not news, but it is worth your time (or at least mine).
6 KB wasted on fucking VIEWSTATE data. (Score:2, Informative)
I just looked at their article, and 6 KB of the page was near-useless VIEWSTATE data. If they can, they really should disable the generation of that. It's a useless artifact of the broken ASP.NET WebForms approach, which isn't really even necessary for a blog like theirs.
Seriously, with a typical Slashdot posting resulting in 80,000 unique hits for the target site, they're going to waste over 480 MB of bandwidth serving up just that useless VIEWSTATE data.
Re:6 KB wasted on fucking VIEWSTATE data. (Score:4, Informative)
The blog module uses templated controls. When you bind data to a templated control, all that data get's stuffed into the viewstate as default. To fetch the comment content, the template for a comment has this snippet:
<%# Container.CommentContent %>
As default in
<asp:Literal runat="server" ID="litBlogCommentContent" EnableViewState="false" Text=""></asp:Literal>
I usually check the size of the viewstate, but none of the blogposts I checked had any comments, so I didn't catch it when we created the site. I'll update it as soon as traffic slows down a bit.
Re: (Score:2)
Re:6 KB wasted on fucking VIEWSTATE data. (Score:5, Funny)
If they can, they really should disable the generation of that. It's a useless artifact of the broken ASP.NET WebForms approach, which isn't really even necessary for a blog like theirs.
Are you suggesting a rewrite?
Re:And why? (Score:5, Interesting)
I have been through a similar project - rewriting a solution that did run under OpenVMS using Basic, Java, C++, C and a bunch of DCL scripts (that confusingly enough for DOS persons have the file extension .COM)
Target environment was Linux and language used was Java 1.6.
My experience when rewriting a legacy system that have a crapload of varying solutions that has evolved during 25 years or so you will find that there is always yet another functionality that nobody told you about - effectively doubling the development time. (This "Multiply estimated time by PI factor" statement isn't that far off.)
And there were some traps involved too - migration of the system had to be seamless for the users as much as possible and with minimal downtime. Since there were over 400 different customers with everything from 1 to 1000 users each involved this was to say the least "tricky". Especially since this was a 24x7 system. The solution was to write a replication protocol that replicated data between the old system and the new. The old system used OpenVMS indexed files while the new system runs a MySQL database and the data structures were different too, which made it necessary to write a replication solution. So when a customer was migrated it was effectively done by setting a flag that redirected them from the old system to the new system and they could continue working.
Of course there were bugs in the beginning, and user errors since the new system did have different functionality and behavior compared to the old. Bot none of them were causing any irrecoverable problems. Invoice printing was delayed, but no major amount of money was lost. The majority of the problems appearing didn't affect the end users at the customers, only the helpdesk service personnel and they were prepared for limitations ahead of time.
The amount of downtime for the system during the two years it has been operational has been very low. And this has given a different concern - too few "problems" with a system is also a problem because tech support will almost forget that it exists.
Specific problems with the application - especially in the beginning has been running out of PermGen space in Java. This at least partly due to design mistakes. But memory leaks that grows over time are very low. And the use of FindBugs [sourceforge.net] has been very useful to trap a lot of errors (potential and real).
What the application does? - It's a management application for short-term lease of telephony at hospitals and similar (almost 400) and other services (a few) which enables and disables phone extensions, assigns numbers, allows instant move of an extension and provides invoicing for the rental and phone usage through processing of CDR:s.
Re:Missing the point (Score:5, Informative)
Actually, from what I got from the article it seems they also felt that the basic design of the original version of application just wasn't good enough, that it was in fact seriously lacking and that a gradual rewrite would take longer and not accomplish what they wanted (to clean up and future-proof their application).
Re: (Score:2)
Besides, it's not like a clean-room implementation when you do a rewrite.
Sure, the architecture and language may be different, but a lot of the old code would likely still be used as "inspiration".
Re:Missing the point (Score:5, Interesting)
Well here's a story from the stone knives and bear claws-era (early 80s):
Two programmers were tasked to convert the Atart VCS/2600 game Pitfall 2 to a Commodore=64 and Atari 800 computer. One said, "The Atari console is so primitive that it's easier to recreate the whole game from scratch," and the other said, "No just copy the 6502 code and then modify it for the varying graphics/sound chips." They then went their separate ways.
- The Commodore=64 programmer recreated the whole game from scratch, and produced a slightly-flawed but decent port.
- The Atari 800 programmer simply dumped the code directly, and then modified it. He produced a port that played identical to the original PLUS he had enough time left-over that he added a whole other game (basically Pitfall 3). So Atari 800 purchasers got two games for the price of one.
Reworking is faster than starting over. Even if the design is a complete mess, there's typically SOME modules that can be reused, and that's time saved
Re: (Score:3, Insightful)
You're assuming that the original design isn't an organically grown mess of code that's grown and mutated over the last 10+ years (this is pretty common on the business world), preferably written in some proprietary and deprecated language (ASP + VBScript is a current classic, most likely pushed to the company as a good "business language" by some MS sales drone). After ten years of "organic" growth of such apps cleaning them up generally takes longer than just rebuilding from scratch in a sane language wit
Re:Missing the point (Score:5, Insightful)
Whenever I hit my thumb, I blame the stupid hammer as well.
Or, in other words: a fool with a tool is still a fool.
You can use assembly and have decent code, with clear separation of concerns. Or you can have a 4GL programming tool and still make a mess. Which is exactly why some programmers are 10 times more productive than others.
So where I worked we had ASP+VBScript (supplemented with VB6 COM+ modules running with transactional integrity on an Oracle database) and clean modules, separation of concerns and code that we could easily understand and maintain (even the junior programmers had no trouble getting used to it in a few weeks). We built most of the business apps in the last place I worked on such a design. It still works, is very easy to maintain and transfers cleanly to IIS 8 and Windows 2008. All our database code is in a single (small) module, same as the business layers. Presentation layers is a bit more complex but when transferred to .NET you can just get rid of it altogether because .NET takes over that part. Which is exactly what is happening now, ofcourse.
Don't blame the tools for the lack of ability of most programmers.
Re: (Score:2)
I didn't blame the tools and since asking you to re-read my post will probably do nothing I'll try to explain it.
Certain tools are more dangerous than others. Few people can cause large-scale damage with a screwdriver but most people can do serious damage with a few sticks of dynamite even though both can be equally useful for different purposes. A bit extreme perhaps but the point is that VBScript is not a very good language for apps that you wish to keep maintaining and adding features to, add into this t
Porting games written in assembly (Score:2)
Two programmers were tasked to convert the Atart VCS/2600 game Pitfall 2 to a Commodore=64 and Atari 800 computer. One said, "The Atari console is so primitive that it's easier to recreate the whole game from scratch," and the other said, "No just copy the 6502 code and then modify it for the varying graphics/sound chips."
But then how would one have ported a 2600 game to the ZX Spectrum, ColecoVision, MSX, or Sega Master System, all of which use a Zilog clone of Intel's 8080 CPU?
Re: (Score:3)
(Japanese beat-em-ups for example are notorious for vast wodges of copy-and-pasted assembly code that is hard to understand if you can't follow the Japanese comments).
Anyway, the kernel of a 2600 game (which is half the code) is completely unlike a C64 game engine. The game logic would have been easy to port so it would make sense to keep that. And I would hope the graphics and sound were much
Re: (Score:2)
>>>But "dumping" the code (i.e. reverse engineering a 2600 game from just the ROM)
Why on earth would the two Activision programmers need to do that? They could just walk down the hall to David Crane's office and say, "Hey the boss wants us to port Pitfall 2 to the Atari and Commodore computers. Can we have your 2600 source and notes? Thanks." Simple.
As for the C64:
It doesn't play "right". I first played the original game on the Atari console, and later the C64 game. The character doesn't act
Re: (Score:2)
*time already invested in internal projects that will be used in the rewrite version (e.g. experience writing the NeXT OS greatly reduced the time it took to write MacOS X 10.0).
As I covered elsewhere [slashdot.org], OSX is just the latest revision of NeXTStep. I did make ONE error in that comment: there is one thing in OSX that didn't come with NeXTStep or come from legacy MacOS (like the toolbox code) in some form: Grand Central Dispatch. That's not a central API, it's totally optional, but it is an exception to what I said, so it's fair to mention it. It's not the experience that made NeXTStep [apple.com] 3.4, er I mean OSX 10.1 so quick/easy to release, it's the fact that it's made of more legacy code t
Re: (Score:2)
Re: (Score:2)
Re:Why I want to rewrite (Score:5, Interesting)
I want to rewrite my old code at work... But only for one reason: I am a lot better programmer now than 5 years ago. And 5 years ago, I was a lot better than 10 years ago. And in 5 more years, I have no doubt I'll feel the same way.
There's actually one other reason most programmers would like to rewrite what they're working on: They've solved the problem once, and now they understand it.
IMHO, you can't solve a problem properly without solving it twice. Unfortunately, that's just not, in general, tenable in the industry, and so instead we have things like XP, which encourage prototyping and refactoring, which accept that maxim and attempt to allow for it in the process. Unfortunately, *that* requires preeminent design skills, and that's something lacking in your average developer.
Re: (Score:2)
But only for one reason: I am a lot better programmer now than 5 years ago. And 5 years ago, I was a lot better than 10 years ago. And in 5 more years, I have no doubt I'll feel the same way.
These strike me as not terrible reasons to re-write your code, but also not particularly good ones either. It seems more ego driven than an economic one.
In the end, the major expense of software development is time. Time to fix bugs in the code, time to add new features to the code, etc. It doesn't matter if its OSS
Re: (Score:2)
I disagree.
I want to rewrite my old code at work... But only for one reason: I am a lot better programmer now than 5 years ago. And 5 years ago, I was a lot better than 10 years ago. And in 5 more years, I have no doubt I'll feel the same way.
Code I write today is cleaner, easier to read, more efficient, and easier to work with for new projects.
And what if your company started a re-write, and you left 2 weeks in? What would happen then?
(Or are you a one-man shop, in which case pretty much nothing in this th
Re: (Score:2)
Mostly agreed, and I'll sign my name to it.
Spolsky's got some decent ideas, and knows a lot about what he talks about, but his idea of "good" and "bad" makes me cringe. He's one of those folks who's built up a following on what little they had to talk about, then ceased to think further. Now (and for several years), he writes about half-baked lists of do's-and-don'ts that never even come close to covering exceptions.
Re: (Score:2)
Re: (Score:2)
Real artists ship.