Slashdot Log In
Interview With Martin Fowler
Posted by
timothy
on Thu Nov 28, 2002 04:41 PM
from the fowling-piece dept.
from the fowling-piece dept.
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Refactoring. (Score:4, Funny)
hint...
hint...
Developers love him; Managers hate him (Score:5, Interesting)
Re:Developers love him; Managers hate him (Score:5, Informative)
Refactoring isn't about making your project buzzword compliant or supporting distributed OLE-foo++. That's adding new features. From the article:
Refactoring is usually saying "hey, I implemented that function the wrong way so I'm going to rewrite it properly". The right way of doing something is often obvious after coding it up the wrong way.Re:Developers love him; Managers hate him (Score:5, Insightful)
Refactoring is also not just something you do (or not) after you've finished (i.e. shipped / deployed). It's something you do as you go along, during the process of building the thing. If you need to refactor substantially after you've finished, then it may be that you didn't refactor enough as you went along.
Refactoring is a fancy word for what many good programmers who are not geniuses do already, a few good programmers who are geniuses don't need to do, and a lot of bad programmers who may or may not be geniuses don't do enough. Speaking as an OK programmer who is not a genius, I feel I need all the help I can get, be it from test-infection or from taking a considered approach to cleaning up after myself.
Periodic refactoring helps me keep abreast of shifting requirements; it isn't about prettying-up something that's about to become obsolete anyhow, but about keeping a check on creeping duplication and tacit dependencies so that the code can absorb new requirements instead of being broken by them.
Re:Developers love him; Managers hate him (Score:5, Interesting)
Refactoring is often about injecting the last good idea you had into working code.
I've got to disagree with you on this one. Refactoring isn't about injecting good ideas into working code, it's about making sure that those new requirements you got last week didn't create spaghetti code. In a development world were you design from a static set of requirements, Refactoring really doesn't have much of a place. In that world, you have a valid point: if a developer can't do it right the firt time, round two won't be much better.
However, that's not the world I live in, and it's not the world that anybody I know lives in. Then again, I don't have friends at NASA. We live in a more realistic development world that changes reqirements after we've designed our framework. We live in a development world where the evolving system makes the customer reconsider their original decisions. Since you didn't design the system with these new requirements in mind, you're usually patching functionality on top of what you got.
Refactoring is about reviewing this patch-work to make sure that the code is written as though it were designed that way from the beginning.
Certain agile methodologies (especially XP) treat refactoring as a tenet. Customers never give you a complete spec, and you never want one. They give you "stories" that you implement quickly and minimally. You don't over-engineer to support infinite extensibility via Web Services and a plugin architecture (etc,) unless the story asks for it.
When your latest minimal implementation starts looking ugly or hacky (ie: 4 page switch statement) then you refactor until the code properly "expresses its intention."
You had some good points, but I think you are a little mistaken about when refactoring comes into play.
Re:Refactoring may very well be a waste of time (Score:5, Insightful)
Don't read this story (Score:1, Offtopic)
Telling Point! (Score:4, Insightful)
Just my $.02
Missing some crucial bits there... (Score:5, Insightful)
What bothers me, is that the process being plugged here doesn't address methods for keeping all of the expressions of the program consistent. Tests are a (limited) alternate expression of the program's functionality that are trivial to compare to the original. Yes, they need to be good. What about documentation (which I define separately from annotated indexing, such as doxygen or Javadoc produce), which is another, separate expression (usually not in an executable language) of the program?
Until these authors address how to keep the entire structure from docs down to code consistent during a refactoring I think they are missing an important point. I've pointed this way before on Slashdot, but the only good way I've found for ensuring that non-executable forms of the program are consistent with executable forms, is formal Software Inspection (see Gilb's stuff at Result Planning [result-planning.com] ). I've found that the more versions of the code there are (requirements, user docs, code itself, tests etc.) that are kept truly consistent, the more likely it is you will not make mistakes in the final expression (the code you ship).
The refactoring process can be even less "like walking a tightrope without a net"; you're net is a web built out of the relationship between all of these documents, not just the code and tests!
Re:Missing some crucial bits there... (Score:4, Insightful)
I've found that the fewer versions of the code there are at all, the fewer cock-ups happen.
Most documentation that gets printed out is a waste of time. Make the damn code its own documentation, at least as far as implementation details and such. Keep the printed stuff to be higher level overviews of the design -- the big picture -- and for describing the key algorithms and concepts. This way, others on the team or new joiners have somewhere to start and a frame of reference when they check the details (which are described by comments in the code, or indeed the code itself) but you don't waste massive amounts of time writing a silly document that just describes what every method of every class does. We have tools to do that for us; they're called comments. :-)
(By the way, notice that simple refactoring rarely changes any of these higher-level overviews. If you're doing something significant enough that it does, it probably justifies a quick review of the overall design and a suitable update to the document anyway.)
Aside from higher level design docs and feature specs, about the only non-code documents that are often justified are requirements specs and acceptance test specs, which should usually be written in the language of your application domain and not in software terms anyway, and which should be completely independent of the code. On some projects, other forms of testing might usefully be documented as a kind of checklist, but often automated unit tests remove most of the need for this.
So, there you go: I have exactly one version of the code, and it's the code. There might be high-level summaries of features, or some details of the algorithms the code implements, documented as well, but they are somewhat independent. Everything else is for the customer, and totally independent of the code. If you've never tried working this way, and come from a "heavyweight process" environment, try it some time. I'd bet good money you'd prefer it. :-)
Refactoring is a waste of time (Score:3, Insightful)
Your code will be needed for a long time
Your requirements will not change drastically
Even if they don't state these explicitly, it is asusmed in the premise of refactoring. If code did not have a long productive life ahead of it, you wouldn't waste your time continually replumbing it. If requirements did in fact change rapidly, there would not be time to revisit completed working code.
In my experience most code lives for about four years and then dies. During that time the requirements often shift by at least 25%. Given these observations, refactoring appears to be a waste of time.
Of course requirements change. (Score:4, Insightful)
You have to Embrace Change [c2.com], and in that refactoring will really, really help you a lot.
If requirements change so much it is not the same program anymore, well, then I'd not say that requirements have changed in the project. It is a whole new project, right? And then, of course, no rewrite will help you.
But if they change a lot within the same functionality, you use refactoring to get to the new goal without breaking anything, and because you have been refactoring out good interfaces as you went the changes are easier to implement. You do not code for two months, then refactor for two, the code etc. You do both all the time.
I think you are missing the point here.
Funny to see old terminology being the 'new' thing (Score:5, Interesting)
Heavy reading (Score:2, Offtopic)
Read More... | 15 of 20 comments | Developers
That must be the lowest amount of comments to a frontpage story for a long time. Anyway, I think the UML distilled is one of the best organized books I have read. Works for a novice as well as someone who only knows something. Not my favourite though, but a piece of art in clarity :)
Oh, THAT Martin Fowler. (Score:3, Funny)
Automated Testing (Score:4, Insightful)
Often, the only real way to get good data for your tests is to have the software used in the field, and then use some of the more complex cases to test against. Corner cases are also a problem, esp. if you are relying on your tests to be comprehensive, and verify the correctness of your code. Tests are certainly valuable, but are by no stretch infallible. I've found that you don't get any really useful data until around the second revision of the code (assuming fairly wide use and access to end-user data). Sure, running tests against some custom widget might be pretty reliable, but once you run up against stuff that is inherently hairy, you need data that is representative of real-world usage patterns before you can be sure that changes you make won't break it out in the field.
Refactoring vs. Maintenance (Score:4, Interesting)
Just as in any other case in which maintenance must be done, it is quite important that this maintenance be done. It may not change the functionality of the code, but it can help make the software more easily adaptable. It can also help developers understand their own code, view it differently, and find different ways of implementing their systems. It may be more popular with Dilberts than PHBs, but perhaps those in the latter category should understand that even small amounts of refactoring can help save much time later on.
This [amazon.com] is one of my favourite books on programming/software engineering and one of the many topics it covers is refactoring. I'd say that it does a good job arguing the importance of refactoring and how to convince those PHB types to accept it. But if you're just interested in refactoring itself, I suppose that this [amazon.com] one is the best reference on the topic. I must say that for quite a few reasons, refactoring is something that should not simply be considered just another trend/buzzword, but an important part of maintenance, which in turn is an important part of the software development life cycle.
'bout time too... (Score:1)
Has anyone read the interview? (Score:1)
Q:- Why does MF find refactoring / works?
A:- The clue was in the TEST bit
How do you know your test TEST the code? esp. if you write them?
A test could be:-
Syntax, picked up by a good compiler
logic, picked up by gigo test data
response, getting someone to use your CORBA/DCOM/JE++/RPC piece of crap and ask why the new computer is sooo much slower than the old one, hey but nice graphic of a pussy cat licking its arse! I repeat TEST!!!
Well I guess (Score:1)
Refactoring is crucial (Score:5, Insightful)
Some of you people simply have NO idea how code works in the real world, i am sure of it. Hacking perl scripts is so unlike developing the large OO software that drives most information systems.
One of the fundamental issues with software architecture is that more often than not architecture is emergent. 'build one to throw away' is an old old adage (I believe it was Brooks who orginally declared this) and neatly summarises the key problem with developing software architecture.
"Even when one has the time and inclination to take architectural concerns into account, one's experience, or lack thereof, with the domain can limit the degree of architectural sophistication that can be brought to a system, particularly early in its evolution." From the Big Ball of Mud (link below).
We design, we develop, we learn, and then we design again
Pretty much ALL modern software lifecycle models are iterative, simply because up front design does not work. The waterfall model is a naive concept of a bygone era.
Refactoring is therefore a crucial aspect of an efficient design process. Typically, I would suggest that refactoring occurs at the end or begginning of each iteration
To see refactoring in action, join an Open Source project. Most OS teams that I have witnessed in action will employ refactoring of some description, even if they don't call it that. It makes a great deal of sense in OS, because we have large, distributed teams working on software, refactoring helps consolidate changes by disparate team members.
further reading: http://www.laputan.org/mud/
Does anyone else find his arguments a little (Score:1)
So if your classes don't change externally, there should be little merit to this argument. You can extend them just as well as before as you can after, performance considerations aside.
For that reason I think there is a problem with this definition of refactoring, at least in the scope of OO and functional (and any others where you can wrap glue around code) systems. "Refactoring" is necessary in poorly written (ie - code that needs "refactoring", hehe) OO and functional code because it will be buggy, broken and slow (even MS gets around to rewriting their code). It can't be avoided in procedural code because it will sooner or later become impossible to maintain. This isn't as optional a process as he implies, and is not a newfangled invention.
From the trenches... (Score:4, Interesting)
Years ago, I worked on a big software project for the aerospace industry. This was in the early 90s, and I am proud to say we were a bit ahead of our time. Iterative design, unit tests automatically running overnight, peer reviews, refactoring (although we called it "rewriting"), collective code ownership, we did it all.
After extensive integration testing (and finding and fixing many bugs), we installed the system at the customer site. After running for two weeks continuously, the system froze. Fortunately, the operating system was kind enough to report that it had run out of memory.
The cause of the problem was obvious: we had a memory leak somewhere. We had never run our code through a memory leak detection tool. The reason for this was that management did not want to spend money on something like this (...). Fortunately, we happened to have an evaluation copy of such a tool when the problem was detected. Installing the tool took 20 minutes, finding the leak took 2 minutes. It also found a memory leak in on of the OS libraries, but that was a one time leak. Our problem leak was only 8 bytes in size, but since the leaking routine was called several times per second, the system ran out of memory eventually.
Anyway, the leak was all my fault, and fixing it took about 20 seconds. Rebuild, and everybody was happy again.
So, what did we do wrong?
1. We should have had better checks for memory leaks in the first place. So, blame management...
2. We should have tested for a longer period than we did. Our integration tests included stress tests, but we never ran tests for more than 24 hours or so without rebooting (rebooting obviously hid the memory problem). Running for two weeks would have revealed the problem, but that doesn't always cure everything (read on for part two...)
Two years later. I had moved to another country, and was doing consulting work at a customer site. I got a phone call. The system had frozen again, but this time not after 2 weeks, but after 6 months running continuously. I investigated the problem, and after lots of debugging I isolated the problem to a (vendor supplied) device driver. Code inspection revealed that this driver incremented an internal counter every time it was called. This counter was a signed 32-bit integer. So, after 6 months of calling it several times per second, the counter rolled over, and the driver stopped working. Of course, a reboot fixed it, and the system was good to go for another 6 months. I'm not sure if this driver was ever fixed. You could very well argue that it's not worth the effort: just reboot once every 6 months, and you're fine.
What is the point of all this? Well, a lengthy testing period would have revealed our first bug. However, to find the second one, we would have had to do a stress test for more than 6 months. No matter how enlightened management or your customer are, they'll never agree to something like that in real life. Besides, there is a known defect in this system that will manifest itself in 2036 or so...
Also, where should you stop testing? We trusted the 3rd party driver to be coreect, and in this case we were wrong to do so. I see no practical solution for this, though.
Lesson: you can do (almost) everything right, use proper methodologies, test early and often, have a team of brilliant people, etc., and still you can have problems.
Ironically, the fact that we had stable software running on a robust operating system (UNIX), caused the bugs to manifest themselves. Had we been writing unstable software or running on Windows, reboots would have happened more than once every two weeks, and neither bug would have shown up...
Design Patterns (Score:5, Interesting)
Revenge of the Nerds, by Paul Graham
http://www.paulgraham.com/icad.html
Paul Graham is a Lisp guru, among other things. His perspective on "patterns" is quite in contrast to that of the XP fab four.
Specifically, I refer to this fabulous quote:
"If you try to solve a hard problem, the question is not whether you will use a powerful enough language, but whether you will (a) use a powerful language, (b) write a de facto interpreter for one, or (c) yourself become a human compiler for one. We see this already begining to happen in the Python example, where we are in effect simulating the code that a compiler would generate to implement a lexical variable.
"This practice is not only common, but institutionalized. For example, in the OO world you hear a good deal about "patterns". I wonder if these patterns are not sometimes evidence of case (c), the human compiler, at work. When I see patterns in my programs, I consider it a sign of trouble. The shape of a program should reflect only the problem it needs to solve. Any other regularity in the code is a sign, to me at least, that I'm using abstractions that aren't powerful enough -- often that I'm generating by hand the expansions of some macro that I need to write."
He's my hero.
Justin Dubs
Look this guy also recommends... (Score:2)
"The Editing Trap [Substituting Writing for Thinking]
Computers seem to tempt people to substitute writing for thinking. When they write with a computer, instead of rethinking their drafts for purpose, audience, content, strategy, and effectiveness, most untrained writers just keep editing the words they first wrote down. I have seen reports go through as many as six versions without one important improvement in the thought. In such writing, I find sentences that have had their various parts revised four or five times on four or five different days. Instead of focusing, simplifying, and enlivening the prose, these writers tend to graft on additional phrases, till even the qualifiers are qualified and the whole, lengthening mess slows to a crawl.
Drawn in by the word processor's ability to facilitate small changes, such writers neglect the larger steps in writing. They compose when they need to be planning, edit when they need to be revising.
[...]
Reused Prose
Writers easily become attached to what they have written, even when it serves the purpose badly. The computer frees many writers from this attachment by making the text fluid and continuously editable; for some writers, though, computers make this attachment harder to break. Typewriters challenge this attachment; in writing with a typewriter, writers typically retype each passage several times, which forces them to reread word for word and presents an excellent occasion to hear the passage and make changes. By contrast, a word processor enables writers to reuse passages from the developing piece so easily that reuse becomes a universal, invisible step in writing.
Being pragmatic, professionals often reuse blocks of material from previous reports. A good writer can do this well, but a less accomplished writer easily succumbs to a clumsy kind of self-plagiarism. Most of the adult writers I have worked with reuse "boilerplate" materials in a simple, modular fashion, stacking blocks of self-contained material in the midst of new passages, having little sense of how to combine the different parts. Most of them are tone deaf to the lurches, shifts in convention, and changes in tone between new and old writings.
I often advise authors to throw out these drafts and rewrite from scratch, but no one ever has. In part, they are always too busy; but more important, they are not writers. They are unaccustomed to taking responsibility for a piece of writing, devising an effective strategy, and seeing it through. Few of them have developed an effective writing process, and their approaches to writing lack flexibility. Such people do not need an editor; they need a writing instructor-something they lack but your students are fortunate enough to have.
[...]
Distortions of Length: Prolix and Telegraphic
The ease of writing on a microcomputer liberates many writers: And though this liberation helps reticent students, aids brainstorming, and makes many professionals more productive, the very ease of writing can lead to problems. People who have little to say suddenly take a long time to say it. Word-inflation multiplies. Instead of saying it well one time, unfocused writers devise dozens of ways of coming close to saying what they mean. They continue writing. The words pile up. The results look impressive, but I never know quite what the writers meant to say.
Computers have the opposite effect on other writers. Normally intelligible, they become cryptic. Each mysterious word stands for phrases, sentences, even whole pages of unwritten intentions. I have to pry the words apart to uncover the thoughts concealed between them.
[...]"
How Computers Cause Bad Writing [longleaf.net] by Gerald Grow, PhD
Some things that bother me about XP... (Score:1)
How do you warn people not to refactor code that may be ugly but has been performance tuned (or tuned to meet a requirement that can't be easily reflected in a unit test)? Think about the atomicity and performance issues of creating a transaction log. Your unit tests aren't going to test the key characteristics of this code (as fast as possible and uncorruptable in the face of failures). So your culture allows anyone to twiddle with this? This goes in hand with my previous question, really. Are all teams and all code created equal, as XP seems to claim? In my experience, no.
Any anecdotes out there on refactoring wars within development teams?
On making things explicit: it sounds like Fowler is arguing that static behavior is geneally better than dynamic behavior (see his Dictionary vs. hard coded class example). Is it just me or does his comments completely ignore the fact that the Dictionary example has far more flexibility than a hard-coded class?
In all I like alot of XP's characteristics - but they're always taken to Extremes! I'm serious here - many of the ideas are great (and well-known), but to reap the XP benefit it seems you always have to do it in the most extreme way possible. Despite the use of the phrase "common sense" often by XP supporters, the entire practice seems to disallow it. In fact, many web pages on XP tell you to ignore your instincts and blindly follow XP for a sizable chunk of time before judging it. This is common sense?