Why We Refactored JUnit 192
Bill Venners writes "In this article, three programmers tell the story of how their frustration with JUnit's API led to the creation of Artima SuiteRunner, a free, open source test toolkit and JUnit runner. These programmers simply wanted to create a small add-on tool to JUnit, but found JUnit's design non-intuitive and API documention poor. After spending time reading through JUnit's source code and attempting to guess at the API contracts, they gave up and rewrote it."
genericity in testing (Score:4, Interesting)
For example, you have a class that generates an XML document and you want to test it. Which level do you run the tests at? Best to test all levels, of course. How then do you check the accuracy of intermediate results? The answer often is that you can't; the intermediate results are meaningless until you have the complete (or at least almost complete) infrastructure to interpret the data. But once you've run all that, it is very difficult to pinpoint the source of the problem.
I'm glad that Artima SuiteRunner provides a way to handle all of that complexity and write unit tests that are complete but tractable. I just wish it worked with Perl, since I don't use Java anyway.
The truth about XP (Score:5, Interesting)
Well, it also happens that JUnit was developed al la XP by Kent and one other guy (didn't pay attention to who, go look it up!) For a while I was thinking that this XP thing might actually be something, but after skimming through the first 5 chapeters of 'Planning XP' coupled with the statements concerning the JUnit API, I'm starting to think that XP really is just one big hot air balloon.
In his defense though, he did say that refactoring often was a GOOD idea. It's just that he didn't say that you should wait for someone else to do it for you.
My 2 cents.
Re:genericity in testing (Score:5, Interesting)
My big challenges are (1) existing code that wasn't designed for unit testing, (2) testing with databases, (3) testing that involves user interaction and (4) testing that involves multi-tasking. I like unit testing, but it is hard to apply to every task.
Origins of XP (Score:5, Interesting)
The primary one is his book Refactoring. It describes his experience as a consultant refactoring medium sized software projects, and gives a lot of advice on methods for refactoring.
Apparently Fowler decided that refactoring is a good thing. Not just that, he decided that since refactoring is a good thing, and so should be the basis for programming, since most of the results of programming are bad. By that I mean just that most programs suck, not that they are evil or anything.
At that point he joined forces with Beck and formed his second reason for being well known, XP. As far as I can tell, the philosophy of XP is, "Software usually ends up sucking and in need of refactoring after it has been extended too far. Why wait for it to be extended too far? Just make it suck in the first place and refactor all the time!"
It does have a sort of perverse logic to it, but when I say that XP is bad, I don't just mean that it sucks. As well as sucking, XP really is evil.
Re:The truth about XP (Score:1, Interesting)
Read the tests (Score:2, Interesting)
Full disclosure: I haven't screwed with the JUnit internals. But I'm working on a system that has over a million lines of code and JUnit 3.7 works just fine for it. Thousands of tests, too.
refactor == rewrite? (Score:5, Interesting)
free registration required? (Score:2, Interesting)
don't get me wrong - i'm happy the code is open. we use junit for our open-source product and anything that could help us better utilize junit is much appreciated.
JUnit does have problems, but .... (Score:3, Interesting)
However, I wonder why they rewrote it from scratch. Wouldn't it have been simpler to redesign the problematic parts. And I also find it interesting that they have written their project to be compatible with existing tests. Does that means that JUnit's interface is not problematic, only the implementation? Seems to me like more of a case for JUnit 4.0 then a complete re-write from scratch.
The follow up articles should be interesting
I tried too.. (Score:2, Interesting)
Overall though it seems like mostly a documentation issue, not a design issue. Good documentation for the internals of JUnit is pretty non-existent from what I could find. I discovered a lot mostly from examining other JUnit extensions. With some good documentation on JUnit internals and documentation on the internal flow of operations I think I could have hugely cut down on the time I needed to figure out where to plug in. I would be willing to take a crack at this documentation, but I am *definitely* not the best person to do it :) However if anyone else is interested I would be willing to give it a go?
Re:Paradigm shift? (Score:4, Interesting)
And if the methods are there because they have to be to implement an interface, but they return some "Not Yet Implemented" error or exception? How do you handle that with your compiler? I agree with you that they could've accomplished their goal in a much simpler fashion (one test for each API that only fails if NYI is thrown, regardless of any other exceptions or error codes). However, that doesn't mean they shouldn't have re-written the software. They identified issues with extending the existing software, and if they had problems then surely plenty of other people are having problems as well. Assuming that they weren't on a schedule so tight as to prohibit it, they should then do the "right" thing and fix what's broken not only for their current use, but also for their future use and the use of others.
Re:JUnit does have problems, but .... (Score:3, Interesting)
However, once I decided to "refactor" JUnit itself, I chose to not use any existing code. My main problem with JUnit was its API, not its implementation.
This is really a generic issue: when do you tweak existing code and when do you start over? By nature I am highly suspicious of urges to start over. In this case, I wanted to redesign the API first and foremost. JUnit's existing code didn't offer much assistance there.
On the other hand, JUnit's API design and very existence offered a huge amount of assistance. We could learn from what we deemed the mistakes in JUnit's design, and replicate what we felt were the good aspects of the design.
What's the big deal? (Score:3, Interesting)
"Refactoring" - holy fuck, where do you get such words?
*
Re:refactor == rewrite? (Score:5, Interesting)
I interviewed Martin Fowler, author of Refactoring, last year. In this interview [artima.com] I asked him to define refactoring. He said:
We didn't refactor JUnit's code, because we didn't start with JUnit's code and make changes. But we did do what I consider a "refactor" of the design. We started in effect with JUnit's API, and made changes. We started with knowledge of what we liked and didn't like about JUnit's API, and refactored that.
Where you can see the refactoring is not in the code, but in the JavaDoc. Compare JUnit's JavaDoc [junit.org] with SuiteRunner's JavaDoc [artima.com]. I'm guessing it was version 3.7 of JUnit that I had in front of me when I decided I would start over. It may have been 3.6.
Re:Design first, or refactor? Re:Origins of XP (Score:2, Interesting)
Here is an honest question, I know very little about XP, but how does it keep the rate of change consistent with all this "refactoring" going on?
As your code base increases from cycle to cycle and more features are added to the system do you not end up spending most of your time "refactoring" your old code so that the new feature fits into the system?