Why We Refactored JUnit

Bill Venners writes "In this article, three programmers tell the story of how their frustration with JUnit's API led to the creation of Artima SuiteRunner, a free, open source test toolkit and JUnit runner. These programmers simply wanted to create a small add-on tool to JUnit, but found JUnit's design non-intuitive and API documention poor. After spending time reading through JUnit's source code and attempting to guess at the API contracts, they gave up and rewrote it."

Re:The truth about XP

[Anonymous Coward]

I'm surprised, it almost seems like you didn't read much of the information on the website... like you... maybe... just read the slashdot headline and then formed your opinion... because here's what they guy who re-wrote JUnit said:

"at one point I just threw up my hands and said to myself, "Why is this so hard? It would be easier to rewrite JUnit than figure it out."

Just some friendly advice: if you ever find yourself saying it would be easier to rewrite something rather than figure it out, slap yourself. Chances are you are wrong. I certainly was. Creating SuiteRunner was a huge amount of work. Despite my frustrations with JUnit, it would have been orders of magnitude easier to decipher JUnit's source code than to create a brand new testing toolkit."

Intuitive

[nettarzan]

Is intuitive design necessarily good in programming.?
Recursion is intuitive but not necessarily efficient in terms of performance.
At what point do we decide which is better intuitive design or efficient design.

Re:Origins of XP

[Anonymous Coward]

XP is about writing code that can easily be successfully refactored. Total-design-up-front bigots think the code "sucks" because the design wasn't much more general than actually needed at the time, not realizing that making a design at the start of the project that meets the requirements as of the end of the project is almost always impossible, and all the effort in doing so is going to be wasted.

Re:Intuitive

[dangermouse]

Recursion is not a design principle, it's an implementation decision-- like choosing between a switch block and cascading if/else blocks.

The author of this article is talking about the JUnit API. An intuitive API is always a good thing.

Re:refactor == rewrite?

[dubl-u]

Refactoring is improving the design of existing code. So yeah, they're just going for buzzword value here.

Re:The truth about XP

[dubl-u]

but after skimming through the first 5 chapeters of 'Planning XP'

If you're a programmer, that's the wrong book to start with. Read Extreme Programming Installed [amazon.com], by Jeffries, et al.

I'm starting to think that XP really is just one big hot air balloon.

In my experience, it's not. Since adopting it a couple years back, my designs are better, my bug counts are much lower, and I am much happier. Feel free to drop me a line if you have questions.

Paradigm shift?

[HisMother]

This is not a troll, but I'm struggling to try to understand why these guys had to work so hard to accomplish what they needed. By their own account, they were driven towards this because they wanted to write API conformance tests -- tests that pass if an API is implemented, regardless of whether it works or not. They wanted, I guess, to see a test pass for each API from the specification that was actually implemented, so they could compute a percentage.

While I can understand what they wanted to do, I guess I don't see why they didn't do this the much easier and (to me, anyway) more obvious way: you write code that uses the API, and if it compiles, then the API is implemented. You write one tiny test class for each API. You have each JUnit test run a compiler over one single class, this proofing each API. This isn't exactly rocket science: you have a directory full of probe-class source code, a single method that tries the compile, and then any number of TestCases that each try to compile one class and pass if they succeed. This is hard?

If you use the unpublished APIs to "javac," you could do this at lightning speed. Alternatively, you could just use a Java parser like the one that comes with ANTLR (since you don't actually need to generate any code) and do it even faster.

This is what I like about Open Source Software

[Anonymous Coward]

This is one reason I like OSS--when people are not satisfied with currently available solutions, they can take the code and improve it. I have not looked at the new code (and for that matter, am unfamiliar with JUnit), but I think allowing people the freedom to take code and improve it one of the best features of OSS.

Re:Intuitive

[iabervon]

Intuitive deisgn of anything that people will need to change in the future is more important than anything else, because anything you do efficiently but confusingly now will get changed in the future to be broken. If a future programmer can understand your code easily, it can then be optimized if it turns out to be too slow. And good design often turned out to be both efficient and intuitive; it tends to be free of the junk that both makes things confusing and slow.

For that matter, how fast do you want the testing infrastructure to run?

Re:Intuitive

[dubl-u]

Optimization should never be done until after you've run real-world use cases on code with a profiler. Until then, one should write code that clearly communicates its intent.

"Premature optimization is the root of all evil."

says Knuth, and Martin Fowler says

"Any fool can write code that a computer can understand. Good programmers write code that humans can understand."

Remember: Computers double in speed every 18 months, but programmers stay just the same. If we want our code to live a long and happy life, then clarity should almost always win over speed.

Re:The truth about XP

[babbage]

On the other hand, there's Fred Brooks' advice from The Mythical Man-Month [aw.com]: "Plan to throw the first one away. You will anyhow."

Sometimes, it really is easier to treat the first one as a prototype of what you'd really like, and then write that second one correctly, from scratch. Witness Mac Classic -> Mac OSX, Win9x -> WinNT, Perl5 -> Perl6, etc.

A lot of these prominent "next-generation" systems may share ideas & even code from their predecessors, but the essential point with each is that they all represent deliberate jump-cuts from the past. Sometimes it really is easier and better to rewrite something, whether or not you fully grok the original.

The real trick is to design systems well enough that, when someone comes along with better ideas, the framework you provide is robust enough to adapt. Mac Classic, Win9x/DOS, and Perl5 are all too inflexible for future use, though at this point each of them is still useful to certain groups. Their successors, however, are all designed with future expandability in mind, so that the "second system curse" can hopefully be avoided. History will tell if it ended up working in any of these cases...

if it ain't broke....

[the-build-chicken]

and also, I don't have to put my email in the database of a company that may or may not honour their privacy policy when times get tough, when I download junit. Maybe I'm a little jaded about open source sites that look flashy and require emails to download...burnt by the whole 'jive' from coolservlets.com fiasco (making lot's of cash are you now boys...remember, you can't buy back your dignity). The simple concepts are often the best, junit works great. I see no real advantage in this one.

Re:The truth about XP

[tupps]

I think comparing operating systems as a way of throwing out your first born is not correct. Eg MacOS Classic was a direct descendent of the original MacOS that launched in 1984. I am sure that there is very little or no code from the original. Win9X was built on top of DOS which I believe has a lineage older than MacOS classic. Interesting note that both MacOSX and WinNT both came from an external base (NeXT and OS2 sort of). If you want a better example IE 3.0 --> IE 4.0, Microsoft is a classic for working out the problems in the first version or two and then releasing a quality product.

Design first, or refactor? Re:Origins of XP

[2nesser]

making a design at the start of the project that meets the requirements as of the end of the project is almost always impossible

I believe that this quote is true for commercial software. The word processors, email clients and web browsers in the world should not be designed at the beginning of a project. The cost of doing that far outweighs the benifits. But...

When you are designing life critical applications (fly-by-wire, ABS, pacemaker, cancer radiation machine...etc) it is not acceptable to be redesigning while coding.

If you cannot model the way a system must react at all times, under any input, then what makes you think that your software should be responsible for someones life.

More and more software is being designed for life critical applications. If XP is being used to develop them, I'm worried. You cannot stumble onto 100% correct software. In a large system it takes a lot of money - an exponential curve, the last 3 bugs will likely take more money than the first 100 - time and a good design.

Re:Paradigm shift?

[The Musician]

To help answer the grandparent's question: compile != matches API.

Can you write a test-compile program that checks whether the proper public/private/proctected access modifier keywords are set up right on each method? That the method was specified for 'short foo' but the implementor used 'int foo'? Or like Mithrandir said, that no additional methods have been added (such as an overloaded version)?

Basically, compile-compatability is not the same as using the precise APIs in the spec.

Re:genericity in testing

[Anonymous Coward]

My biggest problem is writing code that works in the first place so that you can even test it ;)

Ah, you do not understand the zen of XP yet. First you write the test. Then you write the code. Then you correct the code so the test passes. The sun is warm; the grass is green.

Re:genericity in testing

[Anonymous Coward]

The sun is warm; the grass is green.

But you, poor Grasshopper, do not truely understand Zen.

Re:The truth about XP

[WaKall]

If YOU write the first one, throw it away, and write the second one, then yes, you may be on to something. However, if someone else writes the first one, and you can't understand it/use it, it doesn't that the first solution wasn't good. Maybe the problem lies somewhere else?

You don't learn as much from reading someone else's code as they did from writing it. Don't try mis-quote Brooks to justify not understanding an existing product. Brooks was referring to large-scale team oriented development (like the System 360, or IBM 650 probjects). He wasn't claiming that you should throw out someone else's code because it was the first solution to a problem.

Re:I tried too..

[Pastis]

First, to answer somebody's else comment, the other guy who coded JUnit that nobody can't remember was Erich Gamma [slashdot.org]. If that name doesn't tell you anything, go back and read some books. It's impressive how the most important people 10 years ago can be forgotten so quickly. Makes you understand why Software is often called Software Craft.

Open source software don't come up often with design documents. This one does. To those who say JUnit is hard to understand, go and read the Cook's tour [sourceforge.net]. Then you'll understand everything. Things have changed a little bit but not that much. Of course if you are not familiar with Design Patterns, I recommend to read Design Patterns [hillside.net] first. Yes that's by Erich Gamma and co also called the Gang of 4 [c2.com]

I see allready people coming and say design pattern is crap, and that's a proof. But before coming in and complaining, remember the lessons learned by the people who have worked longer than us in this industry. KISS(Keep It Simple...) [techtarget.com] and don't reinvent the wheel: I am pretty sure that the elegant design used in JUnit could have been refactored quickly to do what those guys wanted without having to rewrite everything from scratch.

The author of this new test suite says it all:

Creating SuiteRunner was a huge amount of work. Despite my frustrations with JUnit, it would have been orders of magnitude easier to decipher JUnit's source code than to create a brand new testing toolkit.

Solid Foundation For Software Refactoring

[Bart Du Bois]

To get to know refactoring, I can refer to http://www.refactoring.com [refactoring.com] (by Martin Fowler).

For those interested in the value of refactoring (whether it's merely a buzzword), I can refer to a research project of ours, at http://win-www.uia.ac.be/u/lore/refactoringProject [uia.ac.be]

By the way: since the article does not go into behaviour-preserving restructuring of JUnit, they shouldn't mention 'refactoring' in the title.

Re:Origins of XP

[Ed Avis]

If the code is well written, there's few need
for writing tests (except for sophisticated
algorithms)

Well, duh. If it is well written it won't have bugs, by definition. But how do you find out whether the code is well written, except by thoroughly testing it?

Re:genericity in testing

[Nyarly]

That's extreme programming? Good thing I didn't spend time reading up on this new methodology since I've been doing that for 10+ years.

You'd probably love XP then. A lot of the XP methodologies (with the possible exception of Pair Programming) make heaps of sense, and it's extremely nice to have them all in one tidy package that you can sell to management.