Joel Test Updated 182
An anonymous reader writes "In 2000, Joel Spolsky wrote the Joel Test, an excellent and simple way to evaluate a software company. While the test is still used, it's getting outdated, as many companies are moving to web technologies, and new development tools exist. In his blog, Marc Garcia wrote about what could be an update to Joel Test."
Who is this guy? (Score:1, Insightful)
Seriously, who the fuck is this "Joel" person, and why does anyone give a shit what it thinks?
Is how open source devs would like corps to be run (Score:3, Insightful)
*Shrug* - just comes off as a wish list of how this developer thinks software companies should work. IMHO part of the attraction of the original Joel list was that it was more or less applicable regardless of product audience / build tools etc. The core principles *really were important*.
Total failure (Score:4, Insightful)
Example: Allow users direct access to a bug database? It's hard enough to train testers to give you good bug reports. You won't get anything usable from an end user without some severe filtering.
Re:Total failure (Score:5, Insightful)
You won't get anything usable from an end user without some severe filtering.
Indeed. This attitude is one of the biggest failures in Open Source software development, and why companies like Microsoft flourish. Microsoft software has many issues, but listening to the End User is not one of MS's problems. On the other hand, pipe up to just about any Open Source project about End User issues, and "STFU and submit a patch" is about the nicest thing you'll hear. Even as responsive as Apache and Mozilla are, End Users still feel this wrath. It a fact, most companies that want their products to flourish and make money are responsive to the people that actually use them. The GIMP Team? Not so much.
Re:Who is this guy? (Score:4, Insightful)
Do you sell your car every year to buy a new one? There's a cost to converting, so you have to make an engineering decision about making the conversion. The automated tools don't always work with old and complex repositories.
Re:Old system is fine. (Score:5, Insightful)
His "updates" just sound like re-statements of the original questions for a particular situation (i.e. less applicable to all modern software companies than the original).
Joel Spolsky assumed you would be intelligent enough to adapt the list to your specific situation.
For example, what good is "source control" if it doesn't effectively control the source code? There is no need to specifically mention distributed source control; if your source control is doing its job then you have good source control. If it isn't doing its job because you've got developers all over the country, then you need a distributed source control. It's built in to the question.
Customers directly reporting to a bug database, as others have mentioned, can be disasterous. However, Joel's flagship software is bug tracking software, and from what I've heard it's very, very good. His bug tracking uses a combination of silent reports from the software, direct customer input, and support service input. Specifically stating bug tracking must be entered directly by the customer is stupid and inflexible, and does not apply to all situations. The point of the software test is to apply to all situations.
It goes on, most of them are similar, but this one is egregious:
Do you have automated build or deployment procedures?
What the hell does he think "Can you make a build in one step?" means?! That's automated build and/or deployment!
Also:
Do you fix bugs before implementing new features?
Uh... frankly, that sounds worse than "Do you fix bugs before writing new code?"
Do you have a roadmap, and you don't make important changes to the short term priorities?
A) That's not the programmers job nor responsibility, and B) "Do you have an up-to-date schedule?" Hello?
Seriously, what does this guy think all these words mean? Just because they were written 10 years ago doesn't mean the meanings of the words changed. Apply them to your situation, they fit just fine.
Last but not least:
Do your team work in good conditions (quiet environment, flexible schedule, freedom to choose development software, fair paycheck...)
That's a dream of every office worker in America, and if you refuse to work at companies that don't have an office culture like that, well, you won't be working much unless you are seriously hot shit.
Point-by-point analysis (Score:4, Insightful)
The guy's apparently from Belgium, so English is quite possibly his fourth language, so I won't bother ripping on his grammar. His content is another matter...
Original: Do you use source control?
New: Do you use a distributed source control system?
My current Big Work Project has a whopping four coders, so I can't speak to when distributed source control is a big deal and when it's overkill.
Original: Can you make a build in one step? Do you make daily builds?
New: Do you have automated build or deployment procedures?
Clearly inferior. An error-prone 20-step process that you run once a month is still automated, just not automated enough and not used enough.
Original: Do you have a bug database?
New: Do you use a bug database where users can report bugs directly?
The BWP is still small enough to get by on Excel lists, with changes manually merged back into the master copy by the project manager, or just e-mail for quick-turnaround items. Excel is noticeably clunkier than an automated system, but you may want to start there to get a feel for what the automated system should do (e.g. separate status fields for "the coder did some testing and thinks it's fixed" vs. "the tester did some more thorough testing, confirmed that there were no misunderstandings, and couldn't find any more edge/corner cases").
Original: Do you have testers? Do you do hallway usability testing?
New: Do you have a testing protocol, and specific resources for testing?
I hate calling people "resources". Also, your protocol should stick to the right things (e.g. "when you find a problem, report X and Y and Z"); an example of a wrong thing is "test this specific way of using the system", because real users will go off the rails.
Original: Do you fix bugs before writing new code?
New: Do you fix bugs before implementing new features?
More or less equivalent.
Original: Do you have an up-to-date schedule? Do you have a spec?
New: Do you have a roadmap, and you don't make important changes to the short term priorities?
These have become fuzzy for no good reason that I can discern.
Original: Do programmers have quiet working conditions? Do you use the best tools money can buy?
New: Do your team work in good conditions (quiet environment, flexible schedule, freedom to choose development software, fair paycheck...)
More or less equivalent. "Fair paycheck" is so blindingly obvious that it shouldn't need to be pointed out. "Flexible schedule" is a genuinely good addition; I've personally gained some peace of mind by saving some tasks for evenings/weekends when I knew I wouldn't be interrupted by other work stuff (family stuff is another matter, but easier to control), and consequently taking it easier during normal business hours.
Original: Do new candidates write code during their interview?
This has been omitted completely for no good reason that I can discern. Maybe he's lucky and hasn't had to clean up after a bad coder yet.
Re:Nobody takes this seriously... (Score:4, Insightful)
If you think Microsoft products are bad, imagine how much worse they would be if Microsoft wasn't answering yes to a lot of those questions. Also understand that the article was written in 2000, when the computer world landscape was vastly different than it is today. Google and Apple where nowhere near as relevant, Linux wasn't a viable option for grandma, and it appeared as though Microsoft would continue to dominate the industry just as it had in the 90's. If you wanted to name drop a company that everyone would know and recognize as a leader in the software industry, you would have used Microsoft as well.
Simple test for when a company is too big. (Score:5, Insightful)
Re:Who is this guy? (Score:4, Insightful)
If it means redoing all of your established routines and teach people the new routines, rewriting all your automation, and obsoleting existing ticket or work log systems (or otherwise run two repositories in parallel, with the problems of authorativity that entails), "nicer" doesn't necessarily cut it. There has to be a gain that measurably outweighs the inconveniences.
The Joel Test does not need updating (Score:4, Insightful)
as many companies are moving to web technologies, and new development tools exist.
Web technologies change nothing in his test. And his test does not mention any specific tools, just general classes of tools. "Do you use source control?" and the catch-all "Do you use the best tools money can buy?" are asking if you use the general types of tools that distinguishes good shops from bad shops. You could add "Do you use a mock-objects framework?" but now it isn't universal, because that doesn't always apply and could be subject to debate.. Then it just becomes someone's checklist of "Have you used every tool that I use and endorse?" The Joel Test is universally applicable, covering the kinds of things all shops should do.
Most of the updates in his blog are a pedantic rewording of the existing ones.
Re:Nobody takes this seriously... (Score:4, Insightful)
The idea that borland didn't know about the performance of loading a segment register is ridiculous. It's in the intel manual. Everybody I knew who cared about the performance of software had that manual handy. Then eventually the compiler just took care of it for you and we all stopped caring.
Re:A serious question (Score:5, Insightful)
Has Joel Spolsky done anything that's worth a damn?
Not really, but he seems to be an expert bullshitter who throws around the fact that he once worked at Microsoft every chance he gets. As for what he's done, let's see:
City Desk - some sort of program for creating and managing websites. Little or no mention of it on his website anymore and the City Desk forum is long gone from the website.
Fogbugz - The Forgbugz forum is also gone from his website. Here's a blurb from Joel this past September: "Thanks to the hard work of the Fog Creek team, including ten great summer interns, we have just released amazing new upgrades to FogBugz " Thank god for free labor.
Co-Pilot - a remote access program that was written entirely by summer interns. Really. Thank god for free labor.
Stack Overflow - Not an actual application but simply a website where people can ask questions.. This doesn't stop Joel from proclaiming "I’m the CEO of Stack Overflow".
From what I can see on his website, his main business now is ads for programmer jobs.