The Futility of Developer Productivity Metrics 203
snydeq writes "Fatal Exception's Neil McAllister discusses why code analysis and similar metrics provide little insight into what really makes an effective software development team, in the wake of a new scorecard system employed at IBM. 'Code metrics are fine if all you care about is raw code production. But what happens to all that code once it's written? Do you just ship it and move on? Hardly — in fact, many developers spend far more of their time maintaining code than adding to it. Do your metrics take into account time spent refactoring or documenting existing code? Is it even possible to devise metrics for these activities?' McAllister writes. 'Are developers who take time to train and mentor other teams about the latest code changes considered less productive than ones who stay heads-down at their desks and never reach out to their peers? How about teams that take time at the beginning of a project to coordinate with other teams for code reuse, versus those who charge ahead blindly? Can any automated tool measure these kinds of best practices?'"
Writing for reuse is lacking use case (Score:5, Interesting)
more reasons than just reuse. (Score:5, Interesting)
Writing for reuse can be excessive, but there are a number of reasons to move in that direction even if you don't intend to reuse that specific code block:
1) Unit Tests. If you abstract your functionality in a way that allows reuse, it also abstracts in for extremely easy unit testing. And unit testing will save you an incredible amount of effort in code maintenance.
2) Consistency. If you follow the same design pattern for all of your abstractions, all of your developers should be familiar with it. This makes it significantly easier for different developers to step into projects as the hopefully don't have to learn another person's style for abstraction.
3) Replacement and isolation. Need to implement a functional change? If your code is abstracted, like it would be for reuse, the functional change is limited to a single block, which is easily identifiable and if you're doing it right, unit-testable.
4) Just in case. Most of the time abstracted code doesn't get reused, and event when it does get reused it's usually a copy and paste job instead of a reference. Even so, if anyone ever does need the same functionality, it allows them to quickly rip off the exact piece they are looking for as opposed to trying to strip out your programs logic to get the tiny bit they want.
-Rick
Measure Effectiveness, not Productivity (Score:4, Interesting)
I think the idea of "productivity" is a hold-over of the Industrial Revolution that does not pertain to many of today's jobs; jobs where the unit of work is hard to define, and ultimately irrelevant. Are you telling me you pick your doctor by how many patients he can see in a day? Probably quite the opposite!
In terms of software development, I find that the *effectiveness* of a developer is more important, where effectiveness considers the following (not an exhaustive list):
- Appropriateness of solution
- Thoroughness of implementation (logging, exception handling, graceful failure, input validation, etc.)
- Well-written, parsimonious code that is easy to read and descriptive of what it does
- Works right the first time, no kickbacks from QA or end user
Give me someone who is effective but slow over someone who craps out junk quickly any day of the week and twice on Sunday! In the end, I don't care about productivity metrics, I care that the end users get a useful piece of software that does what they need with a minimum of headaches.
Re:Refactor... (Score:5, Interesting)
Refactoring and metrics are often rarely agree. I remember one job I came into, I was given this steaming horrible piece of php (ugh) mess to fix up and add a bunch of features for the client. I took one look at it, and said "I need a good two months to fix this code" and demonstrated a series of horrible vunerabilities and fuckups in the code. Realising the vunerabilities (hint, demonstrating security holes in existing code is a REALLY good way to convince management you need to spend some quality time refatoring it) where just plain dangerous, I got given the go ahead.
A couple of weeks in, the manager called me into the office asking "Shayne, this is very strange. You know we take a metric of how many lines of code per hour are written into account here?". "Yes" I replied, "You also know I think its idiotic". The guy said "Right , well your registering a highly negative score on this. I dont even know if this is working right?".
The reality was this code was written by someone who really didn't "get" databases. A table would be loaded in, then iiterated over, and each pass would issue new sql calls for data, which would then be loaded in and iiterated again. I'd just delete all of the twenty+ pages of code, write a half-page winner of an SQL query, drop it into a mysql stored procedure, and select from it, then spending half a page formatting it and displaying it. It was robust, safe, and fast as hell. I went through the whole code like this turning hundreds of pages of code into tens of pages, all neatly commented, scrubbed of nasties, isolating out the design from the logic, putting models into model classes, views into templates and controllers to glue it all together.
And the end result was every day the code would lose twenty-thirty pages of code and develop 2 or 3 pages of terse, clean, maintainable code.
And as an end result, my metrics where fucked. I tried to argue that they needed to look at more sane metrics, like what was coming out of the CI software, and if they wanted to be really clever, things like complexity metrics and the like.
Ultimately however ,they still wanted the "codes of line per day" metrics because as non technical people it made them feel like they understood what was going on, but ultimately they didn't. My suggestion to them however was "When developing fresh new code, that number should increase, when refactoring code, it should decrease. But only if the coder knows what he's frigging doing."