Please create an account to participate in the Slashdot moderation system

 



Forgot your password?
typodupeerror
Businesses IBM Programming Stats IT

The Futility of Developer Productivity Metrics 203

Posted by samzenpus
from the measure-of-a-man dept.
snydeq writes "Fatal Exception's Neil McAllister discusses why code analysis and similar metrics provide little insight into what really makes an effective software development team, in the wake of a new scorecard system employed at IBM. 'Code metrics are fine if all you care about is raw code production. But what happens to all that code once it's written? Do you just ship it and move on? Hardly — in fact, many developers spend far more of their time maintaining code than adding to it. Do your metrics take into account time spent refactoring or documenting existing code? Is it even possible to devise metrics for these activities?' McAllister writes. 'Are developers who take time to train and mentor other teams about the latest code changes considered less productive than ones who stay heads-down at their desks and never reach out to their peers? How about teams that take time at the beginning of a project to coordinate with other teams for code reuse, versus those who charge ahead blindly? Can any automated tool measure these kinds of best practices?'"
This discussion has been archived. No new comments can be posted.

The Futility of Developer Productivity Metrics

Comments Filter:
  • Refactor... (Score:5, Insightful)

    by Tsingi (870990) <<graham.rick> <at> <gmail.com>> on Thursday November 17, 2011 @02:30PM (#38089360)

    Refactor, refactor, refactor

    KISS technology, nothing beats it.

  • It's tricky (Score:5, Insightful)

    by LunaticTippy (872397) on Thursday November 17, 2011 @02:33PM (#38089404)
    Whatever you measure will be gamed. Measure bugs fixed, and you will find people wasting time listing each tiny variation of a bug. Measure lines of code, you will get spaghetti code.

    It almost seems better to measure a bunch of things and use a secret formula to determine productivity.
  • Short answers (Score:4, Insightful)

    by drb226 (1938360) on Thursday November 17, 2011 @02:39PM (#38089466)

    But what happens to all that code once it's written? Do you just ship it and move on? Do your metrics take into account time spent refactoring or documenting existing code? Is it even possible to devise metrics for these activities? Are developers who take time to train and mentor other teams about the latest code changes considered less productive than ones who stay heads-down at their desks and never reach out to their peers? How about teams that take time at the beginning of a project to coordinate with other teams for code reuse, versus those who charge ahead blindly? Can any automated tool measure these kinds of best practices?

    It bitrots. Yes. No. Maybe. Probably. Definitely. Possibly.

  • Problem (Score:5, Insightful)

    by bigsexyjoe (581721) on Thursday November 17, 2011 @02:39PM (#38089478)
    A lot of problems rating developer productivity. First, if a system is that good, then managers won't be able to game it to play favorites. Second, writing code for future use is always harder than writing code specific to the problem. Third, almost any metric is going to penalize a simpler solution. (Keep in mind that once you see a simple solution it seems obvious and everyone thinks they'd think of it. Fourth, evaluating developers well would require making the best coders managers, and that rarely happens for several reasons.
  • by Synerg1y (2169962) on Thursday November 17, 2011 @02:40PM (#38089490)

    Wouldn't it be the project leader who monitors these on an individual basis? If a coder isn't pulling their weight its up to the project leader to address it up to the point of termination. Above that you have a suit who monitors the project leader's team performance and decides how well the project leader is doing. Of all the places layered management doesn't work, coding is not one of them. It's a challenge to hold a developer accountable because there are so many different approaches to the same problem in coding and a lot have definitive pros and cons.

  • Pfft (Score:5, Insightful)

    by Allicorn (175921) on Thursday November 17, 2011 @02:41PM (#38089492) Homepage

    Can any automated tool measure these kinds of best practices?

    No. The - for the sake of politeness, let's call them "people" - who invested time and effort into devising these schemes have actually built a complete chain of negative productivity by doing so. Remarkable.

  • by Crashmarik (635988) on Thursday November 17, 2011 @02:41PM (#38089494)

    Personally I am always happy with the guy who can get things done with one line of code instead of a hundred, but what I really care about is that objective is met and we don't have a host of bugs that require 10 times the cost of the development just to maintain. Its not hard stuff but it does require common sense and a hard nosed attitude both of which can be scarce commodities these days.

  • Re:It's tricky (Score:5, Insightful)

    by abroadwin (1273704) on Thursday November 17, 2011 @02:42PM (#38089504)
    Or you could just communicate with your developers, be aware of the work they're doing and judge their performance based on their effective productivity from your perspective. I've heard this called "management" before, but I know that word has been twisted to mean something more sinister as of late.
  • by Anonymous Coward on Thursday November 17, 2011 @02:42PM (#38089506)

    If they know their productivity is being measured, it becomes a contest to see who can cook the books the best anyway.

  • Re:It's tricky (Score:5, Insightful)

    by ackthpt (218170) on Thursday November 17, 2011 @02:43PM (#38089512) Homepage Journal

    Whatever you measure will be gamed. Measure bugs fixed, and you will find people wasting time listing each tiny variation of a bug. Measure lines of code, you will get spaghetti code.

    It almost seems better to measure a bunch of things and use a secret formula to determine productivity.

    In my book there are only three ways to measure code:

    • For speed
    • For size
    • For readability

    Emphasis on any one and the others suffer. They're goal should be on bug-free code which meets a spec. Writing code is like practicing medicine, every patient is different and has its own demands.

  • lazy management (Score:5, Insightful)

    by J-1000 (869558) on Thursday November 17, 2011 @02:52PM (#38089602)
    This all comes down to lazy, gutless management. Why take the time to get to know Dave and monitor the quality of Dave's work when we can just look at a spreadsheet at the end of the month? Managers prefer to tinker with automatic analysis software rather than manage.

    Which is more fun, getting a better handle on what Dave is doing, or researching fancy new software tools that might get you all sorts of praise from metric-craving executives?

    Dave's job, which was once about creating a quality product, now shifts to merely satisfying the metrics.
  • Re:It's tricky (Score:5, Insightful)

    by icebraining (1313345) on Thursday November 17, 2011 @02:53PM (#38089616) Homepage

    So, you want to reward Wally? He probably doesn't introduce much bugs...

  • Re:It's tricky (Score:5, Insightful)

    by Curunir_wolf (588405) on Thursday November 17, 2011 @03:00PM (#38089672) Homepage Journal

    Or you could just communicate with your developers, be aware of the work they're doing and judge their performance based on their effective productivity from your perspective. I've heard this called "management" before, but I know that word has been twisted to mean something more sinister as of late.

    That won't work, because we laid off all the middle managers years ago. Developers are all exactly the same - we just need to know which ones to slot into the critical path.

  • Repeatability (Score:4, Insightful)

    by 3count (1039602) on Thursday November 17, 2011 @03:02PM (#38089700)
    Metrics are valuable if you do the same thing repeatedly. If you build a new building that is like the previous one, you can collect metrics and compare your performance against history. If you write the same search algorithm again and again, you can collect metrics and compare to see how your performance changes over time. Of course, with software, you never repeat. Somewhere around the third time, you move it into some form of library, reuse it, and start on a fresh problem. Perhaps metrics are helpful in some situations, such if your team keeps repeating the same mistakes, you might find similarity in those mistakes (code smells.) There are plenty of people working on these problems and tools. But, from a management point of view, if you keep doing the same thing, you are doing it wrong, and code metrics are not going to help much.
  • by Riceballsan (816702) on Thursday November 17, 2011 @03:05PM (#38089716)
    I don't get the concept of everything needing to be quantified. Does the team accomplish what the goals of having the team are? Does it get developed in a fair timeframe? Is everyone on the team pulling their own weight, or are there complaints of someone slacking off? In the end if the product works then the team is doing well, if the product isn't there should be at least one hybrid manager/coder that actually works with the team members sees who is committing what and can tell off the bat if there is or isn't a weak link dragging the rest down. Actually putting a pen and paper number on a complex project is silly. Do authors get judged by the number of pages they write in a day, no they get paid by the success or failure of the book. You can't judge by the number of lines of code, bugs per line ratio or anything like that, because it is all subjective and has little to no bearing on the end product.
  • by stanlyb (1839382) on Thursday November 17, 2011 @03:15PM (#38089824)
    And every developer starts as an engineer, and ends as an artist. As simple as that. The engineer can do anything, but have not done it yet. The artist knows what to NOT DO, because he has already done it, and does not wanna to repeat all the teenage mistakes again. The artist just gives you the solution. End of story.
  • Re:It's tricky (Score:5, Insightful)

    by nine-times (778537) <nine.times@gmail.com> on Thursday November 17, 2011 @03:23PM (#38089928) Homepage

    Yeah, I've come to realize that nobody understands what a manager is supposed to do anymore. Most people's concept is that managers exist to play the part of the PHB, but they don't do anything useful.

    More and more, I think no one even understands the value of having a knowledgeable person make difficult decisions based on well thought-out judgments. People want procedures that they can just give to anyone and assume that the outcome will be the same, and then they want metrics that they can use to measure the outcome without knowing anything about the situation.

    The proper role of a manager-- aside from particular responsibilities he/she might have on top of this-- is to understand the people working for him and understand his unit's role in the greater context of the company, and then to eliminate obstacles that prevent the people working for him from performing their roles. This might mean protecting the people that work for you from upper management, it might mean recognizing and rewarding good performance, or it might mean all sorts of other things. But to do it right, it takes a lot of work and it takes good judgment.

  • Re:It's tricky (Score:4, Insightful)

    by Rob the Bold (788862) on Thursday November 17, 2011 @03:36PM (#38090096)

    Actually, in my book there is only one feature that i wanna to see in every program/module/library. To.Be.Debugable. If i cannot debug it......sorry, but your code is crap.

    Unfortunately, if you can't debug it, it's your source code that's crap. Because if you're debugging it, it's yours now . . .

  • by MadKeithV (102058) on Thursday November 17, 2011 @03:40PM (#38090140)

    Personally I am always happy with the guy who can get things done with one line of code instead of a hundred, but what I really care about is that objective is met and we don't have a host of bugs that require 10 times the cost of the development just to maintain. Its not hard stuff but it does require common sense and a hard nosed attitude both of which can be scarce commodities these days.

    I am also REALLY happy to have "that guy" that has absolutely shit productivity, but somehow manages to pick up on every time a "solution" is proposed by the rest of the team to a problem that doesn't exist or doesn't matter, and stops THEM from being really efficient at doing the wrong thing.
    I'm also really happy to have "that girl" that doesn't seem to really be doing anything, but take her out of the team and everyone else starts floundering because she's actually constantly helping them be a lot more productive.
    "Meeting the objective" is actually potentially just as bad as any other metric, because it depends on how you define the objective, and meeting it. What the customer asked? What the customer wanted? Or what the customer actually needed?

  • Re:It's tricky (Score:5, Insightful)

    by marcosdumay (620877) <marcosdumay&gmail,com> on Thursday November 17, 2011 @04:07PM (#38090496) Homepage Journal

    Yes, they are objective and quantitative. Managers love that, because being objective excuses them from forming the kind of bias people used to call "leadership", and being quantitative they fit into nice formulas where they can convey no information at all to higher managers that are too insecure to ask what it means.

    Just don't expect those quantitative and objective metrics to be correlated with the overall profit of your business.

  • Re:It's tricky (Score:4, Insightful)

    by nine-times (778537) <nine.times@gmail.com> on Thursday November 17, 2011 @04:35PM (#38090818) Homepage

    Yeah, that's my point. As a manager, it's your job to learn about the people who work for you, to help them understand what they should be working on, to help motivate them, to judge their strengths and weaknesses, and to coordinate/arrange things so that their weaknesses don't keep them from doing their job well. Then you have to weed out people who aren't going to do a good job, and meanwhile reward, foster, and develop the employees who have potential and do good work.

    That's a lot of work, but that's the work that a manager is supposed to be doing. Too often they shirk that work and instead treat their employees like "gears in a box". Sometimes it's laziness, but a big part of the problem is that our society/culture doesn't recognize the work/judgement of a good manager as valuable. We instead expect people to be interchangeable, which is a problem.

  • by Radres (776901) on Thursday November 17, 2011 @06:47PM (#38092760)

    It's easier to just refactor for re-use after the fact than try to design upfront for something that might someday never happen.

  • by c++0xFF (1758032) on Thursday November 17, 2011 @06:55PM (#38092848)

    And the "reusable" version very often isn't reusable at all, since the original coder didn't properly envision what other use cases would look like.

When you don't know what you are doing, do it neatly.

Working...