Forgot your password?
typodupeerror
Businesses IBM Programming Stats IT

The Futility of Developer Productivity Metrics 203

Posted by samzenpus
from the measure-of-a-man dept.
snydeq writes "Fatal Exception's Neil McAllister discusses why code analysis and similar metrics provide little insight into what really makes an effective software development team, in the wake of a new scorecard system employed at IBM. 'Code metrics are fine if all you care about is raw code production. But what happens to all that code once it's written? Do you just ship it and move on? Hardly — in fact, many developers spend far more of their time maintaining code than adding to it. Do your metrics take into account time spent refactoring or documenting existing code? Is it even possible to devise metrics for these activities?' McAllister writes. 'Are developers who take time to train and mentor other teams about the latest code changes considered less productive than ones who stay heads-down at their desks and never reach out to their peers? How about teams that take time at the beginning of a project to coordinate with other teams for code reuse, versus those who charge ahead blindly? Can any automated tool measure these kinds of best practices?'"
This discussion has been archived. No new comments can be posted.

The Futility of Developer Productivity Metrics

Comments Filter:
  • Refactor... (Score:5, Insightful)

    by Tsingi (870990) <graham.rick @ g m a i l .com> on Thursday November 17, 2011 @03:30PM (#38089360)

    Refactor, refactor, refactor

    KISS technology, nothing beats it.

    • Re:Refactor... (Score:5, Informative)

      by ZeroExistenZ (721849) on Thursday November 17, 2011 @03:58PM (#38089660)

      Application design (have someone think about everything in broad lines, set out a main architecture-model in ADVANCE.)

      Iterative design.

      A good project manager prioritizing and communicating current development shielding programmer from clients (who during the "waiting time", are fantasizing changes).

      In contrast, in debugging and QA, free way between the client and the developer (more efficient and gives more motivation to the dev as to slave for )

      Realistic expectations: don't get your devs running around doing "quick" jobs left and right while they're trying to keep a tight schedule.

      Keep everybody current: short standup meeting in the morning

      Motivate communication; devs tend to get absorbed in their coding-problem (nd prioritize being "productive") but forget to be a team

      The right personality on the right segment of a project: developers have their strengths and their weaknesses.

      ...

    • Re:Refactor... (Score:5, Interesting)

      by sg_oneill (159032) on Thursday November 17, 2011 @07:55PM (#38092854)

      Refactoring and metrics are often rarely agree. I remember one job I came into, I was given this steaming horrible piece of php (ugh) mess to fix up and add a bunch of features for the client. I took one look at it, and said "I need a good two months to fix this code" and demonstrated a series of horrible vunerabilities and fuckups in the code. Realising the vunerabilities (hint, demonstrating security holes in existing code is a REALLY good way to convince management you need to spend some quality time refatoring it) where just plain dangerous, I got given the go ahead.

      A couple of weeks in, the manager called me into the office asking "Shayne, this is very strange. You know we take a metric of how many lines of code per hour are written into account here?". "Yes" I replied, "You also know I think its idiotic". The guy said "Right , well your registering a highly negative score on this. I dont even know if this is working right?".

      The reality was this code was written by someone who really didn't "get" databases. A table would be loaded in, then iiterated over, and each pass would issue new sql calls for data, which would then be loaded in and iiterated again. I'd just delete all of the twenty+ pages of code, write a half-page winner of an SQL query, drop it into a mysql stored procedure, and select from it, then spending half a page formatting it and displaying it. It was robust, safe, and fast as hell. I went through the whole code like this turning hundreds of pages of code into tens of pages, all neatly commented, scrubbed of nasties, isolating out the design from the logic, putting models into model classes, views into templates and controllers to glue it all together.

      And the end result was every day the code would lose twenty-thirty pages of code and develop 2 or 3 pages of terse, clean, maintainable code.

      And as an end result, my metrics where fucked. I tried to argue that they needed to look at more sane metrics, like what was coming out of the CI software, and if they wanted to be really clever, things like complexity metrics and the like.

      Ultimately however ,they still wanted the "codes of line per day" metrics because as non technical people it made them feel like they understood what was going on, but ultimately they didn't. My suggestion to them however was "When developing fresh new code, that number should increase, when refactoring code, it should decrease. But only if the coder knows what he's frigging doing."

  • by rednip (186217) <{moc.liamg} {ta} {pinder}> on Thursday November 17, 2011 @03:31PM (#38089376) Journal
    If you don't have a use case for reuse, you shouldn't try to code for it. To many 'interfaces' are single use, see 'servlet' vs. 'http servlet'.
    • by RingDev (879105) on Thursday November 17, 2011 @04:10PM (#38089756) Homepage Journal

      Writing for reuse can be excessive, but there are a number of reasons to move in that direction even if you don't intend to reuse that specific code block:

      1) Unit Tests. If you abstract your functionality in a way that allows reuse, it also abstracts in for extremely easy unit testing. And unit testing will save you an incredible amount of effort in code maintenance.

      2) Consistency. If you follow the same design pattern for all of your abstractions, all of your developers should be familiar with it. This makes it significantly easier for different developers to step into projects as the hopefully don't have to learn another person's style for abstraction.

      3) Replacement and isolation. Need to implement a functional change? If your code is abstracted, like it would be for reuse, the functional change is limited to a single block, which is easily identifiable and if you're doing it right, unit-testable.

      4) Just in case. Most of the time abstracted code doesn't get reused, and event when it does get reused it's usually a copy and paste job instead of a reference. Even so, if anyone ever does need the same functionality, it allows them to quickly rip off the exact piece they are looking for as opposed to trying to strip out your programs logic to get the tiny bit they want.

      -Rick

    • If you don't have a use case for reuse, you shouldn't try to code for it. To many 'interfaces' are single use, see 'servlet' vs. 'http servlet'.

      This is an especially good policy if you're a contractor paid by the hour.

  • It's tricky (Score:5, Insightful)

    by LunaticTippy (872397) on Thursday November 17, 2011 @03:33PM (#38089404)
    Whatever you measure will be gamed. Measure bugs fixed, and you will find people wasting time listing each tiny variation of a bug. Measure lines of code, you will get spaghetti code.

    It almost seems better to measure a bunch of things and use a secret formula to determine productivity.
    • Re:It's tricky (Score:5, Insightful)

      by abroadwin (1273704) on Thursday November 17, 2011 @03:42PM (#38089504)
      Or you could just communicate with your developers, be aware of the work they're doing and judge their performance based on their effective productivity from your perspective. I've heard this called "management" before, but I know that word has been twisted to mean something more sinister as of late.
      • Re:It's tricky (Score:5, Insightful)

        by Curunir_wolf (588405) on Thursday November 17, 2011 @04:00PM (#38089672) Homepage Journal

        Or you could just communicate with your developers, be aware of the work they're doing and judge their performance based on their effective productivity from your perspective. I've heard this called "management" before, but I know that word has been twisted to mean something more sinister as of late.

        That won't work, because we laid off all the middle managers years ago. Developers are all exactly the same - we just need to know which ones to slot into the critical path.

        • by mcmonkey (96054)

          Or you could just communicate with your developers, be aware of the work they're doing and judge their performance based on their effective productivity from your perspective. I've heard this called "management" before, but I know that word has been twisted to mean something more sinister as of late.

          That won't work, because we laid off all the middle managers years ago. Developers are all exactly the same - we just need to know which ones to slot into the critical path.

          Sounds like you did it right. Most places I've seen laid off the developers, out sourced that work, and added a layer of management to manage the over-seas team.

        • by yuhong (1378501)

          The sad thing is to be honest that fear-based management is indeed fundamentally flawed. Next step would be to hire better developers with passion for the job.

          • by yuhong (1378501)

            And probably compensate them using things like options. This way the need for things like lay-offs are reduced.

      • ...and judge their performance based on their effective productivity from your perspective.

        Your perspective will taint those assessments and assign different values to whoever you dis/like most. That's why we use metrics: they are quantitative and objective.

        • Yes, they do measure something, the problem being "...if interest".

        • Re:It's tricky (Score:5, Insightful)

          by marcosdumay (620877) <marcosdumay&gmail,com> on Thursday November 17, 2011 @05:07PM (#38090496) Homepage Journal

          Yes, they are objective and quantitative. Managers love that, because being objective excuses them from forming the kind of bias people used to call "leadership", and being quantitative they fit into nice formulas where they can convey no information at all to higher managers that are too insecure to ask what it means.

          Just don't expect those quantitative and objective metrics to be correlated with the overall profit of your business.

          • It seems you never had to deal with a crappy or unfair boss. Brotip: when you find one, document the crap out of everything. It'll be hard for him not to look like an asshole when you make your point with real data and he ignores it.

      • Re:It's tricky (Score:5, Insightful)

        by nine-times (778537) <nine.times@gmail.com> on Thursday November 17, 2011 @04:23PM (#38089928) Homepage

        Yeah, I've come to realize that nobody understands what a manager is supposed to do anymore. Most people's concept is that managers exist to play the part of the PHB, but they don't do anything useful.

        More and more, I think no one even understands the value of having a knowledgeable person make difficult decisions based on well thought-out judgments. People want procedures that they can just give to anyone and assume that the outcome will be the same, and then they want metrics that they can use to measure the outcome without knowing anything about the situation.

        The proper role of a manager-- aside from particular responsibilities he/she might have on top of this-- is to understand the people working for him and understand his unit's role in the greater context of the company, and then to eliminate obstacles that prevent the people working for him from performing their roles. This might mean protecting the people that work for you from upper management, it might mean recognizing and rewarding good performance, or it might mean all sorts of other things. But to do it right, it takes a lot of work and it takes good judgment.

        • "More and more, I think no one even understands the value of having a knowledgeable person make difficult decisions based on well thought-out judgments. People want procedures that they can just give to anyone and assume that the outcome will be the same..."

          Don't you see several complaints around here that managers treat developers as gears in a box or stuff like that? Do you really think that is restricted to tecnicians?

          • Re:It's tricky (Score:4, Insightful)

            by nine-times (778537) <nine.times@gmail.com> on Thursday November 17, 2011 @05:35PM (#38090818) Homepage

            Yeah, that's my point. As a manager, it's your job to learn about the people who work for you, to help them understand what they should be working on, to help motivate them, to judge their strengths and weaknesses, and to coordinate/arrange things so that their weaknesses don't keep them from doing their job well. Then you have to weed out people who aren't going to do a good job, and meanwhile reward, foster, and develop the employees who have potential and do good work.

            That's a lot of work, but that's the work that a manager is supposed to be doing. Too often they shirk that work and instead treat their employees like "gears in a box". Sometimes it's laziness, but a big part of the problem is that our society/culture doesn't recognize the work/judgement of a good manager as valuable. We instead expect people to be interchangeable, which is a problem.

        • by russotto (537200)

          People want procedures that they can just give to anyone and assume that the outcome will be the same, and then they want metrics that they can use to measure the outcome without knowing anything about the situation.

          That's what modern (from 1911) "scientific" management is all about. That's "best practices". That's ISO 9000, TQM, and all the rest of the alphabet soup. You break the work down into a number of tasks. You measure alternatives and find out the best way to do that task. You document that be

      • by Bert64 (520050)

        Yes that's the only effective way, but it doesn't scale to huge numbers and doesn't fit in with the corporate bean counters who like to see graphs and statistics...

      • by Kjella (173770)

        Yes we've tried entirely subjective management, that's why we're trying to find objective criteria. Nothing like a manager that can set pretty much any performance rating he likes, that would surely not be gamed or abused. Combine that with most everybody thinking they're above average software developers and you got the scene set for feeling unjustly passed up. Not to mention the total lack of transparency, ability to compare with other teams or to measure the performance of the managers. Neither side is e

    • Re:It's tricky (Score:5, Insightful)

      by ackthpt (218170) on Thursday November 17, 2011 @03:43PM (#38089512) Homepage Journal

      Whatever you measure will be gamed. Measure bugs fixed, and you will find people wasting time listing each tiny variation of a bug. Measure lines of code, you will get spaghetti code.

      It almost seems better to measure a bunch of things and use a secret formula to determine productivity.

      In my book there are only three ways to measure code:

      • For speed
      • For size
      • For readability

      Emphasis on any one and the others suffer. They're goal should be on bug-free code which meets a spec. Writing code is like practicing medicine, every patient is different and has its own demands.

      • In my book there are only three ways to measure code:

                * For speed
                * For size
                * For readability

        Perhaps you've got a few pages missing. Usability, functionality, reliability...

        • by ackthpt (218170)

          In my book there are only three ways to measure code:

          * For speed

          * For size

          * For readability

          Perhaps you've got a few pages missing. Usability, functionality, reliability...

          That's the end product. This is about the actual writing of code. You can apply any or all of those in respect to the end product independently. A coder could write very neat code, fast code, or compact code, which has any kind of interface or performance.

      • I think you're missing a few:

        Modifiability - This captures the extent to which a programmer correctly anticipates future changes that the code will need. It wouldn't be wise to judge a developer on this track record for just a single program, but with larger sample sizes, it could be useful.

        Time-to-develop - If you ignore this metric, you're unable to distinguish between a program that gets written in a day vs. the very same program getting written in a year.

        Now, you could argue that these two attributes a

      • by Kjella (173770)

        In my book there are only three ways to measure code:

        For speed
        For size
        For readability

        But not for correctness? Security? Robustness? Scalability? Flexibility? I think a test class I took suggested something like 15-20 separate criteria. There's a reason you can write whole books on what "good code" really is. And none of those really cover if your architecture is borked, even if the code in isolation is pretty well done.

    • by jeffmeden (135043)

      Whatever you measure will be gamed. Measure bugs fixed, and you will find people wasting time listing each tiny variation of a bug. Measure lines of code, you will get spaghetti code.

      It almost seems better to measure a bunch of things and use a secret formula to determine productivity.

      A secret formula like, oh I dunno, customer satisfaction (maybe "how many bugs make it to the customer")? Every piece of software is written for a customer (of some sort) so maybe time could be spent actually thinking about how effective the code is AFTER it's used. Speaking as someone who doesn't write much code, but lives and breathes the consequences of good and bad code every day, I can say with confidence that quality would go up if the only coding "practice" was severe punishment of anyone who intro

      • Re:It's tricky (Score:5, Insightful)

        by icebraining (1313345) on Thursday November 17, 2011 @03:53PM (#38089616) Homepage

        So, you want to reward Wally? He probably doesn't introduce much bugs...

        • by ackthpt (218170)

          So, you want to reward Wally? He probably doesn't introduce much bugs...

          Neither does he produce. If the PHB weren't so inept he wouldn't be there.

          Tell me you've never been in a workplace there didn't have a goldbricker like Wally.

        • by jeffmeden (135043)

          So, you want to reward Wally? He probably doesn't introduce much bugs...

          If his projects are rated as satisfactory by management (this "metric" ALWAYS comes first, no matter what the development style is) then sure. Sometimes the zen of knowing what not to do is more effective than having the energy to code 10,000 lines of something that is buggy and ultimately unneeded.

      • Your policy would certainly render quality code... eventually. In the mean time you would be forgetting that nasty issue called "time to market".

        Remember: if you are not ashamed of showing your code, you waited too much to promote it.

        • by smelch (1988698)
          It is odd how many code-sharing sessions start with "this is all ugly and I haven't had a chance to clean it up yet but...."
    • by sjames (1099)

      Coming up with a developer value metric is NP hard. The best you can do is use human intuition (AKA, a formula so secret even the user isn't sure what it is) to approximate a reasonable result and then look at metrics to see why the intuitive approximation is/is not valid. It cannot be proceduralized without hand waving.

      Note to managers?: You don't WANT this proceduralized or YOUR manager will replace you with a spreadsheet. Watching your former employer tail spin into the water might provide some brief sat

    • by c0lo (1497653)

      Whatever you measure will be gamed. Measure bugs fixed, and you will find people wasting time listing each tiny variation of a bug. Measure lines of code, you will get spaghetti code.

      Two things:
      1. Your point - which is "Tell me how you measure me and I'll tell you how I'll behave" - with the addition that it's so natural/visceral a reaction that one doesn't even need to intend to game the measurement, the behavior of the measured will alter
      2. I'm firmly on the opinion that code is value-spent and is not value-created. From this point of view, seems very strange to me for one to use metrics of cost to measure productivity. Efficiency should be measured in how small the number of lines

      • by smelch (1988698)
        Lines of code doesn't make sense because it has nothing to do with productivity. Some problems are harder to solve than others, some require more lines of code and are harder that way, some require complex logic. Some problems can leverage existing code, some can not. Some features just require a lot of skillful placement within the existing code. On top of that, I could write something with very few lines of code using a very resource-expensive technique, is that better?

        No serious developer thinks measu
    • Whatever you measure will be gamed. Measure bugs fixed, and you will find people wasting time listing each tiny variation of a bug. Measure lines of code, you will get spaghetti code. It almost seems better to measure a bunch of things and use a secret formula to determine productivity.

      The important thing to do before trying to figure out how to measure something is to decide what it is that you want.

  • Short answers (Score:4, Insightful)

    by drb226 (1938360) on Thursday November 17, 2011 @03:39PM (#38089466)

    But what happens to all that code once it's written? Do you just ship it and move on? Do your metrics take into account time spent refactoring or documenting existing code? Is it even possible to devise metrics for these activities? Are developers who take time to train and mentor other teams about the latest code changes considered less productive than ones who stay heads-down at their desks and never reach out to their peers? How about teams that take time at the beginning of a project to coordinate with other teams for code reuse, versus those who charge ahead blindly? Can any automated tool measure these kinds of best practices?

    It bitrots. Yes. No. Maybe. Probably. Definitely. Possibly.

    • by ediron2 (246908)

      Lets see...

      • But what happens to all that code once it's written?
        It Bitrots
      • Do you just ship it and move on?
        yes
      • Do your metrics take into account time spent refactoring or documenting existing code?
        no
      • Is it even possible to devise metrics for these activities?
        maybe
      • Are developers who take time to train and mentor other teams about the latest code changes considered less productive than ones who stay heads-down at their desks and never reach out to their peers?
        probably
      • How about teams that take time at the beginn
  • Unless it's your job to make up the metrics.
  • Problem (Score:5, Insightful)

    by bigsexyjoe (581721) on Thursday November 17, 2011 @03:39PM (#38089478)
    A lot of problems rating developer productivity. First, if a system is that good, then managers won't be able to game it to play favorites. Second, writing code for future use is always harder than writing code specific to the problem. Third, almost any metric is going to penalize a simpler solution. (Keep in mind that once you see a simple solution it seems obvious and everyone thinks they'd think of it. Fourth, evaluating developers well would require making the best coders managers, and that rarely happens for several reasons.
    • by snowgirl (978879)

      First, if a system is that good, then managers won't be able to game it to play favorites.

      This is the most likely reason I see why no performance metric that works would actually be picked up. The ideal rating system for a corrupt manager is one where everyone is rated poorly, and then you selectively absolve bad performance metrics for anyone you don't want fired.

      Of course, one also has to be careful in this situation and not have employees discuss their performance ratings, and especially don't have anyone else in management discuss performance rating absolutions. Think of the horrible legal s

      • by c0lo (1497653)

        First, if a system is that good, then managers won't be able to game it to play favorites.

        This is the most likely reason I see why no performance metric that works would actually be picked up. The ideal rating system for a corrupt manager is one where everyone is rated poorly, and then ...

        An ugly world (I know it from my past experience)... I mean the world in which a manager doesn't compete with other managers (on budgets, prestige, etc) based on successes, but is "chosen to be a manager because we need one and... let's see who's the most notable and less-covered-by-shit on the floor".
        In the last case, the chosen person will be the most-full-of-shit-and-willing-to-spread it one... you can imagine how the managed team stinks.

    • The first big problem though is lack of metrics and insight into what developers are doing. I worked on a team and helped implement big brother software on a large development team's computers. The problem was that there was too much data and the company that implemented the software then tried to provide inadequate visualization (e.g. 40% of time was spent on JSP/Java editing, 10% on a web browser, etc.) and just knowing *what* people are working on is not enough. You have to have metrics based on the outp
    • by MadKeithV (102058)

      Second, writing code for future use is always harder than writing code specific to the problem. .

      It's nearly always wrong, too. YAGNI. You Ain't Gonna Need It.

    • by Smallpond (221300)

      On a recent project there were two developerss fixing bugs. "A" would fix a bug in 4 hours rewriting 50 lines of code. "B" would fix the a bug in 9 hours rewriting 1 line of code. They let B go because A was more productive. Funny thing is they never ran out of bugs.

    • by uncqual (836337)
      If you can't trust your first level managers to understand who is contributing w/o a pile of metrics, there is a simple solution -- get new managers.

      ICs will game metrics much more than managers. Good managers rarely have a vested interest in favoring one developer over another except to the extent that one developer contributes more than another to getting reliable systems meeting requirements to the customer.

      If you have metrics that "managers can't game", it means that ICs can game them because they
  • by Synerg1y (2169962) on Thursday November 17, 2011 @03:40PM (#38089490)

    Wouldn't it be the project leader who monitors these on an individual basis? If a coder isn't pulling their weight its up to the project leader to address it up to the point of termination. Above that you have a suit who monitors the project leader's team performance and decides how well the project leader is doing. Of all the places layered management doesn't work, coding is not one of them. It's a challenge to hold a developer accountable because there are so many different approaches to the same problem in coding and a lot have definitive pros and cons.

  • Pfft (Score:5, Insightful)

    by Allicorn (175921) on Thursday November 17, 2011 @03:41PM (#38089492) Homepage

    Can any automated tool measure these kinds of best practices?

    No. The - for the sake of politeness, let's call them "people" - who invested time and effort into devising these schemes have actually built a complete chain of negative productivity by doing so. Remarkable.

  • by Crashmarik (635988) on Thursday November 17, 2011 @03:41PM (#38089494)

    Personally I am always happy with the guy who can get things done with one line of code instead of a hundred, but what I really care about is that objective is met and we don't have a host of bugs that require 10 times the cost of the development just to maintain. Its not hard stuff but it does require common sense and a hard nosed attitude both of which can be scarce commodities these days.

    • by RingDev (879105)

      That's really the key to it. Focusing on objectives.

      Are outputs matching estimates for time? Are programs being deployed/shipped on schedule? Are bug report rates with in acceptable ranges? etc...

      And once you get good at that, start looking for ways to improve on the metrics. Reducing bug counts, getting more accurate estimates, and pushing for shorter (but still reasonable) estimates.

      I knew a programmer who once solved a problem that was going to be addressed with a $100,000 inventory management system, wi

      • It isn't that cut and clear. I can write a bug-free one line perl solution, but what happens when someone needs to maintain it? Objectives come from all directions. Your shop wants quick turn around and maximum profit. The customer wants quick turn around , solid code, AND a cheap price. Your goal is to be productive and get home before dinner. Notice the conflicting objectives.
        • by Bucky24 (1943328)

          I can write a bug-free one line perl solution, but what happens when someone needs to maintain it?

          If it's in perl? You sacrifice to the voodoo gods whenever you need to maintain it, of course.

          • by murdocj (543661)

            Perl is write-only. You ask someone what the code is supposed to do and rewrite it.

    • by MadKeithV (102058) on Thursday November 17, 2011 @04:40PM (#38090140)

      Personally I am always happy with the guy who can get things done with one line of code instead of a hundred, but what I really care about is that objective is met and we don't have a host of bugs that require 10 times the cost of the development just to maintain. Its not hard stuff but it does require common sense and a hard nosed attitude both of which can be scarce commodities these days.

      I am also REALLY happy to have "that guy" that has absolutely shit productivity, but somehow manages to pick up on every time a "solution" is proposed by the rest of the team to a problem that doesn't exist or doesn't matter, and stops THEM from being really efficient at doing the wrong thing.
      I'm also really happy to have "that girl" that doesn't seem to really be doing anything, but take her out of the team and everyone else starts floundering because she's actually constantly helping them be a lot more productive.
      "Meeting the objective" is actually potentially just as bad as any other metric, because it depends on how you define the objective, and meeting it. What the customer asked? What the customer wanted? Or what the customer actually needed?

      • by Crashmarik (635988) on Thursday November 17, 2011 @05:28PM (#38090726)

        Personally I am always happy with the guy who can get things done with one line of code instead of a hundred, but what I really care about is that objective is met and we don't have a host of bugs that require 10 times the cost of the development just to maintain. Its not hard stuff but it does require common sense and a hard nosed attitude both of which can be scarce commodities these days.

        I am also REALLY happy to have "that guy" that has absolutely shit productivity, but somehow manages to pick up on every time a "solution" is proposed by the rest of the team to a problem that doesn't exist or doesn't matter, and stops THEM from being really efficient at doing the wrong thing.
          I'm also really happy to have "that girl" that doesn't seem to really be doing anything, but take her out of the team and everyone else starts floundering because she's actually constantly helping them be a lot more productive.

        "Meeting the objective" is actually potentially just as bad as any other metric, because it depends on how you define the objective, and meeting it. What the customer asked? What the customer wanted? Or what the customer actually needed?

        That guy and that girl are generally called project/team leaders. Don't fret, You raised an important issue. The guy you don't want on the team is the one that comes up with ridiculous edge cases and is needlessly obtuse. Like someone who invents coders that are doing the work of managers and senior managers then complains a measurement tool doesn't capture their contribution, or goes into a recursive loop trying to figure out what the objective should be.

  • by Anonymous Coward on Thursday November 17, 2011 @03:42PM (#38089506)

    If they know their productivity is being measured, it becomes a contest to see who can cook the books the best anyway.

    • If they know their productivity is being measured, it becomes a contest to see who can cook the books the best anyway.

      Yes. And this would be a Good Thing if the metrics actually measured the right thing. The problem is, the metrics represent something that isn't actually what you want, so developers are motivated to do something that isn't actually what you want.

      So if you are going to be using a metric at all, the first questions you should ask yourself is what the metric measures, and if that is actually what you care about. For example, you could measure how many lines of code people produce, and reward people based on t

  • by naroom (1560139) on Thursday November 17, 2011 @03:45PM (#38089544)

    metrics provide little insight

    If only we had some kind of.... metric metric.

    • metrics provide little insight

      If only we had some kind of.... metric metric.

      Sounds like ISO9000.

    • By analogy with metadata - data about data - "metametrics" was what sprang to my mind. And like most things that do after I've had a few jars, it a) already exists [mac.com] and b) is far too shit to deserve such a goshdarn clever name.

  • by slapout (93640) on Thursday November 17, 2011 @03:45PM (#38089548)

    What about all the time you spend not coding: meetings, documentation, training users on how to use the system, working out the design before you start coding, answering emails, sending status reports, filling out time sheets, coordinating work with other developers, coordinating things with others in IT so your program will have a server to run from, being the go-between for the IT server team and the customer when the server goes down, creating database layouts and writing SQL?

    • by lwsimon (724555)

      Surely writing SQL qualifies as coding.

      • by EricWright (16803)

        Not according to some of the developers around here. Of course, these are the types that put "select * from table" into their Java code, then try to filter data in the application layer. Then they whine that their application is slow.

        Seriously.

        • by lwsimon (724555)

          Well, that's just ignorant, then.

          I write SQL most of the day, and in my opinion, it's more difficult to do well than traditional programming. If I'm writing a webapp somewhere, I only have to keep the individual logic i'm working on in my head - with SQL, I may have to know how a single dataset is processed over a half dozen steps to get the result I want.

  • lazy management (Score:5, Insightful)

    by J-1000 (869558) on Thursday November 17, 2011 @03:52PM (#38089602)
    This all comes down to lazy, gutless management. Why take the time to get to know Dave and monitor the quality of Dave's work when we can just look at a spreadsheet at the end of the month? Managers prefer to tinker with automatic analysis software rather than manage.

    Which is more fun, getting a better handle on what Dave is doing, or researching fancy new software tools that might get you all sorts of praise from metric-craving executives?

    Dave's job, which was once about creating a quality product, now shifts to merely satisfying the metrics.
    • by asylumx (881307)
      I hope it's different where you work, but where I work, the management is lucky if they are truly 100% management. But in reality, the managers of technical teams also have to play the role of project manager, technical lead, and sometimes architect. Yes, it sucks, and I wish it weren't that way, but trying to change that is like pulling teeth.

      So even if Mr. Manager has good intentions, he really is prevented by other duties from doing his job to the fullest extent. It sucks, but I wouldn't feel right
  • Repeatability (Score:4, Insightful)

    by 3count (1039602) on Thursday November 17, 2011 @04:02PM (#38089700)
    Metrics are valuable if you do the same thing repeatedly. If you build a new building that is like the previous one, you can collect metrics and compare your performance against history. If you write the same search algorithm again and again, you can collect metrics and compare to see how your performance changes over time. Of course, with software, you never repeat. Somewhere around the third time, you move it into some form of library, reuse it, and start on a fresh problem. Perhaps metrics are helpful in some situations, such if your team keeps repeating the same mistakes, you might find similarity in those mistakes (code smells.) There are plenty of people working on these problems and tools. But, from a management point of view, if you keep doing the same thing, you are doing it wrong, and code metrics are not going to help much.
  • by Riceballsan (816702) on Thursday November 17, 2011 @04:05PM (#38089716)
    I don't get the concept of everything needing to be quantified. Does the team accomplish what the goals of having the team are? Does it get developed in a fair timeframe? Is everyone on the team pulling their own weight, or are there complaints of someone slacking off? In the end if the product works then the team is doing well, if the product isn't there should be at least one hybrid manager/coder that actually works with the team members sees who is committing what and can tell off the bat if there is or isn't a weak link dragging the rest down. Actually putting a pen and paper number on a complex project is silly. Do authors get judged by the number of pages they write in a day, no they get paid by the success or failure of the book. You can't judge by the number of lines of code, bugs per line ratio or anything like that, because it is all subjective and has little to no bearing on the end product.
    • ... Do authors get judged by the number of pages they write in a day, no they get paid by the success or failure of the book. You can't judge by the number of lines of code, bugs per line ratio or anything like that, because it is all subjective and has little to no bearing on the end product.

      Many successful novels, started out as magazine serials, which were paid by the word. Usually, though, they are edited before they are put into book form.

      I've seen a bunch of movies, where I thought, "This would be a great movie, if they just cut half of it, then sped the result up by 1.5x"

      I've often thought that coders should be paid by how *little* they write, as long as it does the job.

  • by Hognoxious (631665) on Thursday November 17, 2011 @04:06PM (#38089732) Homepage Journal

    Not everything that matters can be measured; not everything that can be measured matters.

    -- Einstein (or maybe it was Franklin)
     

  • by fortapocalypse (1231686) on Thursday November 17, 2011 @04:10PM (#38089752)
    Measuring developer output/metrics effectively is a tough problem. Developers could solve it, but if they do, then they have to both change the way that they work and possibly work harder. Developers are smart enough to know that the metrics will be misused, even if the logic used to produce them is valid. Therefore, any solution will be ridiculed by the development community as insufficient, but the degree to which it is ridiculed will lessen as the solution improves. A solution though, is inevitable if development continues.
    • by DalDei (1032670)
      BS
      • That's what I said, but you said it more concisely. If this were code, your solution would probably have rated higher than mine. But, after many (too many) years as a developer now, I can say that programming is about as much an art as carpentry. You can do beautiful things with both, but in the end it is about the artifacts and how they are used. They *are* measurable, and providing decent metrics for software development *is* possible. The biggest problem is that we don't want to admit that it is just a
  • by Surt (22457) on Thursday November 17, 2011 @04:39PM (#38090128) Homepage Journal

    ... you are already doomed. You've gone so far down the wrong path there's no hope of recovery.

    • by T Murphy (1054674)
      I've generally assumed that standardized evaluation is pushed by upper management to provide a baseline for the direct mangers to follow. Sure, a manager should know which of his direct reports are capable and which aren't, but how can you compare one manager's "adequate" to another's? A few numbers won't get you a complete picture, but combine those numbers with the manager's opinions and you can better calibrate performance than you could with either evaluation on its own. A PHB will be a PHB no matter wh
      • by Surt (22457)

        Yeah, that's an upper management failure in my book. They should be connected well enough to know what is going on at least two levels down, not one (enabling them to check the calibration of their direct reports). If that's not possible, the branching factor is probably too high.

  • This sounds like some PHB how only knows how to be manger wanting numbers and having no idea about how developing works.

  • Code metrics are fine if all you care about is raw code production. But what happens to all that code once it's written?

    Seriously, does the author has to ask? Or anyone that does software development for that matter? Sadly, apparently they have to ask.

    You never get to a point where you can say "code once it's written". Code changes after deployment, there are new releases, bug fixes, shit like that which crops up in real life.

    So assuming you are gathering metrics the smart way and in a manner that your particular project and organization, then, you simply keep gathering metrics as you re-factor your code, add new feature

  • by TheWoozle (984500) on Thursday November 17, 2011 @05:35PM (#38090806)

    I think the idea of "productivity" is a hold-over of the Industrial Revolution that does not pertain to many of today's jobs; jobs where the unit of work is hard to define, and ultimately irrelevant. Are you telling me you pick your doctor by how many patients he can see in a day? Probably quite the opposite!

    In terms of software development, I find that the *effectiveness* of a developer is more important, where effectiveness considers the following (not an exhaustive list):
          - Appropriateness of solution
          - Thoroughness of implementation (logging, exception handling, graceful failure, input validation, etc.)
          - Well-written, parsimonious code that is easy to read and descriptive of what it does
          - Works right the first time, no kickbacks from QA or end user

    Give me someone who is effective but slow over someone who craps out junk quickly any day of the week and twice on Sunday! In the end, I don't care about productivity metrics, I care that the end users get a useful piece of software that does what they need with a minimum of headaches.

  • Instead of trying to micro manage your devs ("Oh, Frank, you created 10 lines less than Bob, you slacker!"), focus on the project instead. Check whether your teams meet their deadlines. Pit the teams against each other. And rely on peer pressure within teams to keep the productivity up and slackers from slacking. Are you judging your beancounters by the amount of bookkeeping lines they fill out per day? Do you gauge the productivity of your project managers by the amount of meetings they go to? No, you meas

  • by PJ6 (1151747) on Thursday November 17, 2011 @07:00PM (#38092094)
    This is the guy [wikipedia.org] who first said it.

    Stop listening to the MBA and metrics nutjobs. Don't try to manage your people like the machines they operate.

We can predict everything, except the future.

Working...