Follow Slashdot blog updates by subscribing to our blog RSS feed

 



Forgot your password?
typodupeerror
×
Twitter Businesses Programming IT

Twitter's Tech Lead On Making Software Engineers More Efficient 146

Tekla Perry writes: "Engineering productivity is hard to measure," said Peter Seibel, the tech lead of Twitter's engineering effectiveness group. "But we certainly can harm it." Seibel spoke this week at the @Scale conference in San Jose, hosted by Facebook. He says in large companies one third of software engineers shouldn't be working on the company's products, but should be dedicated to making other engineers more effective. "As an industry we know how to scale up software," he said. "We also know how to scale up organizations, to put in management that lets thousands of people work together. But we don't have a handle on how to scale up that intersection between engineering and human organization. And maybe we don't understand the importance of that. We massively underinvest in this kind of work."
This discussion has been archived. No new comments can be posted.

Twitter's Tech Lead On Making Software Engineers More Efficient

Comments Filter:
  • Scaling (Score:3, Interesting)

    by fluffernutter ( 1411889 ) on Thursday September 17, 2015 @09:11PM (#50546221)
    I'm kind of surprised twitter has more than one Software Engineer.. Don't they just send short messages around and count hash tags?
    • They also have software they build for their customers......tools for managing ad campaigns, recognizing target demographics, etc.
    • Don't they just send short messages around and count hash tags?

      Even less impressive. IRC over HTML, but with the burden of tiered responses instead of linear. So Newsgroups + IRC jammed into a webpage.
  • by Anonymous Coward

    is cost center. Things like this don't exist because the bean counters only understand cost per head and revenue per head. Long term value is an anathema in the world of quarterly profits.

  • MMM (Score:4, Interesting)

    by phantomfive ( 622387 ) on Thursday September 17, 2015 @09:13PM (#50546245) Journal
    The Mythical Man Month [amazon.com] is still extremely relevant on this topic. It's hard for me to take anyone seriously on this topic unless they've read it. Communication is, of course, one of the major challenges with scaling an engineering team.

    One thing the MMM points out is that some engineers are 10 times more efficient than others. The obvious solution is to teach the "others" to do the things the efficient programmers do.
    • by glitch! ( 57276 )

      Yes, communication is a huge cost. Use it as much as needed, but nor more! Excess communication is expensive! If you don't get it, read the book!!

      • Yes, communication is a huge cost. Use it as much as needed, but nor more! Excess communication is expensive! If you don't get it, read the book!!

        MMM today is like reading "the idiot's guide to the volkswagen beetle" and expecting to be able to fix a modern car.

        • by Cederic ( 9623 )

          Yeah, but imagine a world in which no other books told that cars have four wheels, why, and how to assure that they're in the corners.

          Ok, it didn't know about ABS, ECUs, automagic gearboxes and electric windows, but MMM still contains core basic insights that people need to understand and learn.

          Then read the material that's followed, that advances the discipline, and learn from that too.

        • Re:MMM (Score:5, Insightful)

          by Darinbob ( 1142669 ) on Friday September 18, 2015 @02:55AM (#50547335)

          Not really. People keep coming up with new management fads which then fade away and get replaced, but eventually they all come back to MMM and realize it had it right. If you can't understand the concepts in the Idiot's Guide to the Volkswagen Beetle, you will be utterly unable to understand a modern car. Understanding the older stuff is an absolute prerequisite to becoming an expert, even for software, and it's a shame so many incompetent programmers refuse to believe this.

          It's not even just for programming that's dysfunctional, everything gets screwed up with management. I see software people whine that it should be done like other engineering, except that other engineering is dysfunctional as well. Seriously, picking pre-built and well tested components, applying the math, and creating a circuit is a myth too. Most places just cobble things together, they'll argue forever about saving two cents on a board but then go and add a component that's never used. If the boss designs part of it then everyone's too afraid to correct it, or politics gets involved. When it's done, it's tossed over the brick wall to software who are told to work around any defects.

          • by gweihir ( 88907 )

            It's not even just for programming that's dysfunctional, everything gets screwed up with management. I see software people whine that it should be done like other engineering, except that other engineering is dysfunctional as well. Seriously, picking pre-built and well tested components, applying the math, and creating a circuit is a myth too. Most places just cobble things together, they'll argue forever about saving two cents on a board but then go and add a component that's never used. If the boss designs part of it then everyone's too afraid to correct it, or politics gets involved. When it's done, it's tossed over the brick wall to software who are told to work around any defects.

            I completely agree. It is absolutely staggering how bad a lot of electronics are designed. By now I have a close look at everything I buy before even switching it on, and usually I find some gross developer incompetence at work. These people often do not even understand the basics and manage to screw-up the reference designs by "improving" them. And the number of people that cannot even read the basic information in datasheets (like "Absolute Maximum Ratings") is truly astonishing.

          • If you read between the lines, MMM isn't really about software at all; it's about people.

            They haven't changed much.

        • by gweihir ( 88907 )

          Not at all. In fact, the industry keeps making exactly the same mistakes that Brooks describes. You seem to have missed basically all he writes and seem to have focused to the technology in use back then.

        • Mythical Man-Month has stuff that is now common knowledge among everybody, stuff that we've found out since is wrong, stuff that's irrelevant today, and lessons that people still need. That's why some of us keep recommending it.

    • The Mythical Man Month [amazon.com] is still extremely relevant on this topic. It's hard for me to take anyone seriously on this topic unless they've read it.

      Should we take you seriously? It's clear you didn't bother to RTFA.

      • Should we take you seriously?

        You don't have to lol, up to you.

        It's clear you didn't bother to RTFA.

        The article doesn't say much.......just a few quotes, for me it's hard to get a real idea of what the speaker was trying to convey.

        • The article doesn't say much.......just a few quotes, for me it's hard to get a real idea of what the speaker was trying to convey.

          One thing the MMM points out is that some engineers are 10 times more efficient than others. The obvious solution is to teach the "others" to do the things the efficient programmers do.

          This is like saying we can all hit home runs like David Ortiz if we only take the time to learn how he does it

          • I get your point, that there will always be natural variability in programmer ability.
            It doesn't need to be as bad as now, though. There are a lot of simple things top programmers do that bottom programmers tend not to. For example:

            * checking documentation (or actually reading the body of the function) to make sure the function does what you hope before calling it.
            * when you change a line of code, double-check all places affected by that change code to make sure it won't cause problems.

            I've found one
            • I've found one of the biggest problems can be convincing programmers that they should even try to have less bugs.

              Why don't you actually practice the "modern programming technique" you say you know, and convince the "programmers" to write their own unit tests? You claim expertise in the area and yet you don't even mention the #1 technique for reducing bugs and educating developers about them: write their own damned tests. It's much more effective than your nonsense.

              • Unit tests are good but I doubt there is one single technique that will solve all your problems.

                If you force your programmers to write unit tests, they will write lousy unit tests that don't catch any bugs.
                If the programmers are trying to write better code, then they will think of ways to avoid bugs you didn't even think of.
              • If you're testing a tricky algorithm unit tests are great proving the code works and at preventing breakage. Other than that don't waste your company's time. If you want to build test suites with the most bang for the buck spend your time on integration / regression tests.

                I notice you used the word "proven" in regard to unit testing. I've seen plenty of hand waving and evangelism. No proof.

              • I've found one of the biggest problems can be convincing programmers that they should even try to have less bugs.

                Why don't you actually practice the "modern programming technique" you say you know, and convince the "programmers" to write their own unit tests? You claim expertise in the area and yet you don't even mention the #1 technique for reducing bugs and educating developers about them: write their own damned tests. It's much more effective than your nonsense.

                Because if you are going to convince a team of only one thing you may as well convince them of reducing the bugcount. That will get you the unit tests and a whole lot of other things as well. If you convince them to implement unit-tests all you'll get are unit tests.

                You are basically asserting that it is better to convince a programmer to do a subset of $F00. OP is asserting that it is better to convince a programmer to just aim for $FOO instead.

    • Helps to define what "efficient" is also. Times have changed since MMM. People still think that efficient means writing the most cost. But I test things at least preliminary before checking it in, so I find myself slower than others over all, but I also spend less time going back and fixing it later while spending more time fixing other people's hastily checked in code.

    • by gweihir ( 88907 )

      Actually, you cannot make the less efficient engineers as efficient as the most efficient ones. It is not something they learned that makes them exceptional, it is who they are.

      As to the MMM, I completely agree. And it will stay hugely relevant as long as it gets ignored. Current technical management has usually never even heard of it.

      • Actually, you cannot make the less efficient engineers as efficient as the most efficient ones.

        Why not? I've spent plenty of time working to make programmers more efficient.

        It is not something they learned that makes them exceptional, it is who they are.

        It is "something?" What is "something?" If you can not even say what "something" is, then how can you even claim that it can't be taught?

        • by gweihir ( 88907 )

          Experience and observation. Making bad programmers less bad is not the same as making average programmers exceptional.

          • That's a better point than you had before, but you still haven't defined what "something" is that good programmers have, that mediocre programmers don't have lol.

            My experience is the opposite, that it's something that can be taught.
            • by gweihir ( 88907 )

              Maybe your standards are just too low...

              I would actually not blame you, because most coders out there are abysmally bad. Bringing them up to "fair" is quite an accomplishment. They will still be nowhere near "exceptional".

              • You still haven't said what that "something" is which good coders have, which others don't.

                most coders out there are abysmally bad. Bringing them up to "fair" is quite an accomplishment.

                The sad truth.

  • Don't waste time browsing on twitter.
  • Some good ideas in TFA for optimizing the process. While all or some of the ideas may not be appropriate in your environment, the article is worth the read to see how others do stuff.
  • What is most interesting about twitter is that so many people interact with twitter technology on a daily business without even knowing it. Bootstrap is everywhere and beyond twitter.com is the most influential technology that twitter has built for the web.

  • by SuperKendall ( 25149 ) on Thursday September 17, 2015 @09:58PM (#50546433)

    There are so many times that tooling or system issues get in the way of development - we all know you can get into a nice flow of development and it doesn't take much to bring you to of it.

    Someone spending a little time on preparing the runway so to speak, could have a large difference in productivity. He said with ten people you didn't need anyone dedicated to this but I disagree - just make it a rotating position so each person on the team gets a month at a time to fix whatever endemic company system issues bug them the most. You would get some huge gains from software engineers fixing the systems around the software they work on...

    • just make it a rotating position so each person on the team gets a month at a time to fix whatever endemic company system issues bug them the most.

      Good idea. I like the "rotating position" idea, and I've seen it used successfully with a new scrum master each month, but I've never thought of applying it at that situation.

    • by ADRA ( 37398 )

      The idea ideally looks good on paper, but maybe not so well in reality.

      Some people like me will naturally embrace the concept, because I'm generally always thinking of ways to improve my workflows anyways. Add someone else who could come up with completely different scenarios I miss or don't spend enough time on, and you have a good trajectory of improving productivity.

      On the negative side of things, many people are really good at figuring out how to improve production workflows, and then there are those th

      • That's a great point that not everyone may be interested enough to do anything useful with the time... so perhaps a rotating role among all those interested. Although perhaps everyone should have one pass to see if they get interested?

        Or perhaps something like Google's 20% program but for infrastructure work that actually matters... :-)

    • I would actually recommend dedicated optimization resources. I'm on a small architecture team and this is just the sort of thing we deal with.

      I like to ask both developers and business users "What sucks the most?" and "What makes you feel like a robot?". The last question identifies repeating tasks, in most cases automation can be applied.

      Most of the time, developers and business users stick to the status-quo and suffer with crap processes. I always look for process optimizations, in many cases a bit of

      • I think the nice thing about it coming from inside a working team though, is they really know what they are suffering from, or would be willing to fix things that don't seem like a big deal so they wouldn't mention them to a dedicated team, but that in practice they wold fix when it's up to them to change something...

        Perhaps a mixture of a dedicated team (for companies large enough) that also adds rotating members from key teams who are allowed to choose some items to work on?

        I say this because I worked at

  • by Snotnose ( 212196 ) on Thursday September 17, 2015 @10:04PM (#50546453)
    Let my manager set my goals, then prevent other groups/managers from wasting my time. I spent 14 years at Qualcomm ('94 to '08), I never worked harder, got more done, and had more fun than those 14 years. The secret? Management was very good at keeping distractions to a minimum.

    Sadly, from what I hear now that's no longer the case. At the Parade of Lights last, I dunno, November? I met the new boyfriend of a long time friend. He was in his 50's/60's, spent most of his career in Texas at TI, and had been at QC for a year. He hated it. Why? He didn't want to talk about it and I didn't pry.

    About 6 months ago I ran across a guy I knew at QC, he'd been there from the beginning. He said they'd cancelled the Christmas parties (which were epic), and the summer picnics (which were epic if you had kids). He was about to take a 6 month leave of absence and wasn't sure if he'd go back.

    Then 2 months ago QC announces a 15% layoff in 2 months. That 2 months hit yesterday. I'm hesitant to contact folks I knew when I worked there, but it sounds like QC has gone from good, engineering management, to bad, MBA/cronyism management.
    • by Anonymous Coward
      It only takes those MBAs and their accountant cronys getting a foothold before it all goes downhill... Chasing the next quarter's profits to the detriment of long-term profitability.
      • by gweihir ( 88907 )

        Short-term planning is detrimental to long-term survival. Seems to be applicable on a global level, including survival of the current civilization. I predict that in a few centuries, the current time will be known as the "Age of Stupidity", where we had everything to turn this planet into a paradise and failed abysmally because of greed and stupidity.

    • I interviewed a couple recent layoffs from Qualcomm from a particular group. It seems like both were pigeon holed to work on one particular small part of a design, and neither understood anything outside of their small part. In other words, zero knowledge about the system. They get data from A, massage it, and send it to B, if there's any confusion they let some other group figure it out.

    • I had a number of contracts at Nortel and only one good manager. He did exactly what you were saying and kept the stuff out of my way so that I could do my work. One time the manager of one of our client groups sent me an email asking me why it was taking so long to get his update which happened to be a graph of the number of lines in each load build. Basically something that he wanted to use to show how "productive" his team was. The email was copied to a bunch of the senior managers of the company. W

  • No you don't. The industry certainly knows how to solve trivial problems in hugely over complex ways, but those mountains of code quickly become unmaintainable. One of the prime examples of this is Android which tries to solve the near trivial problem of "application launcher" with so much code, entering a to long password can trigger an application crash which unlocks your phone.

    • One of the prime examples of this is Android which tries to solve the near trivial problem of "application launcher"

      If it's SO trivial then why has Microsoft re-written theirs a half-dozen times?

      • Marketing reasons, it's not like the Windows 8 "application launcher" is any better than the Windows 95 one, or for that matter the Windows 3.1 one.

        • by gweihir ( 88907 )

          Indeed. They had to push some new 'better' thing to make even more money. Remember how DOS used to support slightly larger disks every release and the only way to upgrade was to buy the next version at full price? That is what made Microsoft rich.

    • by gweihir ( 88907 )

      I couldn't agree more. The industry certainly has the arrogance to overlook its abysmal failings, but that is about where its capabilities end.

  • If you've got one third of your engineers working at it, why not work on improving the effectiveness of all the employees? It seems like you'd just be locally optimizing and making everything else a bottleneck -- not to mention generating a potential double-ended morale problem where engineers start complaining about the comparative inefficiency of the rest of the company.

    • If you've got one third of your engineers working at it, why not work on improving the effectiveness of all the employees?

      Because the other employees are already actually doing their jobs. Do you see the empty wastebasket? Does your paycheck show up on time?

  • The easiest way to improve efficiency for the younger IT crowd is to take their smartphone away.

    That way their thumbs can be used on a keyboard instead of in IM's.

    Smartphones waste more time than smoke breaks do. I smile every time a tapatalker gets walked out.

    • The easiest way to improve efficiency for the younger IT crowd is to take their smartphone away.

      That way their thumbs can be used on a keyboard instead of in IM's.

      Smartphones waste more time than smoke breaks do. I smile every time a tapatalker gets walked out.

      it's even easier to just ignore them and fire them when they fail their performance review

  • by jandersen ( 462034 ) on Friday September 18, 2015 @03:14AM (#50547417)

    Time for a couple of rants from an old man.

    Engineer: I think this word is used far too much. People are not engineers just because they can write code or install applications or whatever. 'Engineer' should describe a mindset, I think: the sort of analytical can-do attitude that some people seem to have quite naturally, which makes them look at structure and seek rational solutions. The archetypical engineer, to me, would be the magnificantly named Isombard Kingdom Brunel.

    Effective: This, if I remember correctly, describes the fact that something works. An effective solution is one that gives the desired result - it may not be efficient, though, meaning that it works well, quickly or whatever. No, I couldn't be bothered reaching for one of the countless, online dictionaries, because when you are old, you just know you are right, never mind the facts ;-)

    So, that out of the way, and assuming that we are talking about real SW engineers and their efficiency - is that really what holds back projects? Looking at other sectors of production in wider sense, like manufacturing or building construction, the role of the engineer is not actually to whip out as many engineering solutions as possible, but to find the relatively small number of good solutions to structural problems, and to communicate this effectively (and not necessarily efficiently) to the workers. I think it is the same situation we have in software production; calling the everyday coding staff 'engineers' is little more than a bluff - a trick to make people feel they are somehow at a higher level than the common factory worker, so they don't notice how they are actually treated in a rather shoddy way.

    Don't get me wrong, though, I think being a SW worker is a very worthy profession; I am one myself, and I think it is something to be proud of. I also think it is a load of nonsense to say that it is somehow our fault that SW projects don't go as well as management would like - at the end of the day, this is a management issue. As a coder, developer, or if you must, engineer, you should be able to rely on managers to make sure that things like good communication and documentation practices as well as other best practices are worked into the team. I think this is what the article is saying as well, although from the 30secs of skimming I didn't see him actually pointing the finger at management as such.

    • Another oldish man ranting...

      What I get from this article is that he's saying that some proportion of one's R&D output should be on internal tools or process improvements. He claims 30%, I'd say 10% is probably plenty, but he's far more senior and richer than me, so maybe he's right.

      From my sysadmin (now known as devops engineer) position, I can see some really shoddy tools, systems and processes that applications have to use. A few years back I built a system to alleviate a whole load of productivity p

      • by Anonymous Coward

        Another old man here. I don't see programmer productivity as the problem. What I see is layer upon layer of bureaucracy that has become part of project management. I have four places to log time: HR, Scrum Tool, HR Vacation request system and the team calendar in Sharepoint. I used to just send one email to my manager. Well I still have to send out the email because no one reads the previously mentioned systems.

        Add to the fact that apparently there is only way way to solve a problem. That's usi

  • by gweihir ( 88907 ) on Friday September 18, 2015 @03:33AM (#50547469)

    I seriously doubt that. Creation of large pieces of software is abysmally inefficient in basically all examples I have seen. The only exception are situations where a small team of up to 5 people does it all, akin to Brook's "Surgical Team'. These small teams are routinely much more efficient and effective than teams of 20, 30 or 50 people. It has been known since Brook's seminal "the Mythical Man-Month" that creating software does not scale and the only way to accelerate a project is to use better people, but not to use more of them after a certain, very low limit.

    As to scaling up organizations, measuring things is more difficult here, but my impression is that something very similar applies: In a large organization, bureaucracy, meetings, infighting, careerism, etc. kill so much productivity, that it is often surprising they get anything done at all.

    I think this person has no clue what he is talking about.

  • or just code monkeys...

    your software engineers shouldn't really be coding but getting the design of the application right before it comes to coding.

  • "In a group of 1000, he suggests, 255 engineers... "

    Gah, either you are a computer tech or you aren't! It should be 256 out of 1024!

    Damn irrational fractions, how are we to use it in our sub 30 employee company?

  • by pcause ( 209643 ) on Friday September 18, 2015 @05:45AM (#50547817)

    Since the mid 1970's every single study shows that the ideal work environment for programmers is a private office (even a very small one) where they can shut the door and mute a phone. Interruptions are the enemy of software productivity. Despite all the evidence, Silicon Valley pushes the open office concept. The rationale is that this improves collaboration, but studies have shown this is another myth and that collaboration isn't really improved by these office layouts.

    We all know the reason for this is to save money on office space. But the real question to ask is given the talent shortage and the need to improve productivity is this a "penny wise pound foolish" approach.

    • I never thought I'd see the day where I missed cubicles, but that day has come. At least in a cubicle you can't smell if your neighbor has showered or not.
      • I did a PC refresh project at one Fortune 500 company where the software engineer had 40 half-empty cups of moldy coffee sitting on his desk. I got permission from my boss to skip upgrading that user's PC, my boss escalated the issue with the client company, and the engineer got fired. A hazmat team had to dispose of the cups and sanitize the cube.
  • by ei4anb ( 625481 ) on Friday September 18, 2015 @06:17AM (#50547889)
    Those who have never read "The Psychology of Computer Programming (Weinberg, 1971)" are doomed to re-invent small parts of it over and over and over. http://www.geraldmweinberg.com... [geraldmweinberg.com]
  • by GuB-42 ( 2483988 ) on Friday September 18, 2015 @12:45PM (#50550323)

    "We also know how to scale up organizations, to put in management that lets thousands of people work together. But we don't have a handle on how to scale up that intersection between engineering and human organization. And maybe we don't understand the importance of that. We massively underinvest in this kind of work."

    In other words, you know how to hire more managers to manage managers but have yet to find a reason why you hired all these managers in the first place.

One good suit is worth a thousand resumes.

Working...