Become a fan of Slashdot on Facebook


Forgot your password?
Google Networking Programming IT Technology

Ex-Google Engineer Blasts Google's Technology 158

lee1 writes "Dhanji R. Prasanna, an engineer who recently resigned from Google, describes Google's famous back-end infrastructure as a collection of obsolete technologies, designed 10 years ago for building search engines and crawlers. He blasts MapReduce and its closed-source friends as 'ancient, creaking dinosaurs', compared with outside open source projects like MessagePack, JSON, and Hadoop. He also criticizes Google's coding culture, which has become unfriendly to hacker types due to the company's enormous size." I suspect that most people would be happy to have company infrastructure problems as pressing as Google's, though.
This discussion has been archived. No new comments can be posted.

Ex-Google Engineer Blasts Google's Technology

Comments Filter:
  • by Warlord88 ( 1065794 ) on Thursday June 09, 2011 @05:53PM (#36394068)
    Wait.. isn't MapReduce just a framework and Hadoop one of its open source implementation? How is the former a 'creaking dinosaur' as compared to the latter?
  • by Anonymous Coward on Thursday June 09, 2011 @05:54PM (#36394074)

    Than finding fault in what people choose to do.

  • by Nethemas the Great ( 909900 ) on Thursday June 09, 2011 @05:54PM (#36394078)
    news at eleven.
  • by JobyOne ( 1578377 ) on Thursday June 09, 2011 @05:54PM (#36394084) Homepage Journal
    Don't they mean "Hadoop MapReduce?"
  • by blair1q ( 305137 ) on Thursday June 09, 2011 @05:56PM (#36394102) Journal

    At best, he's lost the plot.

    Google still does a totally amazing thing in as-good-as zero time.

    The essence of hacker coding culture is not giving a damn what everyone else is doing and doing things your way anyway, unfriendliness be damned and ignored.

  • I wonder (Score:5, Interesting)

    by ModernGeek ( 601932 ) on Thursday June 09, 2011 @05:56PM (#36394106)
    Does an obsession with following a certain set of methodologies always benefit the bottom line?
    • Re:I wonder (Score:5, Insightful)

      by Kjella ( 173770 ) on Thursday June 09, 2011 @07:26PM (#36395090) Homepage

      Ideally you'd pick the best solution for the problem every time. The problem is then that you end up with very many solutions, and you need ideal people who understand all of them.

      Methodology is a way of narrowing down the variables, here we do it this way and that's what you need to learn too. That way developers become more flexible and components more reusable.

      Then you go too far and try banging the square peg in the round hole. Obsession is not good. Total lack of methodology is not good. As usual the answer is somewhere in between, that kind of fuzzy answer nobody really likes.

      • by xero314 ( 722674 ) on Friday June 10, 2011 @12:31AM (#36396830)

        Ideally you'd pick the best solution for the problem every time.

        This is really all you needed to say. It's just that a lot of people don't understand what exactly makes the "best solution." The best solution is not always the fastest, or the cheapest, or the newest or the coolest. More often than not, the best solution is one that you can build and maintain with your current available resources. If you have an entire .Net shop with say a hundred experienced .Net developers, that you have to keep to maintain the dozen .Net apps you already have, then the best solution would probably not be to build your web app in PHP. Sure the "best solution" for regex processing might be Perl, but you would be a fool to throw that into your all Java stack without some really compelling reason.

        In Googles case, replacing you well understood and working system with a new, possibly better, implementation, is not necessarily the "best solution."

    • by cshark ( 673578 ) on Thursday June 09, 2011 @10:25PM (#36396234)

      When it comes to methods, you pick the set that makes the most sense, and stick with it. It's like I keep telling my boss. You can't just copy and paste a new method into your culture, and expect it to work properly. Choice of methods should come from trial and error, and a commitment to doing things right. What right means to you may vary. But if you're switching out your method for every solution... you're not using a method. You're playing chicken.

      If you choose wisely. Yes.
      Those methods do impact the bottom line.

      But only if you've picked the right method, and implemented it properly.

  • Haha (Score:4, Interesting)

    by unity100 ( 970058 ) on Thursday June 09, 2011 @06:02PM (#36394168) Homepage Journal
    the guy got so accustomed to good that his standards seem to have perpetually got raised. he thinks google's state is 'bad'. lucky him.
    • Re:Haha (Score:4, Insightful)

      by shutdown -p now ( 807394 ) on Thursday June 09, 2011 @07:51PM (#36395308) Journal

      My thoughts exactly. He sounds like someone in academia land - the same guys who think that most industry players are dinosaurs for not using Haskell and similar bleeding edge stuff.

      In truth, all big players have to be reasonably conservative in the adoption of technology, because otherwise the risks become unmanageable. For example, Google standardized on Java, C++ (or rather a fairly conservative subset thereof), and Python - all mature, established platforms. On the other hand, Google does actively participate in development of those; not sure about C++, actually, but they definitely have a strong presence in Java development process, and Python - well, Guido is a Google employee. And then there is experimental stuff, such as Go, being slowly adopted.

      Few companies can boast being that far ahead from the bulk (think of all the companies still on Java 1.4 for in-house development, for example). If he's not content with this arrangement, then he shouldn't work for a company of that size in the first place. Find some startup where they can implement crazy ideas just like that, just to see if it works or not (and possibly fail if it doesn't, but on that size, who cares?).

  • by antifoidulus ( 807088 ) on Thursday June 09, 2011 @06:03PM (#36394184) Homepage Journal
    This guy seems to have never lost his academic mindset, it's not at all surprising, or bad really, that Google is keeping around old technology. Guess what, they have this thing called operations where they pretty much have to be up 24x7 so that they can serve customer requests. They cannot just start dumping infrastructure that:
    a) work and
    b) they have invested significant amounts of money in
    just because some new technology came around. If everybody in industry did that, it would be absolute chaos and nobody would be able to get anything done. This is just as true in computers as it is with steel mills.

    Now compare this with academia, where they have no real customer base to speak of. They can constantly push the boundaries, try new technologies, change their infrastructure etc. That seems to be where this guys mindset remains.

    Note that I'm not bashing academia as being out of touch with "reality" or anything like that, the entire POINT of academia is to push these boundaries, industry exists to take these advances, combine them with their own, and then deploy them in an operationally efficient manner.
    • by back@slash ( 176564 ) on Thursday June 09, 2011 @06:17PM (#36394346)

      Was going to post something similar. I've observed that at some point most developers go from "must always use the latest and greatest" mindset out of college to "if it ain't broke don't fix it" mindset that comes with a few gray hairs. Just like any company Google would need to justify the cost of upgrading to newer technologies against any new capabilities the technologies would enable to either save costs or drive new revenue. If that cost can't be justified they could be running on existing technology for a long time (of course ensuring that you can hire people that know enough about these technologies is another story..)

      • by wrook ( 134116 ) on Friday June 10, 2011 @12:42AM (#36396870) Homepage

        I've observed that at some point most developers go from "must always use the latest and greatest" mindset out of college to "if it ain't broke don't fix it" mindset that comes with a few gray hairs.

        Yeah. You get to a point where you realise that eating different food doesn't necessarily result in making your shit any more appealing. One thing about using the bleeding edge is that it pretty much guarantees that you don't know how to use it properly. You don't have any experience to tell you what is appropriate and what isn't. You don't even have a lot of examples to show you how to avoid issues and how to write code the way everyone else does (because everyone else *doesn't* yet). The industry is littered with new projects that suck just as hard as the things they were supposed to replace.

    • by trout007 ( 975317 ) on Thursday June 09, 2011 @06:38PM (#36394588)

      What about government? I am an engineer that works for the Feds. I use CAD software daily and had a 5 year old workstation that ran perfectly. But because of some BS contract change with the IT department they "upgraded" me to a new machine. Only it runs the software worse with more errors and crashes. Imalso have this nice feature where the first time I load a .PDF file it crashes but when I open it a second time it works. It's kind of like star trek where only the even numbered films are good. Anyway since I am not and cannot get to be admin on the machine and the contract they hired is clueless I am stuck with this POS machine.

    • by stabiesoft ( 733417 ) on Thursday June 09, 2011 @06:58PM (#36394794) Homepage

      Agree completely, I think he should work at facebook. I hear they roll stuff out all the time for users to test for them. Skype seems to be following that model as well. I am quite satisfied that google operates as they do as I rely on them being there and working for search and so far they have never gone offline. I'm glad he left.

    • by geniice ( 1336589 ) on Thursday June 09, 2011 @07:49PM (#36395290)

      Except funding issues in academic environments means that a lot of old stuff that keeps working is kept on. Even if it is horrific VB hacks that run on windows 95.

    • by kangsterizer ( 1698322 ) on Thursday June 09, 2011 @10:40PM (#36396320)

      Does it matter?
      The point is, Google was once pushing technology. And now, they are not, at least, in these very fields.

      None of your I-like-Google post goes against what the guy says. In fact, you're supporting his claims.

      Neither are bad things - but I can understand an engineer who wants to use the latest tech or invent new innovative tech instead of using 10 year old stuff.

      • by argStyopa ( 232550 ) on Friday June 10, 2011 @08:25AM (#36398778) Journal

        "The point is, Google was once pushing technology. And now, they are not, at least, in these very fields."

        But that's ENTIRELY natural, thus the tenor of posts here, mostly "so?".

        The fact is that what begins as fire-breathing, risk-taking revolutionary, with success and age, ends up being a staunch, reactionary defender of the status quo.

        It's the same for Google as it was for the United States and even (in general terms) individuals.

        To your point - I think the guy's comments smell more of former-employee carping than a reasoned engineering evaluation of Google's use of tech.

      • by Xest ( 935314 ) on Friday June 10, 2011 @08:52AM (#36399016)

        No one is saying he's wrong in wanting that.

        What people are saying is it's fucking silly of him to whine about it, and that if he doesn't understand why Google is loathe to replace it then perhaps he's not hot shit like he thinks he is.

        There's good reason why he can't get what he wants out of Google, and if he doesn't understand those reasons (i.e. tried and tested stability) then perhaps he's just a shit developer seeking attention. In which case, why the fuck should anyone care what he has to say?

      • by benhattman ( 1258918 ) on Saturday June 11, 2011 @11:28PM (#36415390)

        What do you even think that means? Pushing technology. Pfft. Do you mean new technology? I hate to break it to you, but almost all consumer grade technology is old stale bread crumbs. Do you think I mean, is anything that Facebook, Twitter, Groupon, or Amazon doing cutting edge?

        The fact of the matter, Google exists to make money for their shareholders and employees. If they decide to train a billion squirrels to sort nuts in such a fashion that when my query is received it provides 10,000 relevant matches in 0.5 seconds, then more power to them.

        As disappointing as this fact is, there are two kinds of technology companies: successful companies which leverage mature technology (hopefully in novel or innovative ways), and those force their customer base to use bleeding edge technology until they go out of business. Go down the lists. MSFT, Oracle, Intel, Sony. Those companies rarely do anything but iterate their products. Apple and Nintendo use established safe technology; what innovation they provide is almost exclusively in user interfaces. IBM makes money from established technology and generally uses the bleeding edge stuff as a commercial for how great they are.

        Even Xerox and Bell from back in the day, with their vaunted research labs, sold the stable products.

        You can't profit off of bleeding edge stuff. You can just hope to be involved in it, so that when the day comes that it is mature your company is in a place to be the first entrant.

  • Seriously? (Score:5, Insightful)

    by fyzikapan ( 1223238 ) on Thursday June 09, 2011 @06:09PM (#36394250) Homepage
    10 year old tech? My last job was using a bunch of stuff originally built in the 1970s! This guy needs to get a grip on reality. You don't throw out something that works, even if it's a bit kludgey sometimes, simply because there's some fancy new thing.
  • Translation (Score:5, Funny)

    by crow_t_robot ( 528562 ) on Thursday June 09, 2011 @06:11PM (#36394274)
    Heaven SUCKS! The noise of all the angel's wings flapping is making it difficult to truly enjoy my harem of supermodels, swimming pool filled with wine and diamond roller skates! I'M OUTTA HERE, SUCKERS.
  • Actual Link (Score:3, Informative)

    by batrick ( 1274632 ) on Thursday June 09, 2011 @06:13PM (#36394310)
  • by retroworks ( 652802 ) on Thursday June 09, 2011 @06:14PM (#36394330) Homepage Journal
    Wrote former burger flipper Ferd McFaddle in his blog.
  • by MurrayTodd ( 92102 ) * on Thursday June 09, 2011 @06:20PM (#36394384) Homepage

    His book on Dependency Injection is one of the few recent computer books I had to go through carefully, and with notepad and highlighter in hand. His work on Google Guice is really notable. This ain't just some Microsoft-bound disgruntled guy.

    But it's not necessarily surprising. I'm not very familiar with it, but Google's Wave was one of those allegedly killer technologies that just didn't get the corporate support it needed to reach its potential as a disruptive technology. Still, there's a possible tone of sour grapes here. Hard to know.

    I'll just say this: I would love to have the privilege to work with someone of his caliber.

    • by gbjbaanb ( 229885 ) on Thursday June 09, 2011 @06:48PM (#36394704)

      chances are he's just another frustrated genius type, pissed off that everyone else couldn't (or he thinks, wouldn't) see everything his way.

      Often such arrogance goes hand in hand with hubris as he comes up with amazing stuff that sounds great, perfect in theory, but fails somewhat in the real world where we all know imperfection is good enough.

      Maybe he was kicked out, maybe he left. either way, now's his chance to do all that wondrous stuff he dreams of. Slating Google is just unproductive.

    • His book on Dependency Injection is one of the few recent computer books I had to go through carefully, and with notepad and highlighter in hand

      Is that a good thing? Personally, If I am going through a book closely with a notepad and highlighter, I am either:

      • editing somebody's work
      • trying to extract the useful information from a poorly written work, such as Cocker's Arithmetick []
      • Trying to extract meaning from a well written work that assumes knowledge of some topic I am not familiar with (thus I am not the target audience)
      • or making note of all the blatant mistakes/problems in preparation for publishing an article demonstrating the author's sheer incompetence
  • by phoebe ( 196531 ) on Thursday June 09, 2011 @06:22PM (#36394400)

    I think many other companies would be happy to have remotely 'ancient, creaking dinosaur' technology. I ponder to think what the authors opinion of infrastructure technology in the rest of the world that would be lucky to be only 15-20 years old.

    Citing MessagePack is certainly surprising as that particular technology is significantly worse than Google Protocol Buffers, the website is littered with bad test procedures and many errors. Google's serialization doesn't have the speed of say TIBCO's QForms or the compactness of Reuters RForms but it is pretty clear from their documentation that flexibility and easy management were preferred goals over utmost highest performing technology.

  • by rickb928 ( 945187 ) on Thursday June 09, 2011 @06:25PM (#36394434) Homepage Journal

    All my ex'es blast me too. And for good reason. According to them.

    The moved on to another loser of their own making. How's that workin for ya, honey? Gotten through his six months worth stupid stories yet? Ask him how he likes your hyena-like laugh. Later, babe. See if your latest will move you into your new trailer. Love your new hair.

    ps - Hackers struggle in almost every corporation. Something about breaking stuff and not valuing availability over innovation. So do I want a hacker mentality ruling at my bank? Depends. Keep them away from the transaction system and the website, so I can get in and get my money, ok? The ops guys hose it up enough already.

  • by Anonymous Coward on Thursday June 09, 2011 @06:31PM (#36394498)

    I worked at a company where the people deploying Hadoop were like him. Most everyone doing actual work wanted to strangle them while yelling "it works fine, stop upgrading every other month and causing hundreds of hours of unnecessary work." Especially since most of the changes made things worse (read: bug testing, who needs bug testing) or added features no one wanted. But they were "cutting edge" so the academic minded fools in charge just kept putting them into production. The features that would have helped productivity were of course too boring for them to bother with.

  • by HenryKoren ( 735064 ) on Thursday June 09, 2011 @06:35PM (#36394550) Homepage

    Because a simple name-value pair object & array data serialization format is wayyyyyy better than a distributed data storage and retrieval system. Right...

  • by alexmin ( 938677 ) on Thursday June 09, 2011 @06:48PM (#36394714)

    He will see the point in using those "obsolete" (read stable) technologies in 10 years: the goal of business is to make money, not make work by constantly upgrading.

  • Make Money Fast (Score:2, Insightful)

    by Animats ( 122034 ) on Thursday June 09, 2011 @07:12PM (#36394952) Homepage

    I think I see what he's getting at. In the past few years, a few people have gotten rich doing something really dumb, but popular and scalable. Angry Birds, Farmville, and Twitter come to mind. (Not Facebook; there's a lot of heavy machinery behind the scenes making that go.) Google hasn't been doing that kind of thing. Some people think that's a problem.

    In reality, Google has exactly the opposite problem. They've been frantically introducing cool "products" that don't make money. Meanwhile, quality has slipped over at the search engine, which generates almost all the revenue.

    Basic truth: ads on a search engine are presented when someone is actively looking for something relevant to the ad, which means there's a reasonable chance of a sale. Ads on almost everything else are annoying interruptions. Google is in a really good business, one much better than "social". It's worth bearing in mind that Facebook only generates about 15% of the revenue of Google.

    So having APIs which let people do quick little apps isn't going to affect Google's bottom line much. Sorry, hacker types.

  • by CaroKann ( 795685 ) on Thursday June 09, 2011 @07:29PM (#36395124)
    Sounds like to me that Google's all grown up now.
  • by mbkennel ( 97636 ) on Thursday June 09, 2011 @08:17PM (#36395494)

    "describes Google's famous back-end infrastructure as a collection of obsolete technologies, designed 10 years ago for building search engines and crawlers."

    Is this supposed to be a problem?

    What makes Google money? Search and search ads.

    Mature technology designed for search engines by a company with a billion in revenue per year from search ads is probably very good for making money from a search engine.

  • by simoncpu was here ( 1601629 ) on Friday June 10, 2011 @03:44AM (#36397620)
    I know some will see this as a flamebait or something, but the real "sluggish, overengineered Leviathans" in their system is Java.
  • by supersat ( 639745 ) on Friday June 10, 2011 @05:19AM (#36397976)
    Google's "Caffeine" update to their search engine actually scrapped MapReduce [] because they want to do real-time indexing, which is not feasible with a batch system like MapReduce.
  • by drolli ( 522659 ) on Friday June 10, 2011 @05:21AM (#36397990) Journal

    That never occured before. Employee trys to build sth on an infrastructure not created for it and fails. Instead of positively saying: "ok, thats not the right place to do it", he says "this place is wrong totally".

    I mean, i would be a little bit nervous if googles software infrastructure was significantly younger than 10years.

    I think for infrastructure its not unusual to use technologies from 20years back, and only introduce new things very slowly.

  • by kriston ( 7886 ) on Saturday June 11, 2011 @12:34AM (#36408892) Homepage Journal

    You want fun?

    Apply for a software engineering or systems engineering job at Google and see if you can get a phoner with a member of the GMail team. You will sustain an injury as you fall of your chair laughing at how the interviewer describes to you how GMail is "designed," as if it really were "designed."

    Google is a superb search engine, but that's about it.

Research is what I'm doing when I don't know what I'm doing. -- Wernher von Braun