Ex-Google Engineer Blasts Google's Technology 158
lee1 writes "Dhanji R. Prasanna, an engineer who recently resigned from Google, describes Google's famous back-end infrastructure as a collection of obsolete technologies, designed 10 years ago for building search engines and crawlers. He blasts MapReduce and its closed-source friends as 'ancient, creaking dinosaurs', compared with outside open source projects like MessagePack, JSON, and Hadoop. He also criticizes Google's coding culture, which has become unfriendly to hacker types due to the company's enormous size." I suspect that most people would be happy to have company infrastructure problems as pressing as Google's, though.
MapReduce vs Hadoop (Score:1)
Re:MapReduce vs Hadoop (Score:5, Funny)
I'm sure he's pushing his own brand new technology, Sour Grapes
Re: (Score:3)
While I know google doesn't require apologists, I always have a little chuckle when I read articles like this. So what if it's 10 year old tech, it seems to be working well for them. Sour grapes and a healthy serving of plug-my-own-products.
Re:MapReduce vs Hadoop (Score:4, Insightful)
The fat is that Google's products, by and large, work. If the whole damned thing is floating on top of Pentium IV's with 2gb of RAM, or whatever, does it matter? It's a moronic position, and clearly one formulated as an excuse to show how superior what he's doing is. What a prick.
Re: (Score:2)
The issue is that they might not have such a big lead on any competition than people may think. That's an amazing thought because Google are still generally seen as untouchable. What the guy's saying is that a modern, nimble company can come in and eat some of Google's cake.
Re: (Score:3, Insightful)
The big lead isn't the equipment. It's the AdSense algorithms. This guy is blowing smoke out his ass.
Re: (Score:3)
It matters because successful organizations often don't feel the need to continue innovating. "After all Google Is print money, why change?" And small little problems start cropping up. This feature becomes difficult to maintain. This product starts to lag behind the competition a little bit. But all of it can be ignored since they're still so successful.
Eventually some little whipper snapper comes along and eats their lunch (usually founded by 'sour grapes' ex-$organization members).
It becomes par
Re: (Score:2)
First of all, I don't buy this guy's claims. They seem to stem from a combination of sour grapes and self promotion. He strikes me as a dishonorable twat.
But the reason for me mentioning that algorithms are the source of Google's success is to indicate that it doesn't matter if they're doing it on ten thousand 10 year old commodity computers or on an IBM supercomputer cluster. The innovation isn't the hardware.
Re: (Score:2)
First of all, I don't buy this guy's claims. They seem to stem from a combination of sour grapes and self promotion. He strikes me as a dishonorable twat.
Your posts (long string of them) amount to nothing more than a series of content-free ad hominum attacks. If you are a Googler, you discredit the organization.
For what it is worth, author's observations mirror my own, particularly in regard to the inbred culture of turf defense. At Google, endless tweaking is the order of the day and shiny trumps substance. This seems to be some sort of main sequence for technology companies. It is rooted in the belief that "we're making more billions than ever before there
Re: (Score:2)
Rewriting from scratch is bad.
Tweaking something that works is good.
Re: (Score:2)
Rewriting from scratch is bad.
Never rewriting anything is far, far worse.
Tweaking something that works is good.
Not when structural problems remained unfixed because of that.
Platitudes like the ones you offer are part of the explanation why Google's engineering culture has weakened.
Re: (Score:2)
Have you considered that he is probably right? Google was an innovator ten years ago, but the philosophy of don't fix what isn't broken means that eventually they are going to get a fat layer of crust.
Yes, Google products work, I don't remember him saying that they don't, that's not part of the conversation. His point is that Google's technologies are aging and that there's better stuff now, is it really inconceivable that technology has advanced in the last 10 years? Just take his word and move along, ther
Re: (Score:3)
I thought the real insight in the article came from the piece about the coding culture. Staking out territory and maintaining complete control of the design and implementation of systems. If that's the case, is it any wonder the systems are obsolete? The way I see it, this article is really less about how antiquated Google's systems are, and more about how pig headed the culture can be. Made sense to my feeble warped mind anyway.
Re: (Score:2)
Re: (Score:2)
Also how does it compare to MongoDB [slashdot.org]? Thanks to (somebody) for the original reference to msgpack. You can't go to sleep for 15 min without getting behind on something new.
Re: (Score:2)
Not really. MapReduce is more "an idea" with multiple possible implementations and API.
For instance, Hadoop is an implementation of MapReduce. MR-MPI is an other one. But they are totally incomparable in term of programming interface and obtained performance.
There is probably as many differences between Hadoop and Google's MapReduce (or whatever the official name is) than between windows 7 and macos X.
Re: (Score:3, Insightful)
To be honest, it sounds like a guy who thought that he knew best, and wanted to just mash bad code out... Google told him to write good code or fuck off... he chose the latter.
Comment removed (Score:4, Interesting)
Re:MapReduce vs Hadoop (Score:4, Insightful)
Re: (Score:2)
And I have seen internally a next gen HPCC that out performs hadoop by 4X - and I suspect Google's internal systems are as good if not better.
Re: (Score:2)
There is nothing easier... (Score:2)
Than finding fault in what people choose to do.
Former Employee Has Chip on Shoulder... (Score:5, Funny)
Re:Former Employee Has Chip on Shoulder... (Score:5, Interesting)
Not just former employee:
Probably angsty nobody liked his baby.
Re: (Score:2)
Hey, I would be too if I made the mess that is Wave. Difference is I'd be pissed at myself for ignoring basic design principles and writing something no one wanted.
Re:Former Employee Has Chip on Shoulder... (Score:4, Informative)
Wave was misguided. It had really cool tech and it did have potential...
but where they failed was they couldn't even tell people HOW to use or even WHY they'd want to.
I thought of things I could use it for, but when telling others about how it worked, not only could I not explain it well (who could?), no one really saw the point of it.
FAIL is an understatement.
Re: (Score:2)
I thought of things I could use it for, but when telling others about how it worked, not only could I not explain it well (who could?), no one really saw the point of it.
My thoughts as well. Seemed to be the 21st Century equivalent of Lotus Notes.
Re: (Score:2)
It was unfortunately the late 20th century equivalent of Lotus Notes.. ...and was a solution looking for a problem ...
Re: (Score:2)
.and was a solution looking for a problem ...
Nothing wrong with this - that just means it was a marketing failure.
Re: (Score:2)
I thought of things I could use it for, but when telling others about how it worked, not only could I not explain it well (who could?), no one really saw the point of it.
My thoughts as well. Seemed to be the 21st Century equivalent of Lotus Notes.
Right, because no one ever bought Lotus Notes.
Re: (Score:3)
Right, because no one ever bought Lotus Notes.
Not voluntarily, anyway.
Re: (Score:2)
Oh snap!
Re: (Score:2, Interesting)
I read his posting as well as those of his friends who've also recently quit, and one of them had a better critique that seemed to explain why Wave turned out as the jumbled mess that it was. From the post:
If you pitch an idea or a project to Larry and Sergey, their feedback is quite easy to anticipate. They'll tell you you have to solve the problem in a more generic way. ... Come up with something that solves everything!
...
Wave is a case in point. Wave started with some fairly easy to understand ideas about online collaboration and communication. But in order to make it more general and universal, more ideas were added until the entire thing could only be explained in a 90 minute mind blowing demo that left people speechless but a little later wondering what the hell this was for.
To me, that perfectly explains why Wave turned out the way that it did. Rather than building a simple tool and adjusting it based off of how people used it, they tried to come up with the single solution that solved every problem. They ended up producing something that solved many interesting technical ch
Re: (Score:2)
if this attitude is as pervasive as it's claimed, I can see it being incredibly frustrating to work there as an engineer
It is, and it was.
Re:Former Employee Has Chip on Shoulder... (Score:5, Insightful)
I think this is the lesson we can all learn from Facebook. To succeed a technology doesn't have to be particularly well built or ambitious in solving problems. It just has to be easy enough to use that everyone can understand it and use it without too many problems. Facebook has so many users because it can be summed up in very simple terms, 'it's for sharing photos with your friends', 'it lets you see what people are doing'.
Google Wave may have been a technological marvel and a solution to all kinds of problems but when it launched no-one knew what it was for, so nobody bothered using it.
Re: (Score:2)
Wave is actually a great idea, just poorly implemented.
They should have sticked with developing it as a protocol for a few more years, and streamlined it so it's fast, simple, and resource-light so even mobiles can easily use it in a mobile browser.
Then when they built applications on top of it which make the internet instant instead of write-send-write-send, they would really have something new and exciting.
Re: (Score:3)
Probably angsty nobody liked his baby.
I heard 5 people liked it. That's more than anyone can say for MySpace today.
Re: (Score:1)
Actually the blog post itself says almost nothing but great things about working at Google, with a few criticisms interspersed about how he thought they could be doing better. Of course who writes an article "Working at Google, mostly swell, a few complaints"
Re: (Score:2)
Or Today: How do i recommend myself to future employers?
Hadoop? (Score:2)
Re: (Score:2)
The original paper from google was "MapReduce" of which Hadoop is an open source implementation of the concepts described in the paper.
http://labs.google.com/papers/mapreduce.html [google.com]
Re: (Score:2)
But the guy is not talking about the research paper, he's probably talking about Google's implementation of those concepts, which wouldn't be surprising if it were called simply, "MapReduce."
-dZ.
Re: (Score:2)
That may be true, but the whole purpose of most of these comments is to point out, in a very pedantic, nerdy way, that "Map Reduce" is just a specification not an implementation, and by doing so discredit the comments of the ex-googler when he compared it to Hadoop.
It seems clear that he was comparing the actual implementations.
-dZ.
Re: (Score:2)
Sour grapes. (Score:2)
At best, he's lost the plot.
Google still does a totally amazing thing in as-good-as zero time.
The essence of hacker coding culture is not giving a damn what everyone else is doing and doing things your way anyway, unfriendliness be damned and ignored.
Re: (Score:2)
And also on the flipside, thus, why they produce code that works only 70% of the time and will rapidly become 1 tenth as productive once they need to extend things at all.
I wonder (Score:5, Interesting)
Re:I wonder (Score:5, Insightful)
Ideally you'd pick the best solution for the problem every time. The problem is then that you end up with very many solutions, and you need ideal people who understand all of them.
Methodology is a way of narrowing down the variables, here we do it this way and that's what you need to learn too. That way developers become more flexible and components more reusable.
Then you go too far and try banging the square peg in the round hole. Obsession is not good. Total lack of methodology is not good. As usual the answer is somewhere in between, that kind of fuzzy answer nobody really likes.
Re: (Score:2)
Ideally you'd pick the best solution for the problem every time.
This is really all you needed to say. It's just that a lot of people don't understand what exactly makes the "best solution." The best solution is not always the fastest, or the cheapest, or the newest or the coolest. More often than not, the best solution is one that you can build and maintain with your current available resources. If you have an entire .Net shop with say a hundred experienced .Net developers, that you have to keep to maintain the dozen .Net apps you already have, then the best soluti
Re: (Score:2)
When it comes to methods, you pick the set that makes the most sense, and stick with it. It's like I keep telling my boss. You can't just copy and paste a new method into your culture, and expect it to work properly. Choice of methods should come from trial and error, and a commitment to doing things right. What right means to you may vary. But if you're switching out your method for every solution... you're not using a method. You're playing chicken.
If you choose wisely. Yes.
Those methods do impact the bot
Haha (Score:4, Interesting)
Re:Haha (Score:4, Insightful)
My thoughts exactly. He sounds like someone in academia land - the same guys who think that most industry players are dinosaurs for not using Haskell and similar bleeding edge stuff.
In truth, all big players have to be reasonably conservative in the adoption of technology, because otherwise the risks become unmanageable. For example, Google standardized on Java, C++ (or rather a fairly conservative subset thereof), and Python - all mature, established platforms. On the other hand, Google does actively participate in development of those; not sure about C++, actually, but they definitely have a strong presence in Java development process, and Python - well, Guido is a Google employee. And then there is experimental stuff, such as Go, being slowly adopted.
Few companies can boast being that far ahead from the bulk (think of all the companies still on Java 1.4 for in-house development, for example). If he's not content with this arrangement, then he shouldn't work for a company of that size in the first place. Find some startup where they can implement crazy ideas just like that, just to see if it works or not (and possibly fail if it doesn't, but on that size, who cares?).
Re: (Score:2)
FP is not bleeding edge, and neither is strong typing. But Haskell is more than just a functional strongly typed language. ML is also that, but it's definitely not bleeding edge.
Haskell is on the forefront of type system research right now - a lot of various experimental stuff is there, especially if you look at GHC rather than language standard. STM is also actively investigated there. Finally, due to the language being pervasively lazy, it offers some unique optimization challenges.
Re: (Score:2)
Didn't say Haskell is the only thing there. Scala is also very much bleeding edge, though it is mitigated by the fact that the core of the language is less radically different from what people are used to, and so you can use it as a kind of "Java++" while avoiding the complicated and evolving areas.
Re: (Score:2)
... and? Haskell of today is not the same thing as it was when it all got started (neither is Python, but its evolution went at a very different pace).
Re: (Score:2)
Next up, grammar class!
Academia v. industry (Score:5, Insightful)
a) work and
b) they have invested significant amounts of money in
just because some new technology came around. If everybody in industry did that, it would be absolute chaos and nobody would be able to get anything done. This is just as true in computers as it is with steel mills.
Now compare this with academia, where they have no real customer base to speak of. They can constantly push the boundaries, try new technologies, change their infrastructure etc. That seems to be where this guys mindset remains.
Note that I'm not bashing academia as being out of touch with "reality" or anything like that, the entire POINT of academia is to push these boundaries, industry exists to take these advances, combine them with their own, and then deploy them in an operationally efficient manner.
Re:Academia v. industry (Score:5, Insightful)
Was going to post something similar. I've observed that at some point most developers go from "must always use the latest and greatest" mindset out of college to "if it ain't broke don't fix it" mindset that comes with a few gray hairs. Just like any company Google would need to justify the cost of upgrading to newer technologies against any new capabilities the technologies would enable to either save costs or drive new revenue. If that cost can't be justified they could be running on existing technology for a long time (of course ensuring that you can hire people that know enough about these technologies is another story..)
Re: (Score:3)
I've observed that at some point most developers go from "must always use the latest and greatest" mindset out of college to "if it ain't broke don't fix it" mindset that comes with a few gray hairs.
Yeah. You get to a point where you realise that eating different food doesn't necessarily result in making your shit any more appealing. One thing about using the bleeding edge is that it pretty much guarantees that you don't know how to use it properly. You don't have any experience to tell you what is appropriate and what isn't. You don't even have a lot of examples to show you how to avoid issues and how to write code the way everyone else does (because everyone else *doesn't* yet). The industry is
Re: (Score:2, Offtopic)
What about government? I am an engineer that works for the Feds. I use CAD software daily and had a 5 year old workstation that ran perfectly. But because of some BS contract change with the IT department they "upgraded" me to a new machine. Only it runs the software worse with more errors and crashes. Imalso have this nice feature where the first time I load a .PDF file it crashes but when I open it a second time it works. It's kind of like star trek where only the even numbered films are good. Anyway sinc
Re: (Score:2)
>>Imalso have this nice feature where the first time I load a .PDF file it crashes but when I open it a second time it works
That my friend is a problem, you most likely have a pirated version of the software, or a related Adobe product, rather sure of it.
I learned that by helping a friend out after his son installed some downloaded crap and added Photoshop. had to do a clean install
go paid a WD passport for all my problems.
Re: (Score:2)
It sounds like you've got bad RAM or your machine is over-overclocked. Wanna bet your supplier used substandard RAM or juiced the clocks to get a cheap MB to meet the specs?
Re: (Score:2)
What is funny that I got three reasonable answers for free on this forum while all I got was blank stares from our contracted IT department.
Re: (Score:2)
Sounds like they moved you from Unix to Windows ;-)
Re: (Score:3)
Agree completely, I think he should work at facebook. I hear they roll stuff out all the time for users to test for them. Skype seems to be following that model as well. I am quite satisfied that google operates as they do as I rely on them being there and working for search and so far they have never gone offline. I'm glad he left.
Re: (Score:2)
Except funding issues in academic environments means that a lot of old stuff that keeps working is kept on. Even if it is horrific VB hacks that run on windows 95.
Re:Academia v. industry (Score:5, Insightful)
Does it matter?
The point is, Google was once pushing technology. And now, they are not, at least, in these very fields.
None of your I-like-Google post goes against what the guy says. In fact, you're supporting his claims.
Neither are bad things - but I can understand an engineer who wants to use the latest tech or invent new innovative tech instead of using 10 year old stuff.
Re: (Score:2)
"The point is, Google was once pushing technology. And now, they are not, at least, in these very fields."
But that's ENTIRELY natural, thus the tenor of posts here, mostly "so?".
The fact is that what begins as fire-breathing, risk-taking revolutionary, with success and age, ends up being a staunch, reactionary defender of the status quo.
It's the same for Google as it was for the United States and even (in general terms) individuals.
To your point - I think the guy's comments smell more of former-employee car
Re: (Score:2)
No one is saying he's wrong in wanting that.
What people are saying is it's fucking silly of him to whine about it, and that if he doesn't understand why Google is loathe to replace it then perhaps he's not hot shit like he thinks he is.
There's good reason why he can't get what he wants out of Google, and if he doesn't understand those reasons (i.e. tried and tested stability) then perhaps he's just a shit developer seeking attention. In which case, why the fuck should anyone care what he has to say?
Re: (Score:2)
What do you even think that means? Pushing technology. Pfft. Do you mean new technology? I hate to break it to you, but almost all consumer grade technology is old stale bread crumbs. Do you think I mean, is anything that Facebook, Twitter, Groupon, or Amazon doing cutting edge?
The fact of the matter, Google exists to make money for their shareholders and employees. If they decide to train a billion squirrels to sort nuts in such a fashion that when my query is received it provides 10,000 relevant matches i
Re: (Score:2)
Simply put, you did not even get the point.
Re: (Score:2)
You know sometimes they call that insanity [brainyquote.com].
No, "they" don't. Albert Einstein never did. Only AA (and maybe Rita Mae Brown, though I think she used it after AA did) makes that claim, and AA is idiotic and full of shit. (http://www.hulu.com/watch/207926/family-guy-friends-of-peter-g [hulu.com] :) http://rationallyspeaking.blogspot.com/2006/07/alcoholics-anonymous-doesnt-work.html [blogspot.com] ) http://en.wikiquote.org/wiki/Albert_Einstein [wikiquote.org]
Seriously? (Score:5, Insightful)
Is it working, though? (Score:3, Insightful)
Google has had a number of failures, and they do seem to have a hard time pushing out obvious updates and improvements to many of their products. Think about what it says if a company with 26000 employees can't keep a services like Wave going and instead suffers the embarrassment of killing it off three months out of beta.
The reason you still see so much tech coming out of Google is because they have hired a large chunk of the best coders in the world. Google has so many good employees and so few core p
Re: (Score:2, Interesting)
Don't tell him how old NASA's space shuttles are?
Re: (Score:2)
Totally like my VCR, VHS tapes, 20" CRTV from 1996, etc.
Translation (Score:5, Funny)
Re: (Score:1)
Re: (Score:3, Insightful)
Seriously, I read "resigned from Google" and had to take a few minutes to get my mind around that concept. Especially if he was a coder. They treat coders like demi-gods.
Well, maybe that's the problem: He didn't want to be treated as mere demigod. :-)
Re: (Score:2)
Actual Link (Score:3, Informative)
McD's Special Sauce Not So Special (Score:2)
Don't dismiss Dhanji's street cred as a developer (Score:5, Interesting)
His book on Dependency Injection is one of the few recent computer books I had to go through carefully, and with notepad and highlighter in hand. His work on Google Guice is really notable. This ain't just some Microsoft-bound disgruntled guy.
But it's not necessarily surprising. I'm not very familiar with it, but Google's Wave was one of those allegedly killer technologies that just didn't get the corporate support it needed to reach its potential as a disruptive technology. Still, there's a possible tone of sour grapes here. Hard to know.
I'll just say this: I would love to have the privilege to work with someone of his caliber.
Re: (Score:2)
chances are he's just another frustrated genius type, pissed off that everyone else couldn't (or he thinks, wouldn't) see everything his way.
Often such arrogance goes hand in hand with hubris as he comes up with amazing stuff that sounds great, perfect in theory, but fails somewhat in the real world where we all know imperfection is good enough.
Maybe he was kicked out, maybe he left. either way, now's his chance to do all that wondrous stuff he dreams of. Slating Google is just unproductive.
Re: (Score:2)
His book on Dependency Injection is one of the few recent computer books I had to go through carefully, and with notepad and highlighter in hand
Is that a good thing? Personally, If I am going through a book closely with a notepad and highlighter, I am either:
Obsolete is the new stable? (Score:3)
I think many other companies would be happy to have remotely 'ancient, creaking dinosaur' technology. I ponder to think what the authors opinion of infrastructure technology in the rest of the world that would be lucky to be only 15-20 years old.
Citing MessagePack is certainly surprising as that particular technology is significantly worse than Google Protocol Buffers, the website is littered with bad test procedures and many errors. Google's serialization doesn't have the speed of say TIBCO's QForms or the compactness of Reuters RForms but it is pretty clear from their documentation that flexibility and easy management were preferred goals over utmost highest performing technology.
So what? (Score:3)
All my ex'es blast me too. And for good reason. According to them.
The moved on to another loser of their own making. How's that workin for ya, honey? Gotten through his six months worth stupid stories yet? Ask him how he likes your hyena-like laugh. Later, babe. See if your latest will move you into your new trailer. Love your new hair.
ps - Hackers struggle in almost every corporation. Something about breaking stuff and not valuing availability over innovation. So do I want a hacker mentality ruling at my bank? Depends. Keep them away from the transaction system and the website, so I can get in and get my money, ok? The ops guys hose it up enough already.
Re: (Score:2)
Yeah, but I'm sure all your exes are vindictive whores to begin with.
Good Riddance (Score:1)
I worked at a company where the people deploying Hadoop were like him. Most everyone doing actual work wanted to strangle them while yelling "it works fine, stop upgrading every other month and causing hundreds of hours of unnecessary work." Especially since most of the changes made things worse (read: bug testing, who needs bug testing) or added features no one wanted. But they were "cutting edge" so the academic minded fools in charge just kept putting them into production. The features that would have he
Did he just compare MapReduce to JSON? (Score:2)
Because a simple name-value pair object & array data serialization format is wayyyyyy better than a distributed data storage and retrieval system. Right...
Re: (Score:2)
Yeah, that sounded funny to me, too. I'm used to hearing people say "programming HTML", but claiming "algorithms are better than data formats" is just... nonsense all the way down!
The dude needs to grow up (Score:3)
He will see the point in using those "obsolete" (read stable) technologies in 10 years: the goal of business is to make money, not make work by constantly upgrading.
Make Money Fast (Score:2, Insightful)
I think I see what he's getting at. In the past few years, a few people have gotten rich doing something really dumb, but popular and scalable. Angry Birds, Farmville, and Twitter come to mind. (Not Facebook; there's a lot of heavy machinery behind the scenes making that go.) Google hasn't been doing that kind of thing. Some people think that's a problem.
In reality, Google has exactly the opposite problem. They've been frantically introducing cool "products" that don't make money. Meanwhile, quality has
All grown up (Score:3)
What's the problem? (Score:2)
"describes Google's famous back-end infrastructure as a collection of obsolete technologies, designed 10 years ago for building search engines and crawlers."
Is this supposed to be a problem?
What makes Google money? Search and search ads.
Mature technology designed for search engines by a company with a billion in revenue per year from search ads is probably very good for making money from a search engine.
Google's main weakness (Score:2)
Google's moved on from MapReduce... (Score:2)
Hmm. (Score:2)
That never occured before. Employee trys to build sth on an infrastructure not created for it and fails. Instead of positively saying: "ok, thats not the right place to do it", he says "this place is wrong totally".
I mean, i would be a little bit nervous if googles software infrastructure was significantly younger than 10years.
I think for infrastructure its not unusual to use technologies from 20years back, and only introduce new things very slowly.
Yep, sounds about right (Score:2)
You want fun?
Apply for a software engineering or systems engineering job at Google and see if you can get a phoner with a member of the GMail team. You will sustain an injury as you fall of your chair laughing at how the interviewer describes to you how GMail is "designed," as if it really were "designed."
Google is a superb search engine, but that's about it.
Re: (Score:2)
Least you could do is read it on Chrome running on XP like I am.