Linus Chews Up Kernel Maintainer For Introducing Userspace Bug 1051
An anonymous reader points out just how thick a skin it takes to be a kernel developer sometimes, linking to a chain of emails on the Linux Kernel Mailing List in which Linus lets loose on a kernel developer for introducing a change that breaks userspace apps (in this case, PulseAudio). "Shut up, Mauro. And I don't _ever_ want to hear that kind of obvious garbage and idiocy from a kernel maintainer again. Seriously. I'd wait for Rafael's patch to go through you, but I have another error report in my mailbox of all KDE media applications being broken
by v3.8-rc1, and I bet it's the same kernel bug. And you've shown
yourself to not be competent in this issue, so I'll apply it directly
and immediately myself. WE DO NOT BREAK USERSPACE! Seriously. How hard is this rule to understand? We particularly don't
break user space with TOTAL CRAP. I'm angry, because your whole email
was so _horribly_ wrong, and the patch that broke things was so
obviously crap. ... The fact that you then try to make *excuses* for breaking user space,
and blaming some external program that *used* to work, is just
shameful. It's not how we work," writes Linus, and that's just the part we can print. Maybe it's a good thing, but there's certainly no handholding when it comes to changes to the heart of Linux.
Still.... (Score:4, Funny)
...a better love story than Twilight.
Re:Still.... (Score:5, Interesting)
...a better love story than Twilight.
Well, arguably what Linus is doing is a kind of tough love. He did something dumb and he's getting flogged for it. If his ego survives, he'll be a better programmer for it. That said, Linus is too much of a net celebrity to be laying someone out like that. It makes the news, like it did here, and then it makes Linux as an operating system -- and everything that runs on it and depends on it, look bad. It's like if Balmer dragged a programmer up on stage and proceeded to flog the crap out of him for ruining something in the windows kernel -- people would be all over what a horrible platform Windows is to develop for and what a horrible company Microsoft is. And Balmer could have been totally correct in everything he said about said hypothetical programmer.
There's some things you just don't do on the public stage; And an e-mail list is public, yes.
Comment removed (Score:5, Insightful)
Be fair (Score:5, Insightful)
The guy deals with A LOT of bullshit on a daily basis, and this was a BS patch anyway. That would be the end of it if the dude didn't start coming up with excuses. If you know you're wrong, fucking say so and make things right.
Re:Be fair (Score:5, Insightful)
Did you read the thread? I don't think you did. He started off by telling the person reporting the bug that it was a user space app problem (pulse audio).
This is wrong for several reasons.
First and foremost, it is the unholiest of unholy to make a kernel change to an API that results in userland apps breaking.
Second, Linus's policy is that of the above, never is a kernel change allowed to break a userland app.
Third, the way he changed the code was in no way what so ever acceptable. He change the error code for an ioctl to NOENT, i.e. file not found ... ioctl's work on open files, its not possible to have a file not found, you're already past the point where you would get that response as you've already opened the file.
It was a shitty patch AND the patcher blamed someone else AGAINST POLICY for his own stupid mistake. THAT is why Linus tore him a new one.
He didn't get reemed for just fucking up, he got reemed for fucking up in multiple ways and blaming it on someone else, then when that was pointed out, he tried to respond with 'well other people do it' as if that is an acceptable response as well.
Re:Be fair (Score:4, Insightful)
Second, Linus's policy is that of the above, never is a kernel change allowed to break a userland app.
While I don't doubt (reading quickly through the description of what happened) that this change was bad, a policy that "kernel changes can never break a userland app" seems like a bit of an odd statement to make.
In my experience (and I do do development in environments where this comes up frequently), it is not at all unusual for applications to either rely on buggy behavior of another piece of code, or to make unwarranted assumptions about how another piece of code works, that happens to be valid under some circumstances but not others. If an application does not properly handle the documented behavior of an interface, then the application is broken.
In the case of the kernel, it's impossible to validate every single piece of userland software out there will not break due to a change in behavior, and although it can be best to maintain compatibility whenever possible, at some point that can become either impossible or impractical.
Now, if the policy is that you don't break applications that are following the documented kernel interface -- well, then that makes a lot more sense.
Re:Be fair (Score:5, Insightful)
Which is why Linus' policy is that, when the choice is between userland compatibility and better adherence to theoretical documentation of the kernel interface, tie goes to userland compatibility. This is smart - Linux already has enough issues with userland apps breaking backwards compatibility to scratch itches that nobody really has (see GNOME 3, the entire audio stack, etc.); if the kernel is disciplined, at least, it keeps the whole environment from turning into a bug-ridden tar pit.
As for Linus' attitude, well, I have to agree with Linus on this one. Fix the mistake first, either by removing the patch that broke everything or quickly implementing a fix, then ask questions. The first rule of kernel development should always be "Do No Harm". If you're in charge of some part of kernel development and you find yourself breaking that rule, you need to un-break it ASAP, then assess how you found yourself breaking it in the first place. Unfortunately, Mauro wasn't grasping that - he was too busy asking "reasonable questions" to undo the damage that his commit did when his first priority should have been to stabilize the kernel. Don't get me wrong, Mauro's questions are important and they do need to be answered, but only after userland is back to the condition he found it before the commit. And not until then. And *certainly* not with the mainline kernel.
Re:Be fair (Score:5, Interesting)
That seems a bug at pulseaudio.
Which is obviously false and a pitiful attempt to put the blame where it does not belong.
Re:Still.... (Score:4, Insightful)
I think it would be quite appropriate for Microsoft to track down the people responsible for Windows, and force those people to apologize to the rest of us.
Re:Still.... (Score:5, Insightful)
Re:Still.... (Score:4, Funny)
As could you. :)
Re:Still.... (Score:5, Insightful)
I don't give a flying fuck WHO it is, be it Torvalds or Ballmer or Cook or Ellison if someone acts like a total fucking douchebag they should be called out for it and I'm calling him out, Linus Torvalds you sir are a douchebag. you could have solved the whole fucking thing in under 30 seconds by simply explaining why "it must be thus" but instead just acted like a giant flaming prick, it was completely uncalled for, the other guy was acting complete civil and politely, therefor the asshole of the week award should be given to Torvalds for flying so completely off the handle when there was zero point in it.
Actually, the other guy, while being incompetent, was very manipulative, and it worked - with people like you. To be manipulative like that you have to have a lot of social intelligence, and that helps you to navigate situations in which your incompetence surfaces. Some people have little patience with such manipulations and fakery.
Re:Still.... (Score:5, Insightful)
So simply being a nice person is now manipulative because it makes people like you? Do you even realise what you're saying?
There is nothing nice about causing harm for no reason to other people (users in this case) while smiling and talking politely.
If somebody rear-ends my car at a light, they have caused me harm. Do I therefore get out of my car and start screaming at him to "punish" him for his sins? No, I simply call the police to have a report created, and exchange insurance info. It will no doubt cost the guy money, and his insurance company will no doubt give him incentive not to repeat the incident. But, the fact is that it was an accident - that is why we call them accidents.
People mess up - it is a defining characteristic of humanity. Sometimes they even mess up and don't realize that they messed up. If you can't deal with this, then go do the rest of the human race a favor and have yourself committed.
And as far as "for no reason" goes, read the entire thread. There actually was a reason for the patch, and the reason was a good one. The implementation was incorrect, but the intent was not malicious.
Re:Still.... (Score:5, Insightful)
Your analogy is flawed. It was the users that were rear-ended, not Torvalds. Torvalds is the judge throwing the book at the driver who rear-ended you when he keeps trying to say it was all *your* fault.
Re:Still.... (Score:5, Insightful)
Uh, Linus might have good taste in patches, but he has horrible taste in human interaction. Anybody who doesn't think so has equally horrible taste.
The guy who made the commit wasn't a fool - he made a mistake. If he was a fool, then Linus was a fool to grant him commit access.
Re:Still.... (Score:5, Interesting)
This little slice of linux kernel maintainer ethos indicates code quality and its twin sister, system stability, dominate the linux kernel culture as they should. Stability is the prime directive of an operating system. Its functionality must be rock solid. However this objective is enforced, be it code review, peer presure, humiliation, restricting code checkins - it has to happen. Sometimes the uber-egos of certain developers lead them to make brash decisions, and thus have to be dampened, with a club if necessary, for the good of the tribe. In the exotic world of clashing software Prima Donnas, a suit of armor is a necessity. The victim will survive.
Re:Still.... (Score:5, Insightful)
Or it looks like someone had a temper tantrum because he's a celebrity and doesn't have to be decent to other people.
And the thing is, that happens. A lot. Even at microsoft. "That's the stupidest fucking thing I've ever heard!" - Bill Gates. And that was in Paul Allens book, and apparently an oft used phrase.
Linux lives and breathes on its contributors, if you make too many public scenes you can end up with a lot of important people and companies saying it's not worth it to contribute to, or it's not worth it to contribute to under Torvalds, that would be a very ugly mess for the entire industry. Worse still is if this sort of behaviour sets the tone for everyone else trying to be the next Linus, and they try to copy his to behavior (and that definitely happened at microsoft) and you end up with an organization full of asshole managers who no one wants to work for. At least microsoft can hold out the carrot of a fat paycheck to put up with it, Linux relies on people volunteering to help.
Granted, it's not really a shock that Torvalds likes to... speak his mind. That's part of his thing. Still, it poses a lot of questions about the sort of person who's going to be around to succeed him if he gets hit by a bus so to speak. Certainly Tim Cook and Steve Ballmer have not been able to find the right magic replacing their more famous bosses, and one would hope Linux manages better than Apple and MS have, admittedly, those jobs are completely different.
Re:Still.... (Score:5, Insightful)
Or it looks like someone had a temper tantrum because he's a celebrity and doesn't have to be decent to other people.
He cares more about the product being right than other peoples' opinion of him.
And the thing is, that happens. A lot. Even at microsoft. "That's the stupidest fucking thing I've ever heard!" - Bill Gates. And that was in Paul Allens book, and apparently an oft used phrase.
Sometimes the truth hurts.
Linux lives and breathes on its contributors, if you make too many public scenes you can end up with a lot of important people and companies saying it's not worth it to contribute to, or it's not worth it to contribute to under Torvalds, that would be a very ugly mess for the entire industry.
Nope. Linus has been Linus his whole life. This "story" isn't anything new. Oddly enough, Linux keeps marching onward, with plenty of contributors who are both volunteers and paid by various corporations to contribute work.
Worse still is if this sort of behaviour sets the tone for everyone else trying to be the next Linus, and they try to copy his to behavior (and that definitely happened at microsoft) and you end up with an organization full of asshole managers who no one wants to work for.
Again, the truth hurts. Projects like these don't tolerate stupidity well, especially given the fact that it's a community effort and there has to be a lot of lack of testing and communication on the part of the responsible party when something breaks like this. The price to be paid is twofold: (1) very public expressions of the harm caused by such issues, and (2) potential relief from duties related to the issue at hand, whether temporary or permanent in nature.
Granted, it's not really a shock that Torvalds likes to... speak his mind. That's part of his thing. Still, it poses a lot of questions about the sort of person who's going to be around to succeed him if he gets hit by a bus so to speak. Certainly Tim Cook and Steve Ballmer have not been able to find the right magic replacing their more famous bosses, and one would hope Linux manages better than Apple and MS have, admittedly, those jobs are completely different.
Step right up if you can do it better or know others who can. It takes strong leadership founded on the idea that others follow you primarily for you technical ability, and secondarily for your "soft skills" to manage a technical project of this scale. Sometimes people have problems replacing their famous bosses because while the old boss may have been widely regarded as an asshole, he was an effective asshole, and numbers (whether measured in uptime, lack of showstopping functional bugs, level of attention to security issues, or profit per quarter) don't lie.
All things considered, I saw far worse than this in my prior military service (Navy submarine force, ET-COM) in cases where somebody did something stupid that resulted in a systems failure, and oddly enough some aspects of this whole deal remind me a bit of organizational units in the services that pride themselves on putting job performance and technical ability first. Those who get butthurt about being dressed down are free to find something else to do with their time.
Re:Still.... (Score:5, Interesting)
Again, the truth hurts. Projects like these don't tolerate stupidity well, especially given the fact that it's a community effort and there has to be a lot of lack of testing and communication on the part of the responsible party when something breaks like this. The price to be paid is twofold: (1) very public expressions of the harm caused by such issues, and (2) potential relief from duties related to the issue at hand, whether temporary or permanent in nature.
I don't believe you. Someone added a sleep to the USB-HID driver in 2.6.10. The problem is they were holding a spin lock at the time. I submitted a patch for it to every single kernel release from there until 2.6.18 when a Linux kernel developer submitted the same patch and it was accepted. It was a very simple fix. If they don't tolerate stupidity, why did they ignore my one line patch and leave a huge bug in the damn keyboard/mouse (and other devices of course) for so long? I have had zero interest in contributing to these large projects ever since, as they seem to be more about ego and status than moving the project forward.
Re:Still.... (Score:5, Insightful)
"many eyes" making the code work seem to be obsolete when you have a club that you have to be a member of before they take notice of you. Its always been this way in so many fields of human endeavour - if you're expecting the community around Linux to be different and based on technical excellence only then you have a lot to learn about people.
The trick is to acknowledge that you're not "one of them" and approach the task of submitting your patch like this - submit the patch as normal, and then contact Linus or someone with an email humbly telling them that you've noticed a little bug that slipped through and that you've submitted a patch that you think will fix the problem - once you have their attention enough to look at the fix, they'll be ready to accept it, as if it was something they'd noticed themselves. I guess that once you've done this a couple of times, they'd accept your future patches without review.
I know its not supposed to work that way, but it does... I'd be interested to know how you went about the submission process. It could be that they were just too busy to review all the patches that were submitted, so yours languished, un-noticed for so long.
Re:Still.... (Score:5, Informative)
This sounds so gimp like. Like a teen at school trying to get in with the popular set. Or Dobby from Harry Potter.
It's just pragmatism. For every programmer competent to work on the kernel there are a thousand wannabes, so some vetting process is required. In companies, this is done via resumes and interviews -- and then generally by giving new software engineers projects of low importance so you can vet their work before trusting them with stuff that could break the business. Same thing, just a different context.
Re:Still.... (Score:5, Insightful)
Nope. Linus has been Linus his whole life. This "story" isn't anything new. Oddly enough, Linux keeps marching onward, with plenty of contributors who are both volunteers and paid by various corporations to contribute work.
It's more than that.
Linux powers tens if not hundreds of billions of dollars worth of systems. It's unbelievably huge, and it's almost inconceivable that all of the megacorporations whose business depends so deeply on it didn't snatch control of the whole thing away from the lone kid who started it as a hobby project long ago. I mean, who would have predicted that even after it had become so important, Linux would still be maintained by the one guy who started it, rather than some joint committee of top-tier OS engineers -- probably dominated by IBM?
Not only is that not what's happened, but as far as I can see none of those megacorps even try to tell Linus what to do. They submit patches, humbly formatted and refactored into the form Linus wants, and they butter him up with conference tickets, free hardware and the like, and they even compete for the right to pay him a salary.
Why is that? Because whatever anyone says about him, his style, his attitude, his people skills or even his code... the results are fantastic. Linux not only works very well, it does so across an amazing variety of hardware platforms, and the design -- and Linus' good taste and fanatical attention to detail -- have proven to be able to support virtually any new idea that's worth implementing.
Love Linus or hate him... don't mess it up. Luckily, Linus is an egotistical bastard who doesn't care what anyone thinks anyway, so mere words aren't likely to change anything.
Re:Still.... (Score:4, Insightful)
Seriously, if as an enthusiast I submitted a patch that broke user space and got a beat down from Linus I'd be a bit sad. If, as a PAID developer, I submitted a patch that broke user space AND THEN blamed the user space applications and got a beat down I'd wonder wtf am I doing writing software.
You never, ever, blame the end user for your fuck ups. Even more so when you changed the API/ABI and fucked it up yourself.
Linus is an ass, but when dealing with someone who goes full asshole on the user space devs over his fuck up, he was a justifiable ass.
Re:Still.... (Score:5, Insightful)
It'll make Linux better (Score:5, Insightful)
...and then it makes Linux as an operating system -- and everything that runs on it and depends on it, look bad...
While I agree that this incident does put Linux on the front pages in part of the Net, I seriously doubt the "look bad" part.
In fact, imho, this incident will generate more positive than negative for Linux.
Admins and users that are still sitting on the fence would take note, that Linus just don't take fuck as an excuse - and if anyone dares to fuck up like that fella he will get a public tongue lashing - and the top priority of Linux is to *NOT INTRODUCE ANY USERSPACE BUG*.
Re:It'll make Linux better (Score:5, Interesting)
Admins and users that are still sitting on the fence would take note, that Linus just don't take fuck as an excuse
No, instead he apparently creates a contributor-hostile culture based on blame assignment and supported by a hero model. And that is far, far more dangerous, because when your hero isn't there (and he can't be everywhere, and he won't be around forever) you have pushed away good people who could have stepped up to take over the job. Internet-famous celebrities like Torvalds are toxic to a constructive development culture that consistently makes good products over the long term.
Re:Still.... (Score:5, Insightful)
Doing something stupid is not a free ticket for people to be rude.
Doing something stupid doesn't. Doing something stupid, not apologizong for it, failing to fix it in a timely manner and then blaming the stupidity on other people's code - when kernel policy clearly state it's YOUR responsibility - is much closer to the free ticket, wouldn't you say?
Re:Still.... (Score:5, Interesting)
He did apologize for it, quite quickly, and agreed that reverting the change was appropriate. He also asked a good question: why can't pulseaudio deal with other error codes, error codes that might be technically correct, though not in current use?
The developer wasn't being nasty, underhanded, manipulative, etc. It's possible to break user land by fixing broken behavior that userland depends on, and it's been an ongoing issue for all kernels. (Take a look at the history of the egcs variant compiler and Linux kernel compilation.)
Re:Still.... (Score:5, Funny)
Imagine what it's like to be Linus' wife!
"SHUT UP! You burnt the bagels! Take responsibility! You know our policy, no wasted food! How dare you try to blame the toaster oven for being inconsistent? We never blame inanimate appliances Just shut up! You should have been watching the toaster!"
Re:Still.... (Score:5, Funny)
Arsehole (Score:5, Insightful)
Seriously, who'd put up with this crap? There are nicer ways to make ones way through life.
Re:Arsehole (Score:5, Insightful)
I wish my boss would show the same concern for the quality of our software, and scream at the programmers that break it every day. Unfortunately it doesn't happen, and other programmers end up fixing the broken code (and quietly grumbling) because otherwise they would not be able to do their jobs.
Sometimes harsh words can send a message better than a general "please check your code before you commit".
Re:Arsehole (Score:5, Insightful)
Of course, neither harsh words nor general "please check" encomiums are as good as calmly, straightforwardly stating "we do it this specific way, you did it that way instead, as a result we have this undesirable result. Do it again, correctly. In the future, check X, Y and Z to ensure that you don't repeat this mistake."
Only in the socially retarded world of /. is this sort of behaviour lauded because the best behaviour that I described above seems too goddamned difficult to manage.
Re:Arsehole (Score:5, Insightful)
While I agree somewhat with what you say I have to say that sometimes it is necessary to chew people out. I don't know what the situation is or the personality of the maintainer but people are different. What works with some doesn't with others. I have supervised people in a maintenance environment and while some just needed to be told what to do and let them run with it there were others that had to be kickstarted. Some people do actually benefit from a good ass chewing.
Re:Arsehole (Score:5, Interesting)
Re:Arsehole (Score:5, Funny)
come on it's pulse audio related, of course people get nervous, angry and easily irritated ...
Re:Arsehole (Score:5, Interesting)
The rant wasn't about bad code, but a bad mindset. It was an apt response to Chehab's disregard for the integrity of existing Linux applications.
It also wasn't really aimed at Mauro Chehab. Even the worst manager knows better than to discipline an individual employee in front of his/her peers, right? No, that flame was aimed at the entire mailing list. Linus obviously felt it necessary to drive the point home in a conspicuous way that wouldn't be forgotten or blown off.
If there were still people at Microsoft with the technical and moral authority to send emails like that, the company's flatlined stock price graph would look very different.
Re:Arsehole (Score:5, Interesting)
Pics or it didn't happen. Apple had Asshole-in-Chief Stephen P. Jobs with the technical and moral authority to do exactly that. And guess what? OS X took almost a decade before it was anything other than crap.
That's an interesting way of phrasing your point, because of the parallels that can be drawn with Linux itself. Linux has been "crap" for well over a decade, in the sense that it has never made much headway against competing OSes in some key markets. In the past, Linux's competitors -- from Apple, Microsoft, and Sun to vendors of RTOSes that everybody uses but few people have heard of -- were all just too effective at meeting the needs of their customers.
Right now things are finally looking up for Linux. Android is proving to be a key player, if not the key player, in the most important and most promising sector in mass-market computing. Meanwhile, Apple is hell-bent on repeating every mistake from its proprietary past, making its usual heroic effort to snatch defeat from the jaws of victory. At Microsoft, there's a certified Sales Guy(tm) in charge, beneficiary of the luckiest dorm-room assignment in history. Right now Ballmer's monkey-see-monkey-do act is stampeding the company's herd of very valuable sheep, from Valve to Dell. Seismic changes are happening underfoot, some visible to the casual observer and others not.
In short, Linux's enemies are making mistakes at an utterly unprecedented rate. Leadership and vision are important, but given Apple's and Microsoft's behavior of late, it will be enough for Linus to keep his eye on the ball and his mind on the needs of the end user. The future of personal and mobile computing is up for grabs to an extent that hasn't been true since the early 1980s. All he has to do is not fuck up.
This email thread was about just that: the need to not fuck up.
Re:Arsehole (Score:5, Funny)
"Only in the socially retarded world of /. is this sort of behaviour lauded because the best behaviour that I described above seems too goddamned difficult to manage."
Tut, tut, good sir. Next time please do try to deliver your observations in more civilized and diplomatic language.
Re:Arsehole (Score:4, Insightful)
I self censor my emails all the time. You wouldn't believe how many long diatribes I've thrown away. How many heart felt impassioned speeches I've given up on as offensive or too harsh. I guess in some ways it makes the harsh emails I do send that much more potent. The point I'm trying to make is that it must take a special kind of person to stand up in front of more than 100,000 people and say "you're a moron, shut up!"
He either realizes that this would cause a big fallout where his statement would be examined in detail by everyone on the mailing list + everyone on slashdot and a few other communities and just doesn't care; or he somehow hits the send button without ever realizing the consequences.
I think it's the former, and despite his outburst he's still managed to steer the Linux community through 20 years of massive evolutionary changes in computing. It seems that even with armchair analysts saying he could get the job done without yelling, maybe should attempt the same thing, building a peaceful community of kernel hackers with no tyrant at the top and figure out a way to measure productivity, code quality and other metrics.
Re:Arsehole (Score:4, Insightful)
I wish my boss would show the same concern for the quality of our software, and scream at the programmers that break it every day. Unfortunately it doesn't happen...
Sure, it does happen. I have fired programmers who could not do their job. The firing was done in my office with the door closed. I did not post news of the firing on the internal company forums. I did not drag the body through the streets as an example.
.
Why do you rationalize poor management techniques as an excuse for poor management?
Re:Arsehole (Score:5, Insightful)
Bullshit. If someone is fucking up (especially repeatedly), the boss should talk to him. Not scream at him- talk in a professional manner. Or let him go, if the mistakes are bad enough. You do not disrespect your employees by screaming them- it solves nothing, causes workplace friction, lowers morale of both the one being screamed at, the one screaming, and their coworkers. Its counterproductive.
If my boss came to my office and calmly told me I fucked up, I'd immediately jump on the problem and fix it. If my boss came in screaming at me, I'd tell him to get the fuck out, send an email to HR detailing his unacceptable behavior causing a hostile work environment and go home- telling him I'd return upon getting a sincere apology, and that I would expect full pay for every day in the interim. Oh, and my resumes would be on the net that evening. If he really pissed me off I'd go to his boss and tell him its me or him and I want a decision by the end of the day. There is no excuse for not behaving like an adult in the workplace.
Re:Arsehole (Score:5, Funny)
Ahh, you think linux developers aren't paid, that's precious.
Re:Arsehole (Score:5, Insightful)
And that's understandable. Bad programmers will cost you more time than they save.
Re:Arsehole (Score:5, Insightful)
I think the point is, Linus would rather the guy not do any work than to continue to produce such garbage code. He would rather make every change himself, than to rely on fellow-programmers who will only make things worse.
There's a term for people with attitudes like this; who express them by public tantrums: A Prima Donna.
Re:Arsehole (Score:5, Insightful)
His attitude is bullshit, his intent is not.
That said, it's really not clear why this was on a kernel dev mailing list and not in 1:1 emails. One would hope this sort of public thrashing is the last result of frustrated people.
Re:Arsehole (Score:5, Insightful)
That said, it's really not clear why this was on a kernel dev mailing list and not in 1:1 emails.
So that anyone else who thinks of making a similar mistake will think twice.
Re:Arsehole (Score:4, Insightful)
No, don't confuse narcissism for good management. Not the same.
Re:Arsehole (Score:5, Insightful)
First off this is Linus's project, his name on this project, and a bug that would not look well for his project.
Secondly the programmer in question screwed up, and instead of admitting it and asking for help, he blames others. (read, programmer has a big ego)
Linus called him out on it, and the programmer should take it, because the programmer caused the issues.
In short, this whole thing happened because of the programmer, not Linus.
Re:Arsehole (Score:5, Insightful)
There's a term for people with attitudes like this; who express them by public tantrums: A Prima Donna.
Linus isn't a Prima Donna. A Prima Donna is someone who wants to draw attention to herself. Linus is not like that, he's perfectly willing to stay in the background and write code.
However, he doesn't tolerate incompetence or idiots; his flaw is not knowing how to communicate that in a polite way, and being a bit hasty to judge. Which is different than a prima donna.
Re:Arsehole (Score:5, Insightful)
And his attitude isn't that unusual for men in charge. Gates has done it as has Jobs and Ballmer.
Re:Arsehole (Score:5, Insightful)
And his attitude isn't that unusual for men in charge. Gates has done it as has Jobs and Ballmer.
Yeah, actually.
And getting it right is surprisingly difficult. His primary goal is to communicate, "you did something incredibly stupid, and are in danger of getting kicked out of the team."
How do you communicate something like that without sounding rude? It is possible, and if you can do it then you might have a future as a diplomat, but not many people can do it. Developing those skills while simultaneously developing the skills to be a kernel programming lead is not very common.
Re:Arsehole (Score:5, Insightful)
From the summary it appears to be at the end of a string of failed communication, in which case being blunt can be very appropriate.
Re:Arsehole (Score:5, Insightful)
A hockey analogy would be if your teammate took a shot at his own goal, then tried to blame someone else, are you going to talk to him nicely or are you going to punch him in the face and tell him to cut it out?
Re:Arsehole (Score:5, Insightful)
You don't tell a welder that their weld is just not quite right and to please do it again if it's not too much trouble. You tell them it's fucked and to grind it out and do it again before everything falls down and kills someone.
I'm not sure how you manage your welders, but I would go about like this:
"Bill, this bead is too shallow. It won't bear the load. The bottom of the weld is cold, and there is no metal there. You will have to grind it out and redo. This weld will be inspected with X-ray to make sure you do it right this time. We will have to pay you overtime, by law, for this. But if you do it again there will be no overtime - you will get your final check instead. The reason is that if these welds fail someone will be killed. It is that important. Please do your best."
This explains the problem, explains what to do to correct, and also spells out the consequences for continued bad work. It ends with an encouragement; you never want to end the conversation with a threat. But the overall content of the message is exactly the same as yours, and the welder will not feel offended. He will feel ashamed of his poor work. It won't make him happy, but by the end of the day he will know that I was right - actually, as soon as he grinds his superficial bead away and exposes the unwelded groove.
And if in my opinion the guy is hopeless I will simply summon HR to have him dismissed right away. There is no need to offend him by telling him that he is an idiot. He may well be, but it's not in my interest to infuriate people. I simply don't have a use of him anymore. Nothing personal.
In that aspect, try to not enrage your welders when you chew them up at a height of 100 storeys. It's a long fall to the ground, and people do see red when they are angry.
Re:Arsehole (Score:5, Insightful)
"It is not my fault boss, the beads are totally fine, the fault is that they shouldn't be putting the load where they always did, and where it worked before, they should put the load now in this nice little place I arbitrarily chose. Anywhere else the thing will fall apart, but it is their fault."
Re:Arsehole (Score:5, Informative)
That's one hell of a good excuse for some of the brain-damages of minix. I can only hope (and assume) that Amoeba doesn't suck like minix does.
Nope, that was when he was still in Finland. Not a cultural thing, apparently.
Re:Arsehole (Score:5, Insightful)
I think the point is, Linus would rather the guy not do any work than to continue to produce such garbage code. He would rather make every change himself, than to rely on fellow-programmers who will only make things worse.
There's a term for people with attitudes like this; who express them by public tantrums: A Prima Donna.
Ok, I'll bite: How is an email in a private mailing list a "public tantrum"?
LKML is hardly a "private mailing list", given that anyone can read it, and anyone can subscribe to it. There's even a link to a public archive in TFS.
Re:Arsehole (Score:5, Insightful)
I would happily put up with that and more just for the credit of being one of the select few who get to work on the kernel. To have the ear of the man(or team) that gets to decide the direction it goes for the future.
I guarantee you I'd never make that same mistake again. I'm sure if he hadn't tried to blame the user Linus would have been a little nicer about it.
Re:Arsehole (Score:4, Informative)
I would happily put up with that and more just for the credit of being one of the select few who get to work on the kernel.
It's not that hard, go find some hardware that isn't supported yet (or even that's not supported very well, like a lot of sound cards), and start writing drivers. If your code is decent, it will be accepted.
Re:Arsehole (Score:5, Informative)
And getting code accepted != kernel maintainer.
This guy (Mauro) is trusted to make commits to the kernel without direct hard core review by Linus. If I make a code change to V4L and it gets accepted to the kernel, it goes through Mauro, not Linus.
Theres a big difference between me committing code and me being a reviewer who is trusted to know what I'm doing rather than just some code monkey that someone else checks out first.
Re:Arsehole (Score:5, Informative)
Linus hates ethnic minorities. He sees daily what they are doing to his beloved homeland and regrets that he has to work with them.
Don't feed the trolls, they say. But in this case, there may be a few readers here that don't know that Linus Torvalds belongs to an ethnic minority of Finland - the Swedish speaking minority [wikipedia.org] who are not always well accepted in either Finland or Sweden.
And, of course, that after moving to the US, he's still in an ethnic minority. Perhaps even more so.
Re:Arsehole (Score:4, Interesting)
Hmm, the mental demands and sleep deprivation are right on target. Don't know about people trying to blow you up or kill you, but then, I haven't had a chance to work on jet control software or the like (presumably when the military's latest and greatest aircraft falls out of the sky because of a software bug, there are probably quite a few people considering killing you, especially if those onboard are harmed / killed).
Re:Arsehole (Score:5, Informative)
true, an application in theory COULD be prepared to respond to all potential errors, as long as you don't mind a 200 megabyte binary for hello world
not good management technique (Score:4, Insightful)
It's just a kernel patch, nothing to be actually "angry" about. Why so emotional? And especially when it's somebody who is not an employee.
Just say, "It is our firm policy that we do not break XXX, even when any given developer wishes they could. If you wish to continue contributing, you must follow the policy. Please apply the fix ASAP and acknowledge your understanding of these conditions."
Re:not good management technique (Score:5, Insightful)
Re:not good management technique (Score:5, Insightful)
"At his position Linus does not have to put with childish behavior from his staff"
a) it wasn't his staff, it was somebody from Redhat. And even if it were....
b) Linus was the only childish one getting all pantytied and emotional.
Did you read the rest of the thread? And how the other people responded?
It seems to be a complicated issue. The other people acknowledged the bug but showed it in context of some rather difficult hardware & driver issues across multiple areas.
Re:not good management technique (Score:5, Interesting)
No, if you bother to read the rest of the thread, the guy had a good and valid reason that makes sense
No he didn't. Sound worked, and after his patch, sound didn't work. Why would you ever commit a patch that breaks sound?
In his case, he did it because he is trying to add drivers for new hardware that behaves differently than a lot of other hardware. That's not a good and valid reason to break stuff that already works.
Re:not good management technique (Score:5, Informative)
It's not like he wrote a patch with a changelog of "Break sound. Users have had functional audio capability for far too long."
It was a bug in the patch which broke the kernel interface (by returning different error codes), which was not the intended effect of the patch. Laurent Pinchart, the author of the patch, explained in the thread:
The reason Mauro Chehab got chewed out was because he refused to acknowledge that it was a kernel bug, didn't see anything wrong with a sudden change in the kernel return value, and then proceeded to blame pulseaudio for not handling it. That's why, other than his patch being called "total and utter CRAP" and "incredibly broken shit", Laurent was not mentioned in the email, whereas Mauro was told to shut up and called out as flat out incompetent.
Re:Arsehole (Score:5, Insightful)
Everybody makes mistakes, but the right posture when it was obviously your fault is to take the blame and learn from them, not to try and make excuses.
Re:Arsehole (Score:5, Insightful)
How technical do I need to be before I can get away with telling people to "SHUT THE FUCK UP!". Script monkey? OS Kernel developer?
Actually it has nothing to do with technical ability. You can tell people to shut the fuck up if you're not in a position to get fired when your boss finds out. I.e. you own the company or, like Linus, you are Benevolent Dictator For Life on some open source project.
Of course - as Ulrich Drepper found out - even Benevolent Dictators For Life can look out of the window and see that the peasants are tearing down their statues, Ceausescu style. And most people reading this would get shitcanned from their sysadmin job a lot quicker than that for not jumping when their boss tells them.
Really the tech world would be a lot more pleasant if people didn't conflate rudeness and technical competence. Most people seem to master the rudeness, neglect the technical competence and then wonder why they are undervalued.
Re:Arsehole (Score:5, Interesting)
>Really the tech world would be a lot more pleasant if people didn't conflate rudeness and technical competence.
The thing is that Linus is technically competent. He's in the 1% of technically competent people, so his "benevolent dictator for life" status is not changing any time soon. His ability to herd cats is manifested in his managing the Kernel for as long as he has. I'm sure if you approached Linus when he was 19 and told him that he was going to be managing one of the most important software projects on the planet for more than 20 years, he'd tell you that you were nuts.
But here we are, and Linus is 43, and he's been doing this for quite a long time and Linus *doesn't* have a reputation for being a firebrand. And honestly, to even call Linus rude ignores the fact that when he's rude, it's actually news. Unlike Theo, who is just rude all the time, so it's never news.
--
BMO
Re:Arsehole (Score:5, Insightful)
He isn't trying to make friends. He is trying to manage the efforts to implement a particularly complex piece of software, whilst hearing bullshit from one of his developers who made a very big mistake. Everybody makes mistakes, but the right posture when it was obviously your fault is to take the blame and learn from them, not to try and make excuses.
You can't fix unprofessionalism by being unprofessional. Linus's response is uncalled for, and if anyone I work for ever tells me to shut the fuck up, he can start looking for my replacement, and won't ever need to hear me speak up again.
I agree with Linus's reasoning for why it's really a kernel bug, not a pulseaudio bug. The correct way of handling this would be to simply say, "you need to own up to this thing, because it's our policy that any kernel changes that breaks userspace applications is our problem. Fix this now." If for some reason this is a constant issue with the guy, than you remove him from that role as maintainer, but based on the lkml thread and the responses from Mauro, it seems Mauro had good reason to believe some things would remain broken after reverting the changes to the kernel, so he was just trying to understand the problem better. He behaved professionally, calmly explaining his position and agreeing with Linus about his mistake, on the face of having an adult throwing a tamper tantrum at him. Kudos to his patience, I don't share it. You simply could not pay me enough to work with someone who very publicly disrespected me in the way Linus did with him.
Re:Arsehole (Score:5, Interesting)
It's more insidious than that.
If Linus coddles people, they get a false sense of approval. His brash, direct, and overtly emotional tone is intentional. It's to let the other party know exactly how he feels without accidentally leading them into a false sense of "oh, keep working on it and I'll approve it."
Watch that video where he flips off Nvidia.
He describes how being nice to people is the wrong approach, and makes things worse when the other side gets the wrong view, is rejected after months of effort in the wrong direction, and then Linus gets the blame for their suicide.
If he was paying them, he could fire them. He doesn't have that luxury, so he has to be clear and direct.
Don't like it? too bad. That's Linus, and he makes no excuses for how he behaves.
Re:Arsehole (Score:5, Insightful)
No level of "professionalism" will make most people understand how badly they screwed up when they screwed up badly. I have personally tried the professional route numerous times, only for the message to not get across. At that point your subordinates think you're a push over and that the problem wasn't that serious. Do what Linus did here when it's important and only when its important and you will accomplish two things. The first of course is getting your point across clearly. The second is that you don't tolerate any BS when someone has clearly screwed up.
This is somewhat related... (Score:5, Funny)
This is somewhat related with this Linus' post [google.com] on G+?
Don't piss off the boss (Score:4, Interesting)
Not the bug... (Score:5, Informative)
I'm not a big fan of this style of management, but I can't fault any of the content of Linus' rant.
Re:Not the bug... (Score:5, Insightful)
That was my reading exactly. As a programmer, I've had to condition my responses to bug reports to move from, "What?! It's totally your fault!" to, "Let me investigate this on my end." It became fairly obvious from the get go that the patch made some changes that were overlooked in the patch. The patch commit message failed to give any reason or justification for the change (which was a 1 line change, possibly in a sea of other changes). Furthermore, the automated testing application that the patch submitter was using had a bug in it.
It's likely the testing apparatus said, "OMG! THIS ERROR VALUE IS WRONG!" Then, the submitter corrected it to fit the bad test. Honestly, while Rafael signed off on it, it would appear the submitter set him up the bomb, so to speak. Who would make a change like that without further investigation and then submit it is a serious question. I would hope the patch submitter is placed under some additional scrutiny.
All in all, Rafael took it in stride in his response. He also explained further, but appears to have capitulated that his initial reaction was wrong and that he should have fixed first, asked "Why are you using that, anyway?" later. Also, he should have never stated that, "So, on a first glance, this doesn't sound like a regression, but, instead, it looks that pulseaudio/tumbleweed has some serious bugs and/or regressions." It's quite obvious that changing not only what error code is returned but returning one that was literally impossible without telling anyone is a Bad Thing(TM).
As for Linus's style of management, I don't know. I mean, Rafael was way, way, way out of line with that statement I quoted. He really failed to grasp the situation, even after it was clearly spelled out for him.
All in all, I think he's learned his lesson and will be better for it. He seems to have taken it well enough, at least publicly. :-)
Re:Not the bug... (Score:4, Interesting)
I don't think you quite got it, as I understand it they have more error states internally than they do in the kernel API. The patch was designed to pass an internal error state that should have been replaced by the one in the kernel API before being returned but was accidentally pushed to userspace as well and the bug in the compliance tool meant they didn't notice. So far nothing majorly wrong, a poor patch and a testing tool with less than 100% coverage which caused a nasty bug to reach the first RC, a little sloppy but nothing that obviously should have been caught. The reason Linus gets so fired up is because it just doesn't matter what pulseaudio/tumbleweed is doing, there's zillions of applications out there who may be doing something crazy. If you broke one application you probably broke many more and it just doesn't matter, it's the kernel's fault.
It's been a cornerstone of Linux for as long as I can remember, you can always upgrade your kernel and all your applications will continue to work, period. New APIs are of course added but old userspace APIs are forever. It just doesn't matter how crazy the userspace code is, you don't break it anyway and even if they fixed their crazy code it doesn't solve the problem which is that there'd be two kernels with different behavior So even if he was right, he's still wrong.
Re:Not the bug... (Score:5, Informative)
No. He clearly stated that he saw this, "at first glance," as something that Pulseaudio had been doing all along. The patch made a function not only return a different error code, but an error code that was never possible before the patch. The change was not documented, at all.
The submitter really set him up, though. If you look further up in the thread, the "rollback" was a simple one line change from ret = -ENOENT; to ret = -ENVAL;. I'm fairly certain that it was just overlooked in the code review for the entire patch.
What got him was that instead of going back and saying, "Huh. Why did we change this error code? Oh my, ENOENT was never even possible before this. This'll break all sorts of crap!" he blamed Pulseaudio. That's a serious no-no that's been covered several times in the history of the kernel.
This is not news. (Score:4, Insightful)
I could kiss the guy... (Score:5, Insightful)
I am so tempted to kiss the guy! Linux can't have this type of _OBVIOUS_ audio sabotage at the kernel level if we expect game developers to ever make their products available on Linux. Linus NEEDS to reign this in, we just can't afford it. He is 110% right.
Keep the helm firm, man! That's exactly what we need!
Qybix
Re:I could kiss the guy... (Score:4, Insightful)
As an application developer (and game developer by night), I concur. Test the patches before submitting them upstream. If you don't understand what the patches do, you don't approve them. Hell, that's the same requirement I have for all code in my projects. Even if it's a comment or whitespace change, you compile and (unit) test the module before committing it. It's a pretty brain dead rule of thumb: If you don't know what the fuck you're doing, don't do it! It's when you pretend to know more than you do that you get in trouble. The stupid thing is that there's no excuse for not knowing -- If all else fails, you ask the submitter who sent you the patch.
Also, take Linus' wrath with a grain of salt. He enjoys being blunt and offending those that get offended. His opinion is that you can't actually offend someone, they have to take offense themselves. He get's most pissed when he knows he's right. He accepts people calling him stupid when they're right and he's wrong, like in this talk where he gives the finger to Nvida devs... WRT an early version of the Linux kernel: "We didn't have init. We don't need init, just a boot image and root console, that's how real men do it... Then someone came along and said, 'This is stupid, you need init', and they added init" -- I'm paraphrasing, that's pretty much how I remember it from the video. [youtube.com] Sorry, too lazy to re-watch and find the exact quote. Also contained therein is his position on being harsh...
Also, Bill Gates, Steve Jobs, Steve Ballmer -- Come on, they're all notorious as being harsh At least Linus isn't parking in handy cap spaces, or throwing chairs like a real asshole.
Dude needs to read Dale Carnegie... (Score:5, Insightful)
I wouldn't speak to anyone that way unless I never wanted to see them again. Shaming someone in public like this is a way to not only get them to quit, but to make a permanent enemy out of them.
What this tells me is that Linus is better suited to be a cult leader than a manager of any kind.
Finally! (Score:5, Interesting)
News for Nerds, Stuff that Matters! Thanks Slashdot! Its been a while...
Shit Happens (Score:5, Insightful)
Sometimes shit happens. Sorry for that.
Cheers,
Mauro
Wrong. Answer.
Re:Shit Happens (Score:5, Insightful)
IMHO, this is the exact attitude that got Linus pissed off.
This is an inkblot (Score:4, Interesting)
whiny mauro (Score:4, Informative)
An anonymous reader points out just how thick a skin it takes to be a kernel developer sometimes...
Mauro points out just how thick a skin it takes to be a kernel developer sometimes...there FTFY
It needs to be said. (Score:4, Insightful)
Sometimes stupid people need to be told they are stupid, otherwise they wont stop being stupid.
The real problem is in the API spec (Score:5, Interesting)
Take a look at the call for which ENOENT was returned. [linuxtv.org] The API spec says "EINVAL: The struct v4l2_queryctrl id is invalid. The struct v4l2_querymenu id is invalid or index is out of range (less than minimum or greater than maximum) or this particular menu item is not supported by the driver."
This is from a generic driver mechanism for USB camera-like devices. Because such devices aren't as standard as they should be, there's an insane number of options and possible errors. The spec says to return EINVAL for both incorrect calls and calls for which the device does not support the requested item.
The problem here is that the EINVAL error status doesn't distinguish between "program made bad call and is broken" and "we're iterating through the device functions to discover which ones are available, and this ID isn't meaningful on this device so skip it" EINVAL is supposed to mean "One or more of the ioctl parameters are invalid or out of the allowed range." A correctly made call should not return EINVAL.
The alternatives are limited, though. This is related to a historical Linux design problem, which comes from a historical UNIX problem - system call errors are reported using one error code, chosen from a short list written in the 1970s. "ENOENT" isn't really appropriate. "ENOTTY" ("The ioctl is not supported by the driver"), might be appropriate, except that the usual message for that is "Not a typewriter".
The API is a rather lame and excessively complex way to return what is merely a variable-length list of fixed-format structures. One would think that Linux would by now have a generic way to do that, since it comes up in other contexts.
Here's an example of an application which seems to be crashing because the programmer did not understand the nonstandard semantics of EINVAL in this API. [stackoverflow.com]
Re:if only more technical leads had this mindset (Score:5, Interesting)
I have a "chain of clients" theory of system design.
The kernel developers protect the library developers from having to deal with the hardware.
The library developers protect the applications developers from having to deal with the kernel.
The applications developers protect the users from having to deal with all of the above.
Everyone has shit to deal with, but if you break the chain and let some of your shit spill into another layer, then someone else has to deal with way more shit than they can reasonably be expected to handle.
Re:Linus is an asshat, imho (Score:4, Insightful)
I actually disaggree. This is not a private company where you are trying to spare people's feeling. It is a case of "you screw up, you get yelled at." Because it is a publicly managed project, the yelling happens in public. It also set the standard for every single other developper. The next time a user space bug is introduced. The kernel developer will not try to swipe it under the rug.
Re:Linus is an asshat, imho (Score:5, Insightful)
Bottom line, Linus has been in charge of the kernel for a couple of decades now, and his process is working. He doesn't flip out very often, so when he does, every who counts knows that some serious fuckups happened. And anyone who doesn't count (which in this case, almost certainly includes both you and me) can probably go fuck themselves if they don't like it.
Nobody in the world knows for sure how to manage a project like Linux. It is truly unprecedented. But Linus's way is working better than anyone would have expected two decades ago. If you want to second-guess him, feel free to start your own competing project and manage it your way. His way is working better than anything anyone else has ever tried, even if it's not "the best way".
Re:Linus is an asshat, imho (Score:5, Informative)
For example:
Re:I love linux but... (Score:5, Insightful)
There are some people who you could site down with and explain things for two hours and they would still go away and do their own thing. IT is particularly full of people like this (and I'm aware of and try to avoid my own inclinations in this direction). Sometimes, you just have to make it clear that something is important and that means raising the temperature even as (and especially as) you remain in control internally.
This is also a valuable parenting skill. I don't think that that's entirely coincidental.
Re:I wonder (Score:5, Insightful)
To be honest, I wish it happened more.
I _wish_ management where I am cared about quality at this level. It would be worth getting chewed out once in a while to know that stupid bugs are not tolerated.
Re:I wonder (Score:5, Interesting)
Infrequently. I've seen somebody get hit that hard a handful of times in 20 years. I've had to do it myself twice. (There have been other lower-level chastisements.) Once I was the point man for a major reaming of a contractor that had been deceiving us for months about multiple issues. The other time was a guy that had been concealing his lack of progress on a project I was directing.
When somebody has engaged in completely unacceptable behavior, it's absolutely necessary to do at least one of two things: (1) get them off your project or (2) otherwise make them understand why such behavior cannot be tolerated. The latter is only an option if you believe the person is able and can be convinced to learn from their mistakes.
Doing it by email is a mistake and Linus can be criticized for that and, IMO, that alone. This is the kind of conversation you have face to face if you can and by phone if you can't.
Re:Really, Linus? (Score:4, Insightful)
This isn't backwards compatibility. It's the same basic rule we have to follow where I work: you DO NOT EVER make unannounced/unplanned changes that break the way other modules interact with yours. Period, end of line. Break that rule with any sort of regularity and you'll be looking for a job. If you need to make a change like that, it's on you to justify why and then to coordinate with everyone else to make sure everything that needs changed gets changed. It's not a hard rule to understand or to follow. If you can't, you have no business messing with an operating system kernel.