Linus on GIT and SCM

An anonymous reader sends us to a blog posting (with the YouTube video embedded) about Linus Torvalds' talk at Google a few weeks back. Linus talked about developing GIT, the source control system used by the Linux kernel developers, and exhibited his characteristic strong opinions on subjects around SCM, by which he means "Source Code Management." SCM is a subject that coders are either passionate about or bored by. Linus appears to be in the former camp. Here is his take on Subversion: "Subversion has been the most pointless project ever started... Subversion used to say, 'CVS done right.' With that slogan there is nowhere you can go. There is no way to do CVS right."

Merging *does* suck

[Anonymous Coward]

So don't do it.

It's always done late in a development cycle, in the rush to get the project out the door.

So don't branch, and DON'T allow concurrent checkout of any code - FORCE the DEVELOPERS who need to work on the same code to COORDINATE their work EARLY in the development cycle. Of course they'll bitch.

But so what. Developers always bitch when you make them do things right.

And you make the programmers coordinate their work properly and early by not branching, and not allowing concurrent checkout of the same file. In Subversion, you lock the file, and if anyone breaks the lock they get a very public visit from the lead developer asking why.

It works, too. You won't have any merge surprises two weeks from ship date.

If your technical leadership has the spine to show prima donna twits who won't follow development rules the door. Of the entire company.

Re:Merging *does* suck

[Anonymous Coward]

Wow, that's the worst idea I've ever heard

Re:Merging *does* suck

[Anonymous Coward]

I hope you're working for one of my company's competitors, if you are so eager to hamstring your developers and limit their productivity! Having to wait for someone else to finish a major piece of development before I can fix a bug in an unrelated section of a file they happen to be modifying... yeah, that's the way to turbocharge your development process.

Re:Why winge?

[Brandybuck]

The problem with CVS and Subversion are one of fundamental design. At least, that is what Linus is suggesting. You can't fix them without rewriting them completely from the ground up.

Well, Linus is an ass, what's new.

[suv4x4]

No one said that if you're famous and contributed something incredible to the world (such as Linux) you can't speak out of your ass most of the time, just because you enjoy how everybody listen and try to decipher if they should care about it, or just laugh and pass by.

I use SVN if a medium sized team and see SVN used extensively in all kinds of projects around the globe with great success. I personally love the workflow of SVN.

The only thing that they need to work is merging of branches, and incidentally I've talked to the developers, they're quite aware of this flaw of SVN and working on it. We'll see new versions that can track changes in each branch and even attempt automated merges with good success.

I know a guy who has the same personality like Linus. The guy is very smart, he single-handedly is coding an application which is very popular in its area (won't mention it since that's internal stuff). He keeps bitching all the time: about customer feature request, about random products and how sucky they are, how people can't see that. And he could also change his opinion overnight for no apparent reason and go in the other extreme. But he's a friggin' programming genius and what he does is great, despite is takes a lot of effort to deal with him.

Well, probably those two go together: being an amazing creator, and being an amazing ass with huge ego. Who knows.

Re:Why winge?

[zzatz]

Linus isn't saying that CVS and Subversion have fixable bugs or missing features. It's not about the code.

He is saying that they solve the wrong problem. The Subversion team wants to solve Problem A, and Linus wants to solve Problem B. No amount of code will turn the solution to Problem A into a solution for Problem B. Bothering the Subversion team with code addressing Problem B will only irritate them, since they're working on Problem A.

The right way to handle differing goals is to start a different project. That's what he did.

Don't be confused by the labels. Source Code Management means different things to different people, and there isn't always much overlap in how each person defines it. Ships and airplanes are both 'vehicles', but that doesn't mean that a few changes will turn one to the other.

Re:Merging *does* suck

[chez69]

if your working with a good SCM and have somebody with a clue who administers it (I've worked in a large clearcase setup for years, with a great admin staff) concurrent development isn't that hard to do. Good tools make the job easy.

file locking is ok for 2 or 3 developers, any more then that, it sucks bad.

Re:Lemme check my last home appraisal...

[feronti]

The thing is, you've got the wrong solution to the problem. Rather than not allowing branches, you need to control when and how often they're made, and how long they're allowed to survive. Your fixing a policy problem with technology, which never works well. If the branches are kept under control, you don't have the last-second merge problem. Merges should be happening constantly throughout the process so everyone stays in sync. If someone isn't committing their work at least once a day, that's when they get a stern talking to from the lead developer. Because if a developer needs to coordinate with another developer to change one line of code, then you've wasted two people's time instead of one.

~$ mv CommitAccess MergePrivileges

[Excelcia]

Linus talks about his distributed model, how everyone has a branch, and how this avoids politics associated with who gets commit access. He claims (and I admit I've seen this happen in some) that many projects have quite the internal politicking on who has CVS commit access. But then he claims that Git's special sauce eliminates these internal politics. Ok, I was intrigued, so I listened on.

Essentially, he explains, the secret with Git is that everyone has commit access on their own branch - they do whatever they want. He says that the way it works is that someone does something cool with their own branch, then they start hollering to say "Hey, I have a good branch, merge mine" and it will get merged. Politics over.

Ok, so now I'm scratching my head. How is this a fundamentally different paradigm? In CVS, basically anyone can check out the whole tree and make any changes the like. They can then say, see, my changes are good and ask for them to get committed or ask for commit access themself. In Git, this commit access bottleneck is just moved from the commit stage to the merge stage. You make your changes, commit them to your separate and unique branch, and then ask someone with to merge it, or give you the ability to merge it in to mainstream. How exactly does this eliminate the politics? You are still going to have some people with "the power" and some people without. In any project where you have people who are going to fight about who gets commit access, you'll just have a fight about who has the ability to merge into mainstream.

So, ok, distributed is nice (though for some projects central may be preferred) but I don't see how this magic system bypasses politics. In fact, I can potentially see more internal politics over this method. I can see factions gathering to support this or that branch, arguing about which is better, fighting about which one gets merged in. I can see the potential for branches going longer between merges, and more changes happening at once, making it harder to track problems. I don't claim these scenarios are more likely, but I do claim that this changing from a commit access to a merge access paradigm is just renaming the problem.

Re:Lemme check my last home appraisal...

[KyleCordes]

I wrote about Linus's talk a few weeks ago:

http://kylecordes.com/2007/05/17/linux-git-distrib uted/ [kylecordes.com]

Looking back at that, and at your comment, some things come to mind:

* the tool Linus is pushing, greatly facilitates the idea of frequent, easy merges, and Linus mentions that a tool with great, fast merges, helps you merge early and often.

* on the other hand, your comment is about "you need to control when and how often [branches] are made...", while a big point of distributed SC tools is the opposite of that control: these tools make the power of the tool fully available to all users. A "main" repository may (and probably should) have permissions/hooks set to enforce some policy about what happens to what branches. Individual users can always create local quasi-branches by simply not checking things in; with a tool like they can can create real (local) branches too, which can then be promoted to official status (i.e. on a blessed central repository) if needed.

Re:Merging *does* suck

[Timothy Brownawell]

So don't do it.

The Wise adapts himself to the world. The Fool adapts the world to himself [revctrl.org]. Therefore, all progress depends on the Fool.

It's always done late in a development cycle, in the rush to get the project out the door.

Why? It doesn't have to be. At least if you use something that isn't horribly broken [red-bean.com].

So don't branch, and DON'T allow concurrent checkout of any code - FORCE the DEVELOPERS who need to work on the same code to COORDINATE their work EARLY in the development cycle. Of course they'll bitch.

Yes, they will. Because this is a monumentally stupid idea. Because the entire *purpose* of revision control systems (note: "CVS" stands for "Concurrent Versioning System") is to make it possible for developers to work on things at the same time. The idea is that you can get more benefit from the concurreny than you get difficulties from merging.

If your technical leadership has the spine to show prima donna twits who won't follow development rules the door. Of the entire company.

Rules like "merge early, merge often", perhaps? Fixes the problem, and *doesn't* cripple development horribly like your idea would.

it'll never catch on

[sohp]

Distributed version control the way git does it (conceptually, not necessarily the implementation) is the best idea in SCM since concurrent development and optimistic merge conflict resolution on check-in.

Notice how, even years after better ideas superceded the lock-modify-unlock paradigm, many tools and shops still use exclusive-lock SCM.

It could be quite a while before you see anything like the way git does SCM in use in the majority of programming shops.

Re:Well, Linus is an ass, what's new.

[Anonymous Coward]

Academics usually have some expertise in their respective area, Linus does not. He is not a genius when it comes to OS design or development. The Linux OS broke no new ground; it was an imtation of existing Unix systems. Linus digs in his heals and refuses to cede ground even when he is wrong (or rants about untested design).

Re:There's a difference between GIT and SVN

[Anonymous Coward]

> However, in smaller projects, which really *need* a very specific direction

Even bigger projects need specific directions, witness GCC. I am sorry but everyone who thinks distrubuted SCM are a good thing. I am going to say they are a bad thing for free software because they allow people to develop stuff in private. Yes this already happens. This is why for an example GCC's rules for branches are that they are free for all and they are used a lot.

Take a look at http://gcc.gnu.org/svn.html [gnu.org] and see how many branches there are, though most of them are inactive but some are very active (and getting ready to be merged into the trunk). I guess distrubuted SCM allows for people not to develop in public any more which I say is a bad thing. I have been working on a branch of GCC and who ever says merging is hard is wrong (I am dealing with an IR change which touches all front-ends and 50% of the middle-end, the tree level and I am able to merge at least once a week and the merge sometimes fix some of the regressions I was trying to fix before :)).

I don't know about you but having a policy of branches are free for all is a good thing and it causes what distrubuted SCM will cause which is more development and a "private" tree (though it is public but the branch is yours to deal with).

I guess people don't see the bad side of distrubuted SCM that much because they don't deal with projects like GCC. The kernel has the same issue and I think Linus does not get the idea that public development is a good thing.

I could have developed all of my branch (pointer_plus) in a private local tree but I would not get some of the testing I have been getting from people I did not expect to be testing my branch. Plus I don't have all the resources that other people are putting into the testing that they do.

Thanks,
Andrew Pinski

Re:Linus knows it.

[Anonymous Coward]

Yeah, but he's really mean!

Re:Distributed version control gaining ground in F

[A beautiful mind]

Richard Dawkins spent a good deal of time in his book, "The Blind Watchmaker" talking about what the gradualist and the punctuationist view of Darwinism is. His gripe was that the latter was sold as a whole new theory, opposing the old gradualist view. Dawkins was rightly pissed about this, because the latter is merely an improved version of the former. I feel the same about the Centralized vs. Distributed topic. The distributed system is basically a centralized system where EVERY COPY HAS FULL REVISION HISTORY.

There is still a central or main copy, otherwise you'd be herding a lot of slowly diverging forks! Most projects want to produce a release eventually and there is a main copy of sourcecode which the release is produced from.

Imo, the reason Linus dislikes SVN and CVS and pretty much everything else is because of speed, because most SCMs lack the ability to work with merging different copies of repositories and work on a commit level instead, and do not allow for easy development routing around the central copy.

Re:git

[Just Some Guy]

His opinions should carry some weight, especially since he should know more than anyone what the limitations of SCM software is when it comes to larger projects like the linux kernel.

The thing is, Linux is actually a pretty small project. Much larger projects would include FreeBSD, which uses CVS not only for the kernel but for every line of source of the entire OS. Now, Linus is a smart guy, but I don't know why he thinks CVS (and SVN by extension) won't work for large projects. It clearly can. It may not be suitable for the way he wants to run his project, but that's a different issue.

Re:git

[SnowZero]

Yeah and luckily the whole "haves versus have nots" on who gets CVS commit access rights has never, ever, been a problem in *BSD or XFree86. Right?

Seriously, centralized version control fails for large open source projects for political reasons, not technical ones. That's really Linus' main point, although his lack of tact in presentation is going to cause many people to miss that insight. With a changeset-based distributed version control system, you only have to trust patches and code, not people. The whole concept of "the chosen few who get commit access" goes away, and problems like the XFree86/X.org fork or the EGCS/GCC semi-fork disappear.

Re:git

[EsbenMoseHansen]

Yeah and luckily the whole "haves versus have nots" on who gets CVS commit access rights has never, ever, been a problem in *BSD or XFree86. Right?

I'm sure it has been a big problem, but e.g. in KDE (which is also quite large), getting commit access isn't exactly hard. And anyway, SVK provides decentralized versioning backed by a central repository, so SVN doesn't preclude this.

Seriously, centralized version control fails for large open source projects for political reasons, not technical ones.

I agree completely, if it fails at all.

That's really Linus' main point, although his lack of tact in presentation is going to cause many people to miss that insight.

His lack of tact is legendary. But his lack of tact is what makes people read his every comment, and I hardly believe I am alone in smiling when he spews out those one-liners.

With a changeset-based distributed version control system, you only have to trust patches and code, not people. The whole concept of "the chosen few who get commit access" goes away, and problems like the XFree86/X.org fork or the EGCS/GCC semi-fork disappear.

Disappear? Hardly, the patches still need to be accepted. But decentralized repositories are wonderful to work with, and as such removes a technical hindrance.

Each man differs in his opinion of versioning software. For fun, here is mine: (only those I've tried)

• git is nice (handles e.g. moves better than svn)
• tla is too bothersome
• svn is very well supported
• cvs has too many strange cornercases.

Re:Merging *does* suck

[OrangeTide]

Your method does work, but it is not the best method. coordinated incremental merging of sub-projects throughout the entire development life-cycle also works, assuming you have a few developers who are detail oriented enough to merge code reliably. Also one problem with merging is that most shops don't code-review merges, which is just nonsense in my opinion.

Branching often and avoiding double-commits seems to be the way to enable parallel development. (double commits as in committing the same change to two branches, you should merge before you get to that point)

Also just because developers bitch doesn't mean they are wrong. Developers tend to whine when they think they have some pointless burden placed on them that is preventing them from doing their job. (their idea of what their job is and the management's idea of their job rarely match up perfectly). Developers often promise schedules based on the assumption that they won't have to merge or suffer a "code-freeze" or be blocked by arbitrary rules used to beat developers over the head when management does not think they are coordinating.

the more often you merge the less work it is. when you merge branches constantly the changes end up being trivial and you don't have to stress and fuss over a bunch of conflicting changes. it helps if you can hold a quick meeting with the two or three people that produced a merge conflict, so this technique really only works if your team is all in the same building/campus.

I view development as a dance, we all step on each other's toes for a while but once you get used to your dance partners you get better at it. it just takes a long time with developers because they are antisocial and not hugely team oriented. Developers, in general, hate depending on other people to do their job. They also are quick to blame others for failing to take them into account (another antisocial behavior is assuming everyone around you is a mindreader, another word is egocentric)

it's not that developers are prima donnas, which is something software managers often assume. it's that some personality types don't realize that there is actually anyone else around them. often developers have no idea that any of their peers even do any work. they might as well be equivalent to furniture, something to avoid tripping over in the hallway, but otherwise irrelevant.

Re:Is it my imagination???

[Anonymous Coward]

Or is Linus turning into a replica of RMS, only without the manners?

Nonsense. Unlike RMS, Linus takes himself way too seriously to do anything wacky and self-effacing like say, dressing up as St. Torvaldis, complete with a cardboard halo. But he's still nicer than Theo de Raadt.

Re:Well, Linus is an ass, what's new.

[SnowZero]

No, wait: I recognize the benefits if his system. The problem is, his system has benefits with open source projects at most.

I have several projects which have not been released as open source, and I can state for a fact there are benefits.

But here's what: in OSS, he can afford to "reject 99% of the branches out there". This is because he "believe[s] most of you guys are incompetent idiots".

What you may not realize is that those branches still feed code to his tree, through delegation. Linus trusts 10 people, and they in turn trust 10 people, and in a few levels you can accept code from all those trees through an established web of trust.

In a small team, we don't throw 99% of our work out or keep a consistent base of developers who we believe are incompetent idiots.

Can you give brand-new interns commit access to your core system? probably not. Can I code review a patch from a brand-new intern and accept it if it meets the quality standards for our core system? Yes I can. A distributed VCS can take advantage of possibly less skilled developers because its about trusting the well-defined patch, not the person. While such a workflow is not always precluded by centralized systems, distributed VCS's make this workflow very easy and natural.

Instead, we work together, frequently communicate, have fast turnaround times, and often work with files that can't be actually merged together (such as design related files, AI, PSD etc.).

Centralization of the VCS itself has little or no effect on this. How would a distributed VCS inhibit your team from acting this way?

I clearly see the benefits of his system, but it shines for his own needs, SVN shines for the needs of the majority of small teams out there, and for more linear/classic style organized projects.

Like I said, you just don't know what you are missing. If a developer has ever checked in something to mainline that wasn't done, just so another person could use it, you've just found a case where you really wanted a distributed VCS. If you've ever wanted to commit code on an airplane, so that you could easily revert if a new idea didn't work, you really wanted a distributed VCS. If you've ever wished you could test two patches together before merging them into the mainline, to avoid affecting other developers if the integrated version was buggy, you really wanted a distributed VCS. I've done all those things, and our project has had only 2-5 developers. At the same time, if you want to, you can use a distributed VCS in a centralized and linear manner; The only difference is that your team now has the choice of workflow models.

It also works for the majority of small OSS projects, which can't afford to be spread in hundreds of branches at a time, as features are clearly defined, priorities as well, so there's no need to spread what we do in hundreds of branches by definition.

You aren't being forced to have hundreds of branches, just like SVN doesn't force you to make hundreds of checkouts. This is a red herring.

Also one of the benefits he mentions, basically everyone has his own branch and can diff locally with the other revisions, until someone "pulls" from him. That's handy but in its very basic core is the same as SVN cache. I can diff locally, save files as much as I want, revert files to the locally stored revisions etc., before I commit to the central repo. Not quite all the features he has, but as long as it does the job, that's all that takes.

How about commit so that you have a logical development history, separating changes into logical parts, such as a new feature and a random bugfix you found along the way that you'd like to also push to stable -- all while disconnected. Like you say, if you don't run into that, it's not a problem. However, when I used CVS, I didn't realize all the times where I would have used a distributed VCS feature, because at the time I didn't think in those terms. Ignorance was bliss. Now that I use a distributed VCS, I use it even for single developer projects; The extra features do come in handy, and they aren't a hindrance.

Re:Distributed version control gaining ground in F

[putaro]

What you're citing as advantages for decentralized version control are not the result of decentralization.

Cheap branches? Subversion has cheap branches.
Better merging? This is a result of algorithms has nothing to do with whether the system is centralized or not.

If you're on a fast net with the server you can commit as often as you like. If you can branch/merge easily it's no problem.

If you want to cite advantages for decentralized version control it might be more like:

If you have to talk to a server over slow links, decentralized is much better

Re:Why winge?

[Tickletaint]

But believe me, everyone who works in transportation and transportation policy is secretly thinking the same thing. Our dependency on the automobile is insane, and it is destroying us.

Re:Why winge?

[Eil]

Linus isn't saying that CVS and Subversion have fixable bugs or missing features. It's not about the code.

I think it's more about bashing some thing or another to gain attention.

I liked Linus, and I've held him in high regard for more than a decade for all that he's helped accomplished while still being mostly modest about it. But he seems to be slowly evolving into another Stallman lately. Everything has been, "I don't like x and therefore x is stupid and you're a mentally-retarded asshat if you don't agree with everything I say."

Just last week he started a flamewar on LKML about software suspend. Linus threw a shitfit when some bugs in suspend-to-disk were affecting suspend-to-RAM. After insulting a few other kernel developers without provocation, he basically ended the conversation by saying that the whole thing was going to be ripped out and redone only as suspend-to-RAM because he didn't use suspend-to-disk. Since he himself didn't use it, he postulated that it was a completely useless waste of time for anyone to implement it.

The Subversion team wants to solve Problem A, and Linus wants to solve Problem B.

So why couldn't he have simply said, "GIT solves the problem that I need it to solve, which is different from Subversion's"? Oh, right, because that wouldn't be interesting enough to make the front page of Slashdot. Too bad if the alternative alienates the half of the open source community that likes Subversion for what it does.

Re:git

[Raenex]

So long as he remains consistent, even-handed, and not ad-hominem, it's OK.

He has remained consistently even-handed in being ad hominem. Calling people who disagree with you morons is ad hominem. Calling them ugly and stupid is ad hominem. You can tear somebody's design apart without insulting their intelligence or character.

Somtimes he does this and tries to make a joke out of it, but more often than not you can see the real venom shining through.

OT: Linus' attitude

[SpaghettiPattern]

I watched the video and was baffled by Linus' attitude.

The guy is bright on technology but what he calls strong opinions I -and almost any business person- should call them shortsighted opinions that do not appreciate other people's way of doing things.

Saying stuff like "Those of you that like SVN would probably want to leave the room. Because it's crap too." is very stupid. I for one like SVN. It's good enough for my purposes and it sure beats quite a few commercial tools I've seen. But I sure want to know hear Linus' thoughts on the topic. Mainly because different ideas keep my mind fresh.