Stories
Slash Boxes
Comments
Search

News for nerds, stuff that matters

Slashdot Log In

Log In

Create Account  |  Retrieve Password

Getting a Grip on Google Code

Posted by Zonk on Fri Dec 01, 2006 03:55 PM
from the improving-on-the-old dept.
Businesses Google The Internet
netbuzz writes "Niall Kennedy reports on his blog that Guido van Rossum, author of the Python programming language, has begun showing off his first project since joining Google last year. 'Mondrian is a Web-based code-review system built on top of a Perforce and BigTable backend with a Python-powered front-end,' Kennedy writes. 'Mondrian is a pretty impressive system and is currently in use across Google.' Kennedy's description of Google's current code-review system sure makes it sound like it was in need of an upgrade. 'The Mondrian tool creates a much better workflow by creating task-specific dashboards, in-line commenting, well-tracked statistics, and more,' he writes. 'The application is built on top of Python open source libraries such as the Django framework, smtpd.py mail service, and the wsgiref Web server software.'"
+ -
story
This discussion has been archived. No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

Getting a Grip on Google Code 50 Comments More /

 Full
 Abbreviated
 Hidden
More
Loading... please wait.

  • Perforce? (Score:5, Interesting)

    by Doctor Memory (6336) on Friday December 01 2006, @04:21PM (#17072660) Homepage
    Good idea, building on a closed-source SCMS that's (barely!) a mid-level player in the market. I can understand not wanting ClearCase, but what's wrong with CVS or Subversion? Hell, even Monotone or GNU Arch...

    Oh well, could be worse: they could have gone with StarTeam, PVCS or MKS Source Integrity...
    • Anyone who's used perforce will understand. I read this and it made me very, very jealous.

    • Re:Perforce? (Score:5, Interesting)

      by panaceaa (205396) on Friday December 01 2006, @04:37PM (#17072954) Homepage Journal
      I'm not sure how you decided Perforce is a "barely mid-level player" in the SCM market. Adobe, Google, and Microsoft all use Perforce as their primary source code management solution. (Though Microsoft has highly modified it and calls it something else internally... but my contacts there tell me it's still Perforce underneath.) Perforce does have its problems with scalability, but in terms of merging, collaborating, viewing history, keeping branches, etc, etc, etc, it's pretty awesome.

      • Re:Perforce? (Score:5, Informative)

        by Electrum (94638) <david@acz.org> on Friday December 01 2006, @05:10PM (#17073550) Homepage
        Adobe, Google, and Microsoft all use Perforce as their primary source code management solution.

        Amazon does too.

      • Microsoft uses Perforce?! (Score:5, Funny)

        by mkcmkc (197982) on Friday December 01 2006, @11:24PM (#17077694)
        Microsoft is using Perforce for source code control? Why aren't they using their own product--Visual Source Safe? Does it suck or something?

        • Re: (Score:2, Interesting)

          by zeroduck (691015)
          Simple answer: yes.

          We use it at work, and deal with corrupted databases frequently. There's been talk of moving to SVN for a while, but I doubt they'd make a full switchover. We have years worth of projects stored in VSS, and it's really not worth moving them over (fortunately closed projects, so getting back to them is only necessary if any issues arise with a customer).

      • I'm not sure how you decided Perforce is a "barely mid-level player" in the SCM market

        Why, by using only the finest in high-tech market player profiling, to whit: I Googled for "<scm system> configuration management" and noted how many hits each got. My list worked out to:

        2.8M — Clear Case
        2.8M — CVS
        2.4M — Visual Source Safe
        1.8M — Subversion
        1.1M — CCC/Harvest (now CA AllFusion Harvest)
        900K — RCS
        665K — Perforce
        536K — PVCS
        378K — Aegis
        376K — Monotone
        186K — BitKeeper
        154K — StarTeam
        101K — AllChange
        68K — GNU

        • With good administrators, Perforce is comfortably scalable up to 500 to 1,000 developers. You just start running into lag problems once you get above that. Certain operations like branch integrates can lock up a depot for minutes, which doesn't sound bad until you realize that no developers on that depot can check-out files or make other Perforce client changes that whole time. Technically you'd run into the same problems with just 2 developers, but it gets really noticable when there's 3,000 developers
          • Why does MS not use the tools they sell to the rest of the world? For example, my organization does not use perforce but TFS for version control. Is TFS inadequate for large-scale development?

            I do not have much experience with lots of different versioning tools. I have only used TFS, CVS, and SourceGear.

            • Re: (Score:3, Informative)

              by panaceaa (205396)
              Microsoft dogfoods most everything, including Exchange Server (for @microsoft.com and @hotmail.com), SQL Server (for *.live.com), and internal Office betas (with pushes out to everyone, including admins). But for source control, their own products just don't scale up to 60,000 employees. I've heard there's dogfood initiatives for VSS, but developers in Windows Client and Office vehemently oppose them. I'm not sure if VSS is used in other areas of the company, like perhaps the Live.com groups and Xbox/Zun
              • VSS was used internally at microsoft by some groups for a while, but it's been replaced by their version of Perforce, which is in turn slowly being replaced by microsoft's own TFS which scales quite well to large projects/teams. check out dogfood stats here [msdn.com]
    • Why would linux kernel maintainers have used a proprietary SCMS all these years, if it wasn't simply the best suited tool for that purpose? (bitkeeper)
      • Schapsmann wrote:
        Why would linux kernel maintainers have used a proprietary SCMS all these years, if it wasn't simply the best suited tool for that purpose? (bitkeeper)

        Hypothetically, because the developer was a friend of Torvald's, and he talked him into it.

    • Re:Perforce? (Score:5, Informative)

      by slamb (119285) * on Friday December 01 2006, @04:41PM (#17073060) Homepage
      Good idea, building on a closed-source SCMS that's (barely!) a mid-level player in the market. I can understand not wanting ClearCase, but what's wrong with CVS or Subversion?

      I use both Subversion and Perforce. There's one major feature still lacking from Subversion: merge tracking. There's work underway [tigris.org] to design, implement, and document this feature, but it's not done yet. This is a huge deal for anyone with lots of branches.

      Not that it's all roses with Perforce. My impression is that it doesn't scale very well. Most operations simply lock the entire database. I think it's a reader/writer lock, but it means that (for example) while the hour-long checkpointing pre-backup process happens every night, you can't do any write operations. (And there's a way to do an offline checkpoint, but it's not documented or supported, and is difficult to get right, with bad consequences if you don't.)

      • Re:Perforce? (Score:5, Interesting)

        by slamb (119285) * on Friday December 01 2006, @04:48PM (#17073172) Homepage
        It means that (for example) while the hour-long checkpointing pre-backup process happens every night, you can't do any write operations.

        Let me be a little more specific: while the hour-long checkpointing process is happening, you can't even open files for edit. In addition to having really course locking, Perforce has more write operations than most version control systems. Subversion's CVS-style working copy means the only write operations are commits and revpropsets.

      • Re:Perforce? (Score:5, Informative)

        by Mike McTernan (260224) on Friday December 01 2006, @05:46PM (#17074164) Homepage
        > Not that it's all roses with Perforce. My impression is that it doesn't scale
        > very well. Most operations simply lock the entire database.

        I agree - the backup solution described and recommended by Perforce works well for small installations, but doesn't scale very well in my experience. It's disappointing given that Perforce use scalability as a selling feature (http://www.perforce.com/perforce/products.html).

        I went on a limb and made an alternative way to do checkpoints/backups for exactly the reason you describe - it's difficult to get right and seriously bad if you get it wrong. The write up of what I do is here:

        http://www.mcternan.co.uk/PerforceBackup/ [mcternan.co.uk]

        In my opinion it would be simple for Perforce to implement some simple changes to help large scale backups (e.g. make p4d -jj -c "cmd" work), and I've suggested it to their support staff, some of whom I've met in person at various times. However, I haven't heard or seen any indication that they are going to do this... I'm still hopeful, but less so these days.

        I also believe that Perforce only does locking at the table level (using flock()), which is most likely why the server often sees poor concurrency, especially with write operations as you describe. The more recent versions of the server are apparently better (2006.x), although I'm yet to upgrade. The server itself is based on SleepyCat Berkley DB tables, which Oracle recently took over and look to have improved (http://www.oracle.com/database/berkeley-db/db/ind ex.html). So maybe future versions of the Perforce server will benefit too. I hope.

        • Re: (Score:2, Informative)

          by slamb (119285)

          The write up of what I do is here: http://www.mcternan.co.uk/PerforceBackup/ [mcternan.co.uk]

          Interesting! I'll have to look it over more later.

          For comparison, I've put the latest (not yet deployed) version of our offline checkpoint process here [slamb.org]. (It's a NetVault backup script; pre locks and does the checkpoint, post touches a file signalling success to our monitoring and releases the lock). It's a procedure outlined by Perforce, though they didn't mention error handling...

      • I use both Subversion and Perforce. There's one major feature still lacking from Subversion: merge tracking. There's work underway [tigris.org] to design, implement, and document this feature, but it's not done yet. This is a huge deal for anyone with lots of branches.

        Aye, the SVN team is definitely not sitting on their laurels (yet) after finally hitting the 1.0 release a while back. They made significant improvements in 1.4 and have more up their sleeves for the upcoming 1.5 release. With even more t
    • Microsoft uses a modified version of the Perforce SCM for all of their internal code management. I personally use Subversion on a daily basis at work and on my SourceForge projects but I guess if it's good enough for Microsofts development teams it's good enough for Google to build a custom version too.

    • Re: (Score:3, Informative)

      by novitk (38381)
      The main reason for starting SVN was that a lot of things were wrong with CVS. Arguably SVN(nevermind Monotone, Arch) has only recently approached Perforce level of stability, scalability and functionality. They needed something workable probably at least five years prior. ClearCase is clearly not a Google-style solution.

      Looks like a good choice to me.

    • Re:Perforce? (Score:4, Informative)

      by mattcoug (873342) on Friday December 01 2006, @07:32PM (#17075838)
      FYI - Guido built this system to work within the existing Google infrastructure, he didn't choose Perforce for the project. Guido also wants to eventually refactor it to work with many SCM including Subversion, CVS, etc. BTW, Perforce is used at many very-large-software-companies, so while it is not perfect, it is still very useful.

  • Subversion? (Score:1, Interesting)

    by Anonymous Coward
    Am I missing something here? Why don't the use Subversion?

  • Codestriker is the same thing (Score:2, Insightful)

    by Anonymous Coward
    Codestriker [sourceforge.net] does the same thing. Except it is in perl + GPL, on source forge.
  • Joel Spolsky (of JoelOnSoftware fame) talked yesterday [joelonsoftware.com] about how Microsoft handles their enormous build system. Apparently it's built on a fork of Perforce that MS paid for some time ago. Could this be an early warning sign of Google becoming Micro-soft?

    Disclaimer: I've yet to work with Perforce, having not yet graduated from CVS, but at least I'm not using VSS.

    • Re: (Score:1, Interesting)

      by Anonymous Coward
      1) There are thousands of companies that use Perforce. Does that mean they all want to be Microsoft?

      2) Of the thousands of companies that use Perforce, many of them started using Perforce before Microsoft. So wouldn't that make Microsoft the copycat of them?

      3) Perforce is a popular choice among companies that need a version control system that can handle very large code bases with high speed. Few version control systems can do this adequately. Perforce also has excellent branch management features. The
    • It took you until now to see an early warning sign?
  • You can't check out files with both Unix and Windows line-endings. See http://smithii.com/perforce_bugs [smithii.com] for the ugly details.

  • Blargh! Mondrian is already an open-source OLAP (Score:5, Informative)

    by Anonymous Coward on Friday December 01 2006, @05:25PM (#17073782)
    Blargh! Mondrian [pentaho.org] is already an open-source OLAP engine! Seriously, a casual google search could tell you that. And it's not some sf.net abandonware, it's a mature and powerful OLAP Cube engine used by some big-name corps!

    Oh, and just to rant a bit more: Python WAS ALREADY THE NAME of the Lisp Compiler used in the CMUCL Common Lisp implementation and lately SBCL. And was relatively well known in computing science at the time Guido was naming python because it is a snazzy type inferencing lisp compiler!

    Guido's some sort of naming-dick. What'll he call his next python project? Glibc? Mesa? Gimp?

    • I nominate Gimp. It's represented a crappy image editor with nonsensical UI for too long. It'd be awesome if it was something cool, like Web 2.0 Pong.

    • Re: (Score:1, Funny)

      by Anonymous Coward
      Guido's some sort of naming-dick. What'll he call his next python project? Glibc? Mesa? Gimp?

      Firebird!

    • Mondrian is also the name of an experimental functional language [zoot.net.nz] for the .NET platform written by Nigel Perry.
      It's also a Haskell dialect described in this 1997 paper [chalmers.se].
      I really don't think it's anything worth getting worked up over. This is an inhouse program that was started as a side-project and is unlikely to be released (if at all) for quite some time, I think it's quite likely he just picked the name since it already had a Google-fied logo (they've used that before for their frontpage on (the artist
      • Can anyone explain why you would want to name a software project after the world's dullest modernist painter?

        And I'm not particularly down on Modern art myself... maybe I should name a project "Rauschenberg" (except that he's not dead yet).

  • Google (Score:3, Insightful)

    by synx (29979) on Friday December 01 2006, @05:30PM (#17073878)
    This is why working at google is awesome. Internal code reviewer is big news.

    I use the tool in question, it's good.

    Also I've used perforce at a previous company. Generally most people who talk about SCMs and reference CVS as a potential replacement/alternative to P4 really do not know what they are talking about. P4 has it's problems, granted, but if you are looking to maintain a massive code base, there really are few choices. Atomic change lists, they are fantastic.

    • Re: (Score:1, Insightful)

      by Anonymous Coward
      I'm somewhat surprised that Google is using P4 instead of BitKeeper, given that BitKeeper's author was employee #3 or #4 at Google.

    • Re: (Score:3, Interesting)

      by LauraW (662560)

      Agreed. I use this too, and it's great, especially if you feel like being a Code Nazi and making nit-picky comments about code that you're reviewing. You just double-click on the line you don't like and type in your comment. It's much easier to use than reading the code in a diff tool and then typing your comments into a separate email window.

      As far as perforce vs. CVS goes: I used CVS at my previous company and liked it. It was certainly a big improvement over CMVC when I was IBM and the SCCS wrappe

  • This is a nice design win for Django as a web framework. I wonder how much of the stack he ended up using and whether he used the ORM layer at all.

    • Mmm no, what google knows it's about is "the tool that fits the task best".

      In this case -- the creator of the Python language having to build a webby app -- the obvious tool was much more likely to be Python than, say, Java. Or even actually good languages like Smalltalk, or Forth, or Erlang.

      • Re:I can see why Google stuck with Python. (Score:5, Insightful)

        by Peter Cooper (660482) on Friday December 01 2006, @04:07PM (#17072398) Journal
        8 bit characters is exactly what it /does/ support. It's multi-byte characters that are often seen as the problem, although UTF-8 is also supported (Unicode generally, however, is a different matter). Ruby can also support load balancing and HTTPS.. although since those aren't relevant to a programming language per se, it's intriguing why you bring them up (unless I've fallen for a troll, in which case.. well done ;-))

        • although UTF-8 is also supported (Unicode generally, however, is a different matter)

          Uh, UTF-8 is a Unicode Transformation Format, that's usually (that or UTF-16) what people talk about when they mention "Unicode". And Ruby definitely sucks at anything out of the ascii character space, be it inside or at the boundaries (interfacing with the outer world).

          • UTF-8 is the de facto standard encoding of Unicode characters, but in the interests of not having someone respond with "but Unicode != UTF-8" I thought I'd add the proviso (although it opened me to complaints of the opposite nature, of course ;-))
        • Try giving a deep, suculent "yard of tongue down your throat" [duke.edu] soul kiss to the suspected troll. Watch in the closet door mirror: If their spine lights up IT'S A CYLON! FOR GOD'S SAKE RUN! Otherwise it's probably a troll.

          If you want to go ahead and have sex with the Cylon before you run, that's your business.

        • Re: (Score:1, Insightful)

          by Anonymous Coward
          I think the problem WITH Ruby is that its making the same mistake as Smalltalk. Everything has to be a object, but this isn't always true. Some things are better modeled with generic functions, or other Functional style paradigms. Python is picking from both worlds. I think my ideal language would be a blend of Haskell and Python in some form. Ocaml comes close, but has it's own weirdness, with '+' and '+.' etc etc.

          IE:

          Why do numbers need to know about iteration? From Smalltalk:
          1 to: 3 do: [ Something ]

          There

      • AFAIK stuff like load balancing or HTTPS are handled by HTTP servers/balancers/fronts (or at least TCP balancers for load balancing) and stuff, not by the language used to create the site... If you're going it an other way, you're likely to be doing it very wrong.

        And apart from that, modern Ruby webapps actually have fairly good deployment solutions, e.g. Mongrel.

        Unicode in ruby still sucks though

        • Re: (Score:3, Informative)

          by mini me (132455)
          Unicode in ruby still sucks though

          It could be better, but it's not that bad:

          >> message = "¼ and ½"
          => "¼ and ½"
          >> message.chars.length
          => 7
          >> message.chars.last.to_s
          => "½"
          >> message.chars.normalize.to_s
          => "1/4 and 1/2"

All trademarks and copyrights on this page are owned by their respective owners. Comments are owned by the Poster. The Rest © 1997-2009 Geeknet, Inc.