Follow Slashdot blog updates by subscribing to our blog RSS feed

 


Forgot your password?
Close
typodupeerror
×
Debian Programming IT Technology

The Great Computer Language Shootout Revived 51

Posted by timothy from the sputters-to-life dept.
An anonymous reader writes "Doug Bagley's famous Great computer Language Shootout more or less died in 2001 out of lack of support by its own author. A group of developers have decided to revive it and update it with the latest versions of each compiler and interpreter available on the Debian distribution. The good news is, a wiki has been set up so that people can help improve the shootout, discuss the implemetations of the programs, and suggest optimizations."
This discussion has been archived. No new comments can be posted.

The Great Computer Language Shootout Revived More

The Great Computer Language Shootout Revived

Comments Filter:

  • Judging by the lack of comments... (Score:2, Funny)

    by Anonymous Coward
    Judging by the lack of comments in here, I'd say it isn't just the author who lost interest.

  • And that'd be...? (Score:2, Funny)

    by Anonymous Coward
    ...and update it with the latest versions of each compiler and interpreter available on the Debian distribution.

    I'm fighting the urge to ask how C compiled with gcc 1.3 will compare to COBOL...

  • gcj vs Sun? (Score:3, Informative)

    by BRSloth ( 578824 ) <julioNO@SPAMjuliobiason.net> on Wednesday June 16, 2004 @02:00PM (#9444568) Homepage Journal
    They could have added comparsions between gcj and Java.

    At first, when I saw the Java comparsions, I tought "ok, that will send the last report that says that Sun Java is faster than g++ to space!" but checking the "Implementations" section, it showed that they used gcj for its java comparsion.
    • Actually, it seems that they are using gij, which is the Java interpreter from the GCC team. It would be interesting to also get the results when natively compiled with gcj.

  • Flawed (Score:5, Informative)

    by Jordy ( 440 ) * <jordan@COWsnocap.com minus herbivore> on Wednesday June 16, 2004 @02:11PM (#9444666) Homepage
    Going through some of these sample programs, I see some serious flaws. The C implementation of regex just calls pcre. The C implementation of Hello World calls fputs instead of write(). The C++ implementation of hash uses sprintf and strdup. The C and C++ implementations of the fibonatchi sequence test are recursive. The tests themselves are so short that you are measuring the time to load the binary into memory and cleanup for half the tests.

    Really there should be some automated way to submit a replacement for some of these tests that gets peer reviewed. That way each language has the "best" implementation for the language.

    Of course, it would have to be considered the "best practices" use of the language as you could always write C in C++ (call write() for Hello World for instance).

    • Re:Flawed (Score:4, Informative)

      by Vaevictis666 ( 680137 ) on Wednesday June 16, 2004 @02:25PM (#9444807)
      Keep in mind that the purpose is to test a "method"

      The fibonacci test, for example, is not testing fibonacci number generation, it's testing recursion. (Same deal with the Ackerman test) If you were allowed to do it iteratively, you're not testing the right thing.

      Also, if you want to submit a replacement that you think will do better, feel free. Just make sure it's doing what the test spec says.

      Please, do read the methodology [debian.org] page, it's there for a reason.

      • Re:Flawed (Score:3, Informative)

        by Jordy ( 440 ) *
        Ok, I can understand that one. The rest I'm not sure about. Surely using strdup() and sprintf() in a C++ program to hash is wrong or using fputs() to output a string. Plus, how the heck did they use 2 megs of memory for Hello World?

        Static:

        jordy 21022 0.0 0.0 388 60 pts/1 T 12:39 0:00 /home/jordy/hello

        Dynamic:

        jordy 21038 0.0 0.0 1344 228 pts/1 T 12:40 0:00 /home/jordy/hello
    • Going through some of these sample programs, I see some serious flaws.

      Well of course! How else are you going to promote your own favorite language unless you sabotage those you don't like?

    • Re:Flawed (Score:5, Informative)

      by Sigma 7 ( 266129 ) on Wednesday June 16, 2004 @02:52PM (#9445198)
      The C implementation of regex just calls pcre.
      Yes. If you have a better regular expression library, you're welcome to submit as a more efficient submission.

      The C implementation of Hello World calls fputs instead of write()
      fputs() is an ANSI C standard function designed to write text to an output stream or file. Write() is not ANSI C compatable, requires knowledge of the length of the string (either truncating if the length parameter is too low, or writes garbage if the length parameter is too high), and is designed for binary output rather than text.

      When writing a portable C application, you should *NEVER* assume that a platform is using anything other than the ANSI C libraries except when absolutly necessairy (e.g. a mutlimedia library.)

      The C++ implementation of hash uses sprintf and strdup.
      Correct it then. The webpage containing the benchamarks has links available to contact the maintainers of the website so that you can submit a more efficient function.

      This, of course, assumes that the HM class is not affected by the buf[] string being overwritten. If it is, then you'll have to find another way to eliminate the sprintf/strdup inefficiency.

      The C and C++ implementations of the fibonatchi sequence test are recursive.
      The fibonacci numbers test is explicitly stated that the procedure must be use recursion. If you want, you can ask the website maintainer to add a new test containg the iterative counterpart.

      Just because there is an absolute best way of doing things does not mean that it is permitted in the test. Read the Testing Methodology [debian.org] for more information.

      The tests themselves are so short that you are measuring the time to load the binary into memory and cleanup for half the tests
      That's why the webpage provides an option to exclude startup time in the performance metrics. The option to do so is located at the top of the table, just below the Title naming the test.

      • Re:Flawed (Score:5, Insightful)

        by Jordy ( 440 ) * <jordan@COWsnocap.com minus herbivore> on Wednesday June 16, 2004 @06:04PM (#9447072) Homepage
        fputs() is an ANSI C standard function designed to write text to an output stream or file. Write() is not ANSI C compatable, requires knowledge of the length of the string (either truncating if the length parameter is too low, or writes garbage if the length parameter is too high), and is designed for binary output rather than text.

        write() is POSIX/BSD/SVr4 which is certainly fine for the vast majority of platforms. Even with doing the strlen() and a const, it is cheaper than fputs() due to the locking fputs() puts on stdout (admittedly nothing compared to loading the binary into memory). Further, even if you wanted to use ANSI C, they should have just used puts().

        Correct it then. The webpage containing the benchamarks has links available to contact the maintainers of the website so that you can submit a more efficient function.

        Every single test for every single language is almost certainly flawed in some way. If they wanted to do it right, they should have allowed open submission and peer review of every test so that they could be written and rewritten then tested.

        The point is, the person running this is certainly not an expert in every single language. I doubt very much that they can make a proper decision between four different and seemingly correct implementations of the hash implementation. Just submitting replacements doesn't make much sense because *my* implementation could raise just as many questions as the up right now.

        This, of course, assumes that the HM class is not affected by the buf[] string being overwritten. If it is, then you'll have to find another way to eliminate the sprintf/strdup inefficiency.

        Oddly enough, the "correct" implementation from a best practices standpoint may be slower by using std::string instead. It all depends on what your definition of "correct" is. That is why an open peer review process would be helpful as the feedback would educate people as to why certain design choices were made.

        The question you have to ask yourself is, do the results on this page reflect reality in any way?

    • System Calls (Score:3, Insightful)

      by Detritus ( 11846 )
      The C implementation of Hello World calls fputs instead of write().

      write(2) is a system call on UNIX systems, not a part of the C language or the C standard library. It has no place in a portable C program. You might as well argue that the program should have use DosWrite() (OS/2 system call) or $QIO (VMS system call).


      • There's this little standard called "POSIX". A very large number of OSes, including many that aren't Unix, support POSIX. OpenVMS is one example.
        • A large number of operating systems do not support POSIX, support an incomplete subset, or have a broken implementation. If it isn't in ISO/ANSI C, it isn't standard or portable.
    • That way each language has the "best" implementation for the language.

      The whole point was what a medium - good programmer does rapidly. An immitation of real life corporate programming where the programmer sort of cares, they think about it a little but not too hard and the levle of programmer is good but not great. The purpose was never to test the languages using "best implementation" but rather measure likely implementations of common problems.

  • If it's true, it's great news (Score:3, Interesting)

    by GCP ( 122438 ) on Wednesday June 16, 2004 @02:23PM (#9444777)
    I find it quite frustrating that there is such huge inertia in programming languages. Even when languages have some remarkable advantage, programmers won't use it because "nobody uses it."

    I don't deny that popularity provides some huge advantages. I just hope that there can be mechanisms to bootstrap popularity, and this Great Language Shootout is (potentially) the sort of thing the could do that.

    Some languages, for example, have significant advantages in some specialty area: Erlang in parallelism, Lisp as a language for writing specialty languages, bug minimization approaches taken by Eiffel or the functional languages, etc. They may be awful for other types of work, but the fact that a hammer is a terrible saw doesn't make it any less of a great tool for the right problem.

    If there were an active, busy site frequented by developers at which there were a wide variety of benchmarks that allowed uncommon languages to show off their specialty advantages, more developers would do some of their work in those languages, and the increasing popularity would result in better implementations and a flourishing of new languages.

    Right now, the popular languages don't come close to taking full advantage of what we have learned about programming, and yet you would have to be either an academic or a fool to put a lot of effort into designing a great new language. The way things are right now, a great language designer probably has a better chance of being hit by lightning than he has of ever making his great new language popular.

    Anything that reduces this language inertia (and YES that includes .Net) is a Good Thing.

    • Probably because most of us want to be expert in a few languages than mediocre in many. It's like human language. How many people do you know are fluent in more than one language? Two languages? By the time you get to fluency in three languages, they're getting rare.

      You can't pick the language for the task, because in all probability you don't have the time to learn a new language for each task. There are many times when I have to write a piece of code in an "inappropriate" language simply because I don't

      • Re:If it's true, it's great news (Score:1, Insightful)

        by Anonymous Coward
        Multi-language development is incredibly useful - the last program I wrote has a C/C++/OCaml server with a GUI client written in Delphi.

        Unfortunately, it also means that nobody can easily maintain the program who isn't fluent in all four languages, which is why, while useful, it's also an incredibly bad idea for anything critical.

        The program in question is a quick and dirty solution to a migration issue, and using the mixture of languages allowed me to leverage all their strengths and produce a fast and r
      • As with human language, some people are naturals when it comes to being fluent in multiple programming languages. Unfortunately, I am not one of their number.

        That's unfortunate, because the ability to quickly learn to handle new specialty tools has significant competitive advantages--dare I call it "survival value"?

        And with all the things one needs to learn to move beyond junior programmer these days (things like XML, DHTML, accessibility, SQL, MS Excel, version management utilities, scripting for metapr
        • That's unfortunate, because the ability to quickly learn to handle new specialty tools has significant competitive advantages--dare I call it "survival value"?

          I can definitely learn a new language. There is no problem with that. But if they expect me to aquire complete mastery of the new language in a week, I will have to be honest and say I cannot do it. Frankly, of forty experienced software developers at my work, I can think of one who might be able to perform this feat.

          Can YOU become an expert in Lis
          • Can YOU become an expert in Lisp in one week?

            I've actually spent a number of years using Lisp. I've been paid to write software in somewhere between a dozen and two dozen languages, depending on your definition of language, and I assure you that your "acquire complete mastery of a new language in a week" test is completely irrelevant.

            If you haven't even started to look at a language until a week before you need to be a master, it's not the fault of the project. You simply aren't qualified for that partic
            • But, unless you never do anything until someone else forces you to do it, you usually have years in which to get to know a new language.

              But that contradicts you're earlier statement. Earlier you said that "the ability to quickly learn" new lanaguages was a survival advantage. Now you're saying I have years in which to do it.

              "Survival advantage" in context means that you must quickly adapt to and use the language your pointy headed buffoon has a hardon for. You can't produce good code in this situation. Y
      • It's more than that.

        In the real world someone is paying to get software done. That someone does not want to pay us to learn a new language just so a function doesn't need to be written because in language X it's a one liner. Write the equivalent function or find a suitable existing library. If you program in C/C++ the library probably does exist. In Haskell or Eiffel...

        Also real commercial products do not exist in a vacuum. You will probably have existing code that does at least part of the job. Th

    • I find it quite frustrating that there is such huge inertia in programming languages. Even when languages have some remarkable advantage, programmers won't use it because "nobody uses it."

      You want to look up "positive network externalities" from the field of economics. Basically, the idea says that something is more valuable the more people use it. For instance, if you are the only one to have a mobile phone, it's a worthless hunk of plastic, but if all your friends have one too, it's a convenient way t

      • Now *you* look up what I posted. I never said I couldn't understand why popularity mattered. I said that something like the Great Shootout could serve to bootstrap popularity. In an area like programming languages, which is subject to strong network effects, anything that could get the snowball started for a wider variety of languages would be useful--as I said.

  • Isn't this the same story as the one yesterday about how Java is faster than C++ under "suitable" conditions.

    ----
    Notes on Stuff [blogspot.com]

  • Oh Boy! (Score:2, Funny)

    by Rhesus Piece ( 764852 )
    Oh boy! There is a "Manufacture your own results" cgi! Now we can all play the "change the weighting so your language of choice wins" game!

    I changed everything to '1', and Ocaml won.
  • for(i=0;language[GCC]>language[i];i++) too_true++; Better make too_true a long long.

  • The Python results seem suspicious (Score:3, Insightful)

    by Colonel Panic ( 15235 ) on Thursday June 17, 2004 @07:40PM (#9458609)
    Apparently they used Psycho to get the Python numbers. This is why the Python numbers look so good. However, Psycho is not the standard distribution of Python and it doesn't allow the full flexibility of standard Python. This should be noted in the results. They should create a set of results using the 'standard' Python distro as well. ...otherwise, why not just let us use Inline::C in Perl (or in Ruby) - basically this would allow you to put C code into your Perl/Ruby script (which wouldn't be fair).
  • Where is Fortran, Pascal, and Basic?
    I know these are old but you can get them all under Linux.
    Pascal at http://www.gnu-pascal.de/gpc/h-index.html
    Fortran at http://world.std.com/~burley/g77.html This is g77 I do not know if G95 is ready yet.
    And I have seen a few Basics.

Slashdot Top Deals

"Pull the trigger and you're garbage." -- Lady Blue

Close