Become a fan of Slashdot on Facebook

 


Forgot your password?
Close
typodupeerror
Programming IT Technology

Linux Number Crunching: Languages and Tools 322

Posted by chrisd from the hyperthreading-seems-neat dept.
ChaoticCoyote writes " You've covered some of my past forays into benchmarking, so I thought Slashdot might be interested in Linux Number Crunching: Benchmarking Compilers and Languages for ia32. I wrote the article while trying to decide between competing technologies. No one benchmark (or set of benchmarks) provides an absolute answer -- but information helps make reasonable decisions. Among the topics covered: C++, Java, Fortran 95, gcc, gcj, Intel compilers, SMP, double-precision math, and hyperthreading."
This discussion has been archived. No new comments can be posted.

Linux Number Crunching: Languages and Tools More

Linux Number Crunching: Languages and Tools

Comments Filter:

  • 0th post (Score:3, Funny)

    by Anonymous Coward on Thursday January 02, 2003 @05:10AM (#4997419)
    At least if you're developing in C....

  • Octave (Score:5, Interesting)

    by sql*kitten ( 1359 ) on Thursday January 02, 2003 @05:13AM (#4997425)
    Interesting numbers. Have you considered benchmarking Octave or rlab also? (Or is there a native MATLAB for Linus now?)

  • He didn't include K. (Score:5, Interesting)

    by Jayson ( 2343 ) <jnordwick@gmai[ ]om ['l.c' in gap]> on Thursday January 02, 2003 @05:23AM (#4997441)
    K [kx.com] is a high-performance array language. It is based on APL and Lisp. It really shines when crunching obscene amounts of data. This seems like something that would be perfect for the language. The proof of K's speed lies in KDB, a database written entirely in K. On TPC benchmarks [kx.com] is spanks Oracle and other leading databases (including some amazing scaling [kx.com] across processors: simple table scans with 2.5 billion rows take 1 second and multi-dimensional aggregations take 10-20 seconds).

    There is a quick and dirty intro to K [kuro5hin.org] over at Kuro5hin.

    Some more links for more inforation:
    Kernigan's benchmark test [kx.com]
    more examples [kx.com]
    Kx [kx.com]: the people who make K and KDB

    • The proof of K's speed lies in KDB, a database written entirely in K.

      It looks like an interesting product - I'll definitely take a closer look once it goes MT and 64-bit. Seems a little strange to me that it wasn't built like that from the ground up, since it seems to rely so heavily on clever data structures and virtual memory caching. (Altho' I do note that the slave processes share memory, which is the way Oracle does it if you don't want MT).

      Also, I'm unconvinced the inverse design will work well on sparse data. In every deal there are usually plenty of unused fields on the ticket, unless you fully normalize. It works well enough with rows, you just place all the nullable columns after the non-nullable, and Oracle will simply skip over them to the next-row marker in physical storage. Inverse tables will be fast for simple aggregates, not so sure how well they would perform for complicated multi-table joins and groups with many predicates.
    • From Kernigan's benchmark test


      k is faster (sum of times)
      k(32) perl(95) java(300) tcl(1400+)
      [k is much faster than c on strings and memory management.]

      k is shorter (lines of code)
      k(9) awk(66) perl(96) tcl(105) scheme(170) vb(200) java(350)";

      Interesting comparison. It took 350 lines of java to compare to 9 lines of k and 96 lines of Perl. also Java was 10 times slower than k and 3 times slower than Perl.

    • There are plenty of "high performance array languages". Matlab is one of them, and so is Numerical Python, and A+. I don't see any particular reason to push a commercial product like "K". Languages like that derive their speed from excellent underlying libraries; there is nothing amazing or special about that.

      On this particular benchmark, "K" would probably perform very poorly--because it doesn't involve any big arrays. But, since you like "K" so much, why don't you try for yourself and report back?

  • Mirror (Score:3, Informative)

    by elnerdoricardo ( 637672 ) on Thursday January 02, 2003 @05:26AM (#4997447)
    Here, I put up a temporary mirror in case this site melts...

    Coyote Gulch Mirror [elnerdoricardo.com]

    Be gentle! I'm sure my server is meltable, too! ;)

  • Good. (Score:4, Interesting)

    by neksys ( 87486 ) <grphillips AT gmail DOT com> on Thursday January 02, 2003 @05:29AM (#4997455)
    No one benchmark (or set of benchmarks) provides an absolute answer -- but information helps make reasonable decisions

    Ah ha! Someone who understands what benchmarks are for and how to use them - it sometimes seems like the corporate world uses numbers from benchmarks only when they prove their claims. Of course, that's the difference between open source and the business world - open source (ideally) looks at every benchmark result and asks "now how can we get all of these numbers better than the competition?" while more traditional businesses ask, "Which of these numbers make our product look the best?". *shrug* its just nice to see benchmarks used properly, is all.

  • Intel C++ (Score:2, Interesting)

    by jsse ( 254124 )
    I always find Intel C++ shines in all benchmarks. I wondered if anyone has ever tried to compile linux out of it? I know it might hurt your ideology but just for the fun of it. :)

    • Re:Intel C++ (Score:3, Informative)

      by Anonymous Coward
      Intel have released a separate version of their C++ compiler for Linux, which they claim has good GNU C/C++ compatibility http://www.intel.com/software/products/compilers/c lin/. They say it can be used to build the Linux kernel with few modifications.
      • Each time some people forget that Linux is not only for Intel platform - many users use Linux on Mac/IBM /Amiga PPC, on Sun Sparcs and on other hardware platforms.

        Intel C++ is only for x86, therefore it's for Linux/x86, not for Linux in general. Therefore, Intel C++ should not be used by developers who write the code for other linux users (for Linux in general). GCC must be used instead.

    • Re:Intel C++ (Score:3, Informative)

      by 0x0d0a ( 568518 )
      It doesn't work. Linux uses gcc extensions. Plus, the number of compiler bugs Linux exposes means that running it under icc would probably involve fixing a bunch of icc bugs.

      And you'd probably have to fix about a zillion Linux bugs...

  • SPEC-FP for different compilers? (Score:3, Informative)

    by billstewart ( 78916 ) on Thursday January 02, 2003 @05:44AM (#4997473) Journal
    The SPEC [spec.org] benchmarks are the descendants of the late-80s SPECmark benchmarking projects that did performance comparisons across a wide variety of machines and architectures, using code derived from real applications rather than purely synthetic little benchmarks like Dhrystone. Their benchmark suites were roughly 10 programs, with weightings on each program's results and scaling to compare with some popular architectures. They now have a variety of different benchmarks [spec.org], covering a range of types of applications, including floating point. The benchmarks have tended to be used by hardware manufacturers, so they'll usually have just one result for a given machine, with the options obviously tweaked for maximum performance, but the details are provided and sometimes there'll be tests using different compilers (e.g. because it's a compiler maker doing the test.)

    The benchmark programs aren't free - this is a non-profit industry association that charges money to cover its costs, but there are a number of universities that are members or associates which may be able to do testing that could explore some of the compiler differences; poke around their website to see who's reported what kinds of results.

    • So how much will the beowulf project have to pay for parrallel tasking to rank over serial tasking in the tests!?

      This is a joke, not libel. Beowulf is too poor to bribe.
  • Its nice that the writer went to all the trouble to work on this and share it.

    His conclusions are not very revealing. Anyone doing Java programming will quickly discover how slow it is esp. in regards to floating point. You don't need a benchmark for that.

    That C++ performed as well as Fortran on the author's examples is interesting to me. Is the C++ implementations getting better or is the Fortan complier gone soft?

    His reasons for staying with Fortran made me chuckle. Those are the same answers programmers gave back in 1983 when I asked them why they didn't convert after learning the new fangled languages C and Lisp. (Well new for me anyway at the time.) When it is right it is right. I guess Fortran is still alright.

  • Java is slow? (Score:5, Interesting)

    by nsample ( 261457 ) <nsample@NOSpam.stanford.edu> on Thursday January 02, 2003 @05:59AM (#4997497) Homepage


    I'm hardly a Java junky, but I've spent a lot of time recently with the language and I've heard a lot of complaints from my peers about Java being slow. Most of the time, just like this author, they're wrong! Java isn't slow, but sometimes you do have to program more thoughtfully to make Java fast.


    First things first, though. No one would ever claim that JDK 1.4 is the ultimate Java speed demon. Even the "HotSpot" in server mode is going to be slow if your code isn't written well. But the author fails to do any profiling, and fails to give anyone even a hint as to why Java doesn't perform well. But I shouldn't get on him about his coding, or lack of profiling... neither issue is the reason his test showed Java to be slow.


    The real problem: Firs, I'll cut him some slack for not profiling. However, I won't cut him any slack for using an interpreter instead of a JIT compiler. Java's been shown time and time again to be as fast as FORTRAN/C++ when using a good compiler, rather than an interpreter. *sigh* When will the madness end? A 0.07 second query to Google should explain that one to even a novice. Java IS fast. Interpreted byte-code is slow. Java != interpreted byte code; Java is a language.


    Anyway, here's a link to a weak, biased, and not so rigorous argument backing up that statement. But, it's an easy read for Java newbies, so I'll risk posting it anyway: Java is Fast for Dummies(tm) [javaworld.com]

    • Re:Java is slow? (Score:2, Informative)

      by fobef ( 541536 )
      What do you mean by "using an interpreter"? He used Sun's run time, even with the -server switch, which does some for java quite serious optimizations. The real disadvantage java has (in terms of performance, it is an advantage otherwise) isn't the IEEE requirements, but rather the extensive use of runtime binding of classes. In C code, the compiler can inline pretty much anything for you, and get long runs of instructions to schedule as it likes. The other big performance disadvantage compared to C++ is that in C++ you can often have complete control over how arrays are laid out in memory, and test different ways to see which ones work best with your cache hierarchy. In Java you just 'new' it and then you have no control over it. I often hear that Java is about half the speed of C++, but my own tests put it more in the 10% to 20% range. Maybe half is true for poorly optimized C++ code vs poorly optimized Java code, but for well optimized C++ vs well optimized java the difference seems closer to five times. Which is fine to me most of the time, so I generally use Java.

      • Re:Java is slow? (Score:5, Informative)

        by AG ( 3175 ) on Thursday January 02, 2003 @06:59AM (#4997586)
        gcj really is within 10% of g++ on this benchmark, unfortunately he built the gcj program without the all important -ffast-math option (and -funroll-loops). This is a huge penalty for gcj - more than 2x slower without.

        I sent him a note and hopefully he's update his page.

      • Re:Java is slow? (Score:4, Insightful)

        by X ( 1235 ) <x@xman.org> on Thursday January 02, 2003 @07:17AM (#4997622) Homepage Journal
        The -server option actually will imposive significant overhead for this benchmark. The -server option is not going to do any of it's significant optimisations without a TON of work.

        All your statements about C++ having an advantage over Java in terms of memory management are silly of course, since the Java runtime performs these exact kind of optimisations with Java programs. Because the decision is made at runtime rather than compile time it is actually possible for the Java runtime to make better optimisations than the C++ compiler/developer (who's decisions all have to be made a priori). I'm not saying this means Java always wins, because it most certainly does not, but I'm just saying that the "disadvantage" you are talking about is actually a misunderstanding of the conceptual differences between these two models.

      • Re:Java is slow? (Score:4, Interesting)

        by X ( 1235 ) <x@xman.org> on Thursday January 02, 2003 @07:23AM (#4997632) Homepage Journal
        For the record, I actually worked with the JPL evaluating Java's floating point performance. This was in the JDK 1.3 era, when HotSpot was still new. They had initially ported a highly optimised C library to Java and found the performance about in line with what this guy got (4-10x slower... actually it was an order of magnitude worse than this until they used the JIT ;-). The Java code showed many of the same performance errors that this guy's code has, as is common when you just do a line-by-line translation to Java, rather than rewriting the code from scratch. I did rewrite the code base, and managed to get the performance within the 10%-30% range. Using JDK 1.4 I'd have a few other tricks available to me which would probably get it even closer (maybe even faster).

        • Re:Java is slow? (Score:2, Interesting)

          by fobef ( 541536 )
          This is interesting, but I'm still very sceptical.

          First of all, the possibility to align data to suit the cache is very very real and not silly. Of course it might not make a difference if you're memory bandwidth bound or whatever else is limiting performance, but for critical working sets close to the cache size it can, and most likely will, make tremendous difference.

          The advantage that Fortran has over C++, except for lots of well tested libraries, is the fact that pointers cannot interfere with each other. For example, writing to memory through pointer PA cannot affect a read from pointer PB, so the compiler can rearrange the read and write in any order. This thing alone made fortran faster than C++ on most math benchmarks.

          There now is a restrict keyword in C++ that does exactly the same thing, but what does java have? Nothing like that ofcourse, and IMHO it shouldn't. Because java is a language that makes for bug free software with good-enough performance.

          I have only written two performance critical programs both in java and c++, so my numbers are anecdotal evidence at best, but c++'s victory was devastating both times. And they where no line-by-line conversions, no unneccesary news etc. Of course they where a far cry from the "loop one million times and make a double precision fdiv" of the article linked in the original post in this thread where they found out that java and c++ was about equal. But then, so was the code you wrote, which makes it interesting.

          Is this thing you ported freely available somewhere?

          Also, how do you know that the -server switch hindered performance? It was better than -client, right? I assume it performs its optimizations when loading the class, and the only disadvantage compared to -client would be when inlined, runtime-bound methods have to be un-inlined because of a new class being new'ed in the place of an old object with different class (did that make sense?:-) So is this assumption wrong, or do you know that this is in fact happening in this benchmark?

          • Re:Java is slow? (Score:3, Interesting)

            by X ( 1235 )
            Okay, first of all, the Java VM is fully capable of doing cache friendly memory management... in fact it can actually discover things about the cache's behavior at run time that are not likely to be known a priori by the compiler/programmer. So it's actually quite reasonable for a Java VM to excel in this regard.

            The instruction reordering issues you're talking about tends to only factor in when you have vector-processing CPU's. The limited vector-processing on the CPU's being tested is not enough to break the compiler's back, so it's not surprising Fortran is performing similarly to C++.

            Sorry, but my code isn't available elsewhere. However, I point out a few things this guy got wrong in another post.

            The difference between -client vs. -server is actually probably the opposite of what you are imagining. -client actually performs more optimisations at class load time, but fewer optimisations overall. This is in recognition of the fact that client programs tend to run for shorter periods of times.

            The -server switch actually REDUCES the number of optimizations performed when loading a class. Instead, it adds extensive profiling to all code execution, and then the runtime will gradually optimise frequently executed code based on analysis of how it executes. The optimisations are introduced gradually, almost on an "as needed" type basis. This approach allows the JVM to perform optimisations which simply aren't practical for a compiler to do, because the compiler doesn't have access to the profiling information gathered at runtime. When faced without a JIT, many programmers effectively do peep-hole optimisations of this nature by hand, but they ultimately can't predict what future runtime environments are like, nor can they effectively perform global optimisations on large programs. Of course, there are other differences between -server and -client, but this is by far the most important one.

    • JIT? (Score:4, Informative)

      by EnglishTim ( 9662 ) on Thursday January 02, 2003 @06:34AM (#4997543)
      The article claims that he *is* using a Just-In-Time compiler. What makes you think otherwise?
      • This statement from the author indicates he doesn't understand the JVM versus a JIT:

        "Perhaps Java's Just-in-Time compiler could be enhanced to perform processor-specific run-time optimizations; on the other hand, doing so would require different JVMs (Java Virtual Machines) for different architectures, or a single bloated JVM with run-time architecture detection."

        The Java compiler produces byte-code, and the JVM interprets the byte-codes. A JIT *is* platform specific, and tied to a specific OS and hardware architecture. That's why I think the author was not using a JIT.

    • Re:Java is slow? (Score:3, Interesting)

      by nsample ( 261457 )


      Btw, I am dating myself with the griping about JIT versus purely interpreted and all that, but there is something important here! I decided that my first post was decidedly unclear, and that I should actually profile the dang that and get some real numbers.


      The almabench program spends a lot of time in library routines that the author has no control over, and aren't always written the same way that they are in FORTRAN/C/C++. For instance, almabench makes 5,844,000 calls to java.lang.Math.asin(D), which then calls java.lang.StrictMath.asin(D) 5,844,000 times. The same is true of the 11,688,000 calls to atan2()... they're also passed along to StrictMath (only abs() is called as many times as atan2()). The beauty of writing java code is *not* knowing that these sorts of things are going on, no? For best *performance*, however, we have to work a little harder.


      I really enjoyed the following paper on using some OO programming-style optimizations coupled with a smart runtime to get almost identical FORTRAN and Java performance for linpack: linpack9 [rice.edu].


      A look there will validate the other comment about why OO designs can unecessarily kill performance and why the study's author should have used a JIT with identical libraries for math functions rather than Math.X(). Comparing otherwise is like comparing apples and oranges, or even Apples(r) and BillG.

      • For instance, almabench makes 5,844,000 calls to java.lang.Math.asin(D), which then calls java.lang.StrictMath.asin(D) 5,844,000 times. The same is true of the 11,688,000 calls to atan2()... they're also passed along to StrictMath (only abs() is called as many times as atan2()). The beauty of writing java code is *not* knowing that these sorts of things are going on, no? For best *performance*, however, we have to work a little harder

        Are you crazy?!. You think Java is supposed to let you get away with NOT knowing how your code works and interacts with the code it relies upon?

        Java, like any other language, needs to be scrutinized for performance problems. If you profiled and found that you code called java.lang.Math.asin, which in turn called java.lang.StrictMath.asin, then you should've re-written your code to use the second method directly. If it was in a third party piece of code, then you should've severely questioned the authors, and yoruself for using it. If it was in a benchmark,t hen you have to ask yorself if this benchmark is a good benchmark to use.

        I've encountered this myself several times. For instance, Ineeded to parse a big file based on lines and pipes (|). I figured it'd be easiest (i.e. fastest for me to get the code done) to just use java.util.StringTokenizer. When it turned out to be far too slow for what I needed, I profiled it. It turns out StringTokenizer is optimized to separate Strings into tokens when there are several possible tokens to separate on. In my case,once I rea da line, I just needed one delimitter. Hand-writing my own tokenizer for handling the special case of just one character provided far supoerior results and is in use today.

    • The article compares loading graphics in BMP format for C++ vs. GIF or JPG format for Java.

      It doesn't seem to acknowledge the widespread availability of file format libraries that are suitable for use with C++, such as libjpeg from the Independent Jpeg Group [ijg.org].

      It also repeats the commonly stated claim that Java is free from memory leaks. Nothing could be further from the truth. While it is possible to carefully write a Java program that doesn't leak, I don't think it's any easier than making a leakproof C++ program.

      How does Java leak memory? Simple. Just hold references to memory you don't need anymore. Holding a reference to any node in a DOM Tree will prevent the entire tree from being garbage collected. That's an easy way to leak tens of megabytes.

      Garbage collection is no substitute for responsible memory management.

      It also says that in C++, you have to implement graphics calls by coding to the native API of the OS. It says in Java, you have a portable way to do that.

      But that's just bullshit. The article makes no mention of cross-platform application frameworks, a number of which were around before Java was ever dreamed off. If you use ZooLib [sourceforge.net], for example, to code in C++, you can do everything you want with graphics with no need for platform-specific code. There's lots of other cross-platform frameworks, such as wxWindows and Mozilla, even GTK+.

      • There is no question that you can have a memory leak in a Java program. However, Java does remove a number of potential ways to create a memory leak that do occur in C++ (heck, 90% of C++ programs I see will leak heap memory if you have exceptions). Anyone who claims that "While it is possible to carefully write a Java program that doesn't leak, I don't think it's any easier than making a leakproof C++ program" either hasn't written a lot of C++, or hasn't written a lot of Java. ;-)
        • I prefer programming in C++ because I find it easier to just take responsibility for my memory.

          Using reference counting is one big help. And knowing how to write exception safe code is another. Yes, it's a difficult subject, but it is something you can learn.

          It may be easier to write leak-free Java than C++, but I suspect that because many Java programmers blithely assume their code can't leak, there may well be more leaky Java programs than C++ ones.

          I used to work at a web shop where we used the Enhydra Java application server. Enhydra is pretty well written, but the applications that the company originally developed for it were pretty sad. As a result they had to restart their servlet process every few hours because the JVM ran out of memory!

        • Sun's Wireless toolkit (Score:3, Interesting)

          by FyRE666 ( 263011 )
          Anyone who's spent any time working with Sun's Wireless Toolkit to develop for mobile devices will have witnessed pretty serious memory leakage firsthand. I know I do! After starting an emulator 10-20 times to test code it's using so much RAM that it's necessary to kill and restart the Toolkit to get anything like reasonable performance back again!

          I'll agree that Java makes memory management much simpler, (I've spent a lot of time hacking x86 assembler, Pascal, C and C++ over the years) but bad programming can lead to leaks just as well. You tend to discover leaks pretty quickly with a mobile phone that has only 200K of RAM to play with though ;-)

        • Leaks in C++ (Score:3, Interesting)

          by Antity ( 214405 )

          (heck, 90% of C++ programs I see will leak heap memory if you have exceptions)

          In case any fellow beginning C++ programmer was wondering: This is because objects are only guaranteed to be destructed when an exception is thrown if and only if this exception is caught.

          So no matter what you're doing in your program, its main() should always look a bit like this to make sure every exception is caught:

          #include <stdexcept>
          #include <cstdlib>

          int
          main (int argc, char** argv) {
          int ret = EXIT_FAILURE;
          try {
          ret = do_something_in_my_code (argc, argv);
          } catch (const std::exception& e) {
          std::cerr << "Caught exception '" << e.what()
          << "' in main().\n";
          } catch (...) {
          std::cerr << "Caught unknown exception in main(). Sorry.\n";
          }
          return ret;
          }

          (Sorry, I can't seem to save indentation with /. HTML here)

          Please note that this way, you also always assure to have a valid shell return code (EXIT_SUCCESS from your function() would be the OK code).

    • Java is really, really slow (Score:5, Insightful)

      by 0x0d0a ( 568518 ) on Thursday January 02, 2003 @07:45AM (#4997673) Journal
      ...I've heard a lot of complaints from my peers about Java being slow.

      Allow me to join the chorus.

      Java isn't slow, but sometimes you do have to program more thoughtfully to make Java fast.

      No. Java *can* be made less mind-bogglingly slow by avoiding certain things...preallocating a pool of objects and using primitive types (like int) whenever possible helps. The way the language is designed makes it *easy* to be mind-bogglingly slow. That doesn't mean that going out of your way to avoid these things makes Java fast. It makes it only "slow".

      Java is Fast for Dummies

      Ah, yes. A link tellings how Java isn't *really* that slow on "javaworld.com". I took a skim.

      The first two pages say basically "Java isn't that slow". They then start rambling about various features that make Java a good language.

      They claim that Java programs load faster than native programs. (The article was written, BTW, in Feb '98, to give an idea of how full of BS they are). This is stupid. JVM startup and load time *dwarfs* application link time. Write "hello world" in C++ and in Java.

      First, they laud the small executable size of Java as being a performance boost based on binary format. Everything I've read points the *other* way...Java *is* fairly compact, but can contain data that isn't nicely aligned along host boundaries.

      Second, what they're talking about, if it's even accurate these days, which I doubt, has a lot to do with the lousiness of the Windows runtime linker. This isn't really an issue for Linux.

      Third, while insinuating that minimizing code size provides a performance boost, they talk about how great it is that Java lets you use *built in* libraries, whereas C++ progams need to *bundle* libraries. What? That's stupid. They're shifting the libraries around, but it sure as hell isn't decreasing total amount of data that needs to get loaded.

      Fourth, this gem: Finally, Java contains special libraries that support images and sound files in compressed formats, such as Joint Photographic Expert Group (JPEG) and Graphics Interchange Format (GIF) for images, and Adaptive m -law encoded (AU) for audio. In contrast, the only natively supported formats in Windows NT are uncompressed: bitmap (BMP) for images and wave (WAV) for audio. Compression can reduce the size of images by an order of magnitude and audio files by a factor of three. Additional class libraries are available if you want to add support for other graphics and sound formats.

      They're billing this as *improving* performance? Yeah, I'd love to have my app blow CPU time decompressing a JPEG image instead of reading a slightly larger BMP image if I'm trying to minimize load time. Oh, and have it load all the JPEG loading code, too.

      They then proceed to ramble about selective loading, and try to imply that Java's runtime linking is faster than C++'s.

      They *then* show off smaller binary sizes by embedding a BMP in the C++ binary and a GIF in the Java binary. Impressive.

      They then claim that claims of poor Java performance are based on non-JIT implementations. This neatly lets them avoid actually citing numbers. Sure, I'll agree that Java went from "Performance Hideous" to "Performance Bad". Everyone uses JIT these days, and damned if Java isn't *still* slow.

      They then try to talk about how JIT allows code to be optimized just like C++. Wow. Yup, JIT sure is known for impressive optimization, isn't it?

      They then use the most artificial, contrived benchmarks I've ever seen (which conveniently avoid almost all of the Java pitfalls...they don't need to do array access, they're trivial to implement without heap allocation...)

      They finish up talking about how C++ RTTI performance sucks compared to Java (ignoring the fact that Java hits RTTI code *far* more often than C++ does, like every time it yanks something out of a generic container class).

      Finally, they finish up by talking about a bunch of random Java features that they think are great, like garbage collection "First, your programs are virtually immune to memory leaks." Hope you don't use hash tables, buddy.

      Next, they talk about how a JVM can defrag memory. I'm going to have to just crack up at that. This isn't a performance boost unless you're using a language that *hideously* fragments memory and eats memory like a *beast* (granted, Java is the best candidate I know of). Runtime memory defragmentation went out of fashion with the classic Mac OS...it's pretty much a bad idea as long as you have a hard drive available. VM systems are pretty damn good these days...if you're trying to maximize performance, there are almost always better things to be doing than blowing cycles and bandwidth defragging memory. There's a reason we don't do it any more.

      Basically, my conclusion is that "Java is Fast for Dummies" is primarily aimed at, well, dummies.

      • Re:Java is really, really slow (Score:4, Insightful)

        by 0x0d0a ( 568518 ) on Thursday January 02, 2003 @07:51AM (#4997680) Journal
        Oh, and a follow-up to my previous post. These clowns spent a long time talking about how people can "ignore JIT overhead" because it's almost completely insignificant "most of the time". Fine. Then they spend 80% of the article talking about *binary load time*, which is essentially only an issue under the *exact* same conditions that the JIT is -- once for a single chunk of code. If they're pimping launch time, they sure as hell shouldn't be ignoring JIT time.
      • Here's some the "better" parts of Java:
        • Rapid Development: because there is a huge core set of standard APIs, as compared to C++, any programmer can sit down with an expansive toolkit when writing a new app. This saves a lot of time by not having to reimplement those tools, or find 3rd party tools. Even if you have a set of 3rd party C++ libraries you like to work with, there is no guarantee that every 3rd party C++ library is going to work on every OS (I once worked in a shop compiling their code to 6 UNOIX platforms and NT -- what a pain!)
        • HotSpot: As someone else pointed out, HotSpot will improve things at runtime, where you have a lot more information available to you, instead of just compile-time. The author of the article used HotSpot Server which is made to optimize under very different circumstances (for example, extremely long runtimes).
        • Secure Software: By this, I mean avoiding things like buffer overruns. When you check your array bounds before accessing an array, you can be sure you won't overflow the array. Of course, it is slower. But even HotSpot optimizes a lot of this way (so I've heard, but I don't know exactly how). Sure, you *could* do this in any other language, but opbivously people don't. They get arrogant and assume that they'll never have bad data and just read to the end of a stream. Voila, you've just created a potential exploit!
        • Everything Else: One of the big factors remaining is just HOW you write code in Java. As someoneelse pointed out, there are a ton of Objects in Java, (almost) every function is virtual, everything is linked dynamically, etc. These things slow Java down, btu also make it more uniform which makes it easier (faster) to learn, in my opinion. If you made every function in your C++ classes virtual, used RTTI and Strings to do runtime linking, etc. your C++ programs would be slower too!

        • Re:Java is really, really slow (Score:5, Interesting)

          by 0x0d0a ( 568518 ) on Thursday January 02, 2003 @09:45AM (#4998001) Journal
          Here's some the "better" parts of Java:

          Don't get me wrong. Java's fine for certain applications. For lightweight networking stuff, I think it's almost unparalleled. It's also pretty good for prototyping C++ stuff. It's good for lightweight tasks that break down logically into threads -- Java has nice threading support.

          My beef is that Java is not, despite its supporters' loud claims (which have been going on for years), remotely performance-competitive with C.

          The language simply has some foundational performance limitations in it. It was designed that way, and tweaking implementations cannot get around that.

          I agree that there are some nice things about Java.

          Rapid Development

          Damn straight. Java is a great prototyping language.

          Hotspot

          Not bad, but not that incredible, either. The benchmarks I've seen haven't shown HotSpot to be incredible, and besides, competitors like C (gcc) have branch-profiling code of their own.

          Secure Software

          True. There are some improvements. But buffer overflows are less and less common in C (due to *excellent* libraries like glib), and have been fixed in other languages without anywhere near the performance hit of Java (like Ocaml).

          One of the big factors remaining is just HOW you write code in Java.

          It may be a personal thing, but I have a deep dislike of languages where you have to modify your regular coding style to get decent performance at a given point. It used to be BASICs...you'd use some nasty trick and you could actually get decent performance out of the thing. Then MATLAB. *God* I hate vectorizing operations. I expect that a MATLAB guru simply does this in his sleep, but I find it incredibly frusterating to totally rethink code in an any areas where performance matters.

          These things slow Java down, btu also make it more uniform which makes it easier (faster) to

          A fair number of the uniformity improvements in Java could have come from simply tweaking syntax (int[50] x instead of int x[50], for example).

          I'm all for modern language features...I just think that doing anything that implies a necessary performance hit is a bad idea. If someone wants a given feature, they can slap it on top. I can make C++ have a virtual function, but I can't make Java run quickly.

          If you made every function in your C++ classes virtual, used RTTI and Strings to do runtime linking, etc. your C++ programs would be slower too!

          Ya, but Stroustroup went to a lot of work to ensure that you only "pay for what you use".

          So, I'm not out to bash Java as a usable language. It has some major pluses. However, specifically in the performance arena, Java definitely has issues.
    • > using an interpreter instead of a JIT compiler

      No, he wasnt.

      > Java's been shown time and time again to be as fast as FORTRAN/C++ when using a good compiler, rather than an interpreter. *sigh*

      No it hasn't, because it isn't.

      You can sigh all you like, it doesnt change a thing.

      You acknowledge your link is weak and biased, why not try posting a reasonable link, particuarly one which seems to show that java is as fast as fortran or C++ for anything meaningful.

    • As the author of the article states languages as Java and C++ contain a lot of functionality which has no additional use to the problems to be solved. I can imagine however some functional languages do.

      Object oriented languages do come with a lot of overhead eg. memory management and exception handling, where a language as Fortran is optimized for and has proven its power when it comes to complex calculations.

      As we all know should select the right language for the right job. Last year I've been working on an application which performs complicated flow calculations and presents the results through a web frontend. The easiest and (IMHO best) solution was to use three languages: Java, C and Fortran.

      • Frontend Java (servlet), sending requests to a deamon (running on an another machine).
      • Deamon C, receives requests, calls a Fortran function (which it was linked to) and returns the result.
      • Calculations Fortran, for calculation precision and because the calculation libraries existed and were certified.
      Ah, I forgot to mention it uses SQL to gather data from Oracle.

      Seemed a reasonable solution to me. Offcourse I could have used Perl, assembler and Lisp too and Prolog to solve queries, or Word97 macro's combined with Occam (which is very good at parallel computing) or whatever.

      Sometimes discussions about Java start to look like discussions about religion. Bad. Scares the hell out of me.

  • The article deals with Pentium systems. Is there an AMD optimized compiler out there also?

    Can anyone suggest a good compiler for floting point number crunching on Athlon based systems? (For Linux, and preferably free or not too expensive)

    • try intel's compiler ... it's rumored to get better results than gcc on AMD's machines as well.

      you can do a 30-days free evaluation, IIRC, so my advice would be:
      devellop using gcc/g++, then when you need to go to production, evaluate intel's compiler and see if it's good enough for your needs.
  • He benchmarked gcc 3.2.1 and did not benchmark gcc 2.95.x . This in itself makes me doubt his work, especially if his C/C++ code is just C code "in disguise".

    Plus, parallelization should not have been a part of such a benchmark.
    Parallelization for a cluster _or_ a supercomputer has many issues
    that have nothing to do with compiler/language performance, and has much to do with IPC, memory and cache hit/misses, and other pains, who provide many people with PHDs and careers. For a good reason.

    I know he says it himself, but if you admit your results should not be taken seriously, don't publish them.

    Object-oriented programming adds very little to the core functionality of a number-crunching application -- and performance is adversely affected by the overhead entailed in objects, exception handling, and other object-oriented facilities.

    this means the man does not choose the right tool for the job. I agree that exception handling should not be used with numerical code, asserts work just fine. But objects and higher-level abstractions should very much be used in the 90% of the code which takes 10% of the runtime but still, say, >80% of development time ...

    And don't let me get started on templates, which really made my work much easier when used properly.

    And in many cases, function-oriented code exceeds the clarity of corresponding object-oriented code

    as for function-oriented code supposedly exceeding clarity of OO code, I find this remark, Ahem, embarrassing. If your OO code looks less clear than functional code, you have quite a problem. It means you do not use abstractions to ease design, reduce code-redundancy and reduce module interdependancies.

    It means, in short, that you are going through the motions, but do not think on what you do.

    • He benchmarked gcc 3.2.1 and did not benchmark gcc 2.95.x...

      Why should he have benchmarked gcc 2.95? What critical information would that provide? Other reports seem to indicate that compared to gcc 3.2, 2.95 is slightly slower, especially with processor-specific optimizations, and it has poor support for C99 and C++98. Nothing new or especially interresting here.


      Plus, parallelization should not have been a part of such a benchmark.
      Parallelization for a cluster _or_ a supercomputer has many issues...

      He didn't claim to test parallalization for supercomputers, he just tested to see if the automatic parallelization and openmp directives helped with the intel compiler on the p4 hyperthreading cpu. Automatic parallelization has a lot to do with the compilers ability to analyze the structure of the code and determine which parts can be run in parallel and which parts cannot. And as you can see from the results, in this case at least, there was little benefit from it.


      OO blah blah..

      I agree with you that OO is beneficial (for real life projects). OTOH, if you look at his benchmark code it's just a few pages. For a code which is that small there is probably little benefit from OO. And thus the c++ and java versions use no OO features, rightly so IMHO.

  • IBM java runtime and different platform. (Score:2, Informative)

    by Anonymous Coward
    Here are some results from an AMD system and the IBM java runtime:

    Athlon XP 1600
    -mcpu=athlon-xp
    -O3
    -mmmx
    -m3dnow

    IBM java - build 1.4.0, J2RE 1.4.0 IBM build cxia32140-20020917a
    58.8 seconds
    58.9

    Sun java - build 1.4.1_01-b01 Client VM
    94.2
    94.1

    Sun java - build 1.4.1_01-b01 Server VM
    82.1
    81.6

    gcj - version 3.2
    82
    82.1

    gcc - version 3.2 20020903 (Red Hat Linux 8.0 3.2-7)
    31.4
    31.3
    26.6 -ffast-math
    26.6 -ffast-math
    26.4 -ffast-math -funroll-loops
    26.4 -ffast-math -funroll-loops

  • gcj results incorrect - 2x worse than truth (Score:4, Insightful)

    by AG ( 3175 ) on Thursday January 02, 2003 @06:52AM (#4997572)
    Compiling his benchmark with -ffast-math and -funroll-loops more than doubles the performance of the gcj built benchmark on my P3.

    This brings it within spitting distance of g++.

  • He didn't include python (Score:5, Interesting)

    by more ( 452266 ) on Thursday January 02, 2003 @06:55AM (#4997577)
    Results on My P4 1.5 GHz, RedHat 8.0, gcc 3.2

    time python -O almabench.py user: 22m19.354s

    gcc -ffast-math -O3 almabench.cpp -lm time ./a.out user: 0m50.348s

    C++ is only 27 times faster than Python for planetary simulations.

    Almabench.py is my own conversion from the cpp source. I will send it to the author for possible addition to the benchmark.

    • Since more x86 benchmarks have been posted, people might have a look at some hints for optimization. I will only list some options important for Athlons ATM:

      • -march=athlon-xp Yes, gcc-3.2 not only includes "athlon" but also "athlon-xp" and even more finer-graded pentium architectures. Use them! (implies -mcpu)
      • -O9 Always use it. I've yet to see miscompiled code with gcc-3.2 and -O9.
      • -fno-rtti / -fno-exceptions Only if you know the program isn't using them.
      • -ffast-math "Simplifies" some assumptions the compiler makes over your FPU code. Could lead to slightly inaccurate results (down the commata..), but usually works great, even with xaos and co and gives a huge speed boost
      • -fno-math-errno If you don't use errno after math operations at all
      • -funroll-loops / -funroll-all-loops Well.. unrolls some (1st) or all (2nd) of your loops. The first one is more sane and already included by many -O levels
      • -fprefetch-loop-arrays Important! On newer chips (Athlon/Pentium) this will generate instructions to prefetch memory before / while in a loop. This really works.
      • -malign-double Aligns doubles on 8 byte boundaries (instead of 4 byte boundaries). It's a must on newer x86s but it breaks compatibility with many precompiled libraries. Just try it. If it breaks, you have to recompile all libraries used with this option (which is a pain in the ass with Glib/GDK/GTK+ etc)
      • -mfpmath=sse Lets the compiler use SSE fp math instead of i387 math. pros and cons... it's worth a try for your application.
      • Last, but not least: -fssa This is still experimental! in gcc-3.2, but after applying all other options for almabench it gave me another full 1% of speed increase. Worth a look, but expect broken code to be generated.

      More in your favorite GCC docs.

  • Fortran compilers and Linux (Score:4, Interesting)

    by EmagGeek ( 574360 ) on Thursday January 02, 2003 @07:16AM (#4997620) Journal
    Here [polyhedron.com] is a more in-depth comparison of Fortran 90 compilers for linux. They [polyhedron.com] compared Intel, NAG, Lahey, and a couple of other compilers. Here [polyhedron.com] is a comparison of Fortran 77 compilers from the same folks. GNU g77 is actually the slowest of them all, and I've actually confirmed that it is the slowest of a group consisting of DEC/Win32, Lahey/Linux, and g77. I've always dreamed of the day that open source developers would throw some real brainweight at a really well optimized Fortran compiler for linux, but it looks like I'll just have to keep dreaming. Lahey is only $199 or so, but they place some HORRIBLE licensing restrictions on the binaries that are created with their compiler. The DEC/Win32 compiler is also really nice, but since I'm not in school [gatech.edu] anymore, I'm not licensed to use it, and even if I _wanted_ to whore myself out to Micro$oft, I couldn't afford to.

    Just to put some things into perspective, here are some numerical results. These were obtained on my dual-athlon 1.4GHz w/ 1GB of RAM. The task was to compute the TE and TM surface currents induced on a circular cylinder 10 wavelengths in circumferece and having a relative permittivity equal to 4-j2. The program simultaneously solves the perfect electric conducting case. The surface was discretized into 60 cells using 120 unknowns (way overkill, but just to prove the point) using the Integral Equation Asymptotic Phase [ie-ap.org] method.

    g77 Compiler (-O2 -malign-double -funroll-loops): 24.11s
    Lahey Compiler (equivalent paramters): 16.45s

    As you can see, there's really no comparison, except that the lahey-created binary uses about 10% more RAM than does the one created with g77. This is just a summary comparison as I did not go into measuring the difference in the error of the two results compared to a reference solution. I'm assuming that both solutions are about the same with regard to accuracy.

  • IBM JDK 1.3.1 (Score:3, Informative)

    by MarkoNo5 ( 139955 ) <MarkovanDooren@nOspAM.gmail.com> on Thursday January 02, 2003 @07:33AM (#4997650)
    When I run my QR decomposition "benchmark", IBM's virtual machine always comes out about 20% faster than Sun's. It would still be a lot slower than C++ or Fortran, but the gap should be smaller. On top of that, IBM's license does not require you to accept a license which says the VM may install any software on your machine and that you automatically accept the license of that new software. See http://hal.trinhall.cam.ac.uk/~nrs27/java_eula.htm l [cam.ac.uk] for all the fun.
  • I thought I would try out a few results on my machine, and got some interesting results. I would encourage other people to do the same. I have used his java code, unmodified, save for a System.out.println the time taken.

    My computer is a 900MHz Athlon, 256Mb RAM, running WinXP. Using J2SDK 1.4.1_01.

    javac -g:none -O *.java

    java -server almabench
    Took a total of 107555ms

    jview almabench
    Took a total of 51073ms

    Compare this with 76 seconds on his 2.8 GHz machine. JView (that's Microsoft's VM) was able to run in 51 seconds.

    The two things of note:

    • My results are significantly faster than his.
    • Jview ran at twice the pace of Sun's latest offering. Euch.

    Unfortunatly I have no c compiler on this machine to run them, that would be really interesting.

  • In the article the author writes:

    Object-oriented programming adds very little to the core functionality of a number-crunching application -- and performance is adversely affected by the overhead entailed in objects, exception handling, and other object-oriented facilities.


    Using object orientation in C++ adds a very small overhead (in the vicinity of 10% if you're using virtual functions). Now I understand that the article was mostly concerned with benchmarking the languages, and I applaud the writer for specifiying that benchmarking is no "silver bullet". But I really want to stress:

    Somebody has to develop and maintain the freaking program too!

    Object orientation helps anything but the most trivial of applications to acvhieve better modularity and reuseability. Anyway your program is going to spend most of its time in development, not running, so anything that can help that process along is going to be a big help to your project. Please check out the benefits of the SW industries "best practices" [google.com] and apply them to your project.

    You will save days and months of development time, during which you can run your finished program to your heart's content.

    • Re:Development time vs. run time (Score:4, Insightful)

      by ChaoticCoyote ( 195677 ) on Thursday January 02, 2003 @10:36AM (#4998277) Homepage

      Using Object-Oriented constructs is no guarantee that a program is maintainable or even readable. I have seen some horrifying OOP code in my life, written by people so enamoured of syntax that they drown theircode in it.

      In numerical applications, and extra 10% can be the difference between success and failure. I'm corresponding with a fellow who works in meteorology; his company uses commodity boxes to compete with government-funded monopolies. For him, the ability to gain 10% is crucial.

      I am all in favor of object-oriented programming -- but my philosophy matches that of Bjarne Stroustrup, who refers to his language as a having "multiple paradigms." Use OO when it makes sense -- but use the right tool for the task at hand. C++ does not force you to use OOP when it doesn't make sense.

      Many numerical applications make mroe sense when using short variable names (that match formulas in texts) and a function-based approach (again, matching mathematical idiom).

    • Anyway your program is going to spend most of its time in development, not running, so anything that can help that process along is going to be a big help to your project.


      This is true, but in production use, code performance is critical. If you can cut execution time by a factor of two or more for any given hardware configuration, it can mean the difference between feasible and infeasible. For instance collaborative filtering on the web.

      A lot of optimizations in numerical computing languages involve removing OO features (e.g., matlab). Numerical computing seems to focus on vectorization (re-representing all problems in terms of vectors), a sort of one-size-fits-all data representation, that can than be used in highly optimized algorithms. At the end of the day, one has to wonder about the usefulness of the OO paradigm for heavy duty numerical computing.

  • Using both Sun's and IBM's Java virtual machines I get the following results from 'time'.

    real 0m35.173s
    user 0m35.140s
    sys 0m0.030s

    The results vary from 35.130 to 35.160. When I run the c++ test with the following compiler options : -march=i686 -mcpu=i686 -O (somehow the mmx and sse option are rejected)

    real 0m43.467s
    user 0m41.790s
    sys 0m0.010s

    Can somebody explain this please ?

  • He complains about the Java compilers complete lack of optimisation flags (duh... making optimisations to platform independant byte code is pretty useless, and generally counter productive... it's all done by the JRE, which actually has a number of flags for tweaking things, and there are many JRE's to choose from). He talks about how the JIT doesn't perform processor specific optimisations (it does in fact perform many processor specific optimisations, unfortunately his benchmark is written in such a way that none of them will get used). He talks about there being no interest in high-performance Java (the Java Grande [javagrande.org] group would beg to differ). Best of all, he keeps calling Java an interpretted language... even though he used gcj as part of his own benchmarks... Sigh.

    Worst of all he uses gcj without trying out TowerJ, which is a much more established Java-binary compiler. Sigh.

    • I can only try... (Score:4, Informative)

      by ChaoticCoyote ( 195677 ) on Thursday January 02, 2003 @10:28AM (#4998236) Homepage
      ...the tools I have at hand. I have nothing against TowerJ, but can't test it if I don't own it. As for Java, I made note of a lack of flags, which is different from complaining. The -O flag is no longer supported by Sun's JDK according to the documentation.

      • Re:I can only try... (Score:3, Informative)

        by X ( 1235 )
        Actually, you can test it even if you don't own it. They provide a free trial download right off their home page [towerj.com].

        The -O flag wasn't doing anything that improved performance on a JIT'd system, which is why it was removed. The problem is you're looking for optimisation flags in the wrong space. As long as the compiler is outputing byte codes any performance optimisations it might do are actually quite likely to make things worse. Admittedly, the SunVM doesn't have much in the way of floating point optimisation flags, but that is where performance optimisations kick in for JIT'd runtimes.

        • An old policy (Score:4, Informative)

          by ChaoticCoyote ( 195677 ) on Thursday January 02, 2003 @11:29AM (#4998556) Homepage

          Back when I wrote reviews for print magazines (back when there were print magazines, that is), it was standard policy to limit reviews to the actually, shipping, commercial product. Demos often lack critical features (like optimizers) or are tuned for benchmark tests, so I've kept to that policy now that I write reviews for my own web site.

          I own licenses for the Intel compilers, for example -- and, of course, gcc and Sun Java don't cost anything in the first place. I'm considering my options in this case; long gone are the days when a dozen Fortran or C compilers would arrive at my door for a magazine review. Heck, there aren't a dozen compiler vendors left... ;)

          • In TowerJ's case, the only difference from the production product is the limited time period for use (15 days). However, I checked it out myself and it appears that if you click through they've actually recalled their downloads program. Now you have to register to get to the code (and they may not even give it to you then).

            I can tell you from past experience though that it pretty much killed gcj in performance for this kind of stuff.

  • Comments by the Author (Score:5, Informative)

    by ChaoticCoyote ( 195677 ) on Thursday January 02, 2003 @09:54AM (#4998043) Homepage

    Almost *ALL* of my email is related to Java. I'll be adding the IBM JDK and older versions of the Sun JDK later today, as per reader request.

    I'm making minor updates to the article as the day passes. I appreciate comments from everyone; once I'm through with my e-mail, I'll respond to these Slashdot comments.

    Additional benchmarks will be added to the article with time; I'm putting together a single-precision ("float") benchamrk, for example.

  • awk (Score:4, Funny)

    by Charles Dodgeson ( 248492 ) <jeffrey@goldmark.org> on Thursday January 02, 2003 @11:16AM (#4998463) Homepage Journal
    Surely people remember that most excellent O'Reilly book, Numerical Recipes in AWK. Unfortunately the 1998 review of it has disappeared [segfault.org].
  • The answer is:

    You haven't got an fscking clue Sparky.

    I haven't given you any indication as to what sort of *conditions* performance will be measured under.

    The author of this benchmark test seems to understand this. Did anybody wailing on him for failures of the tests bother to read his benchmark caveat?

    All a benchmark test does is give you some understanding of how various things perform under the *precise* conditions applied during the test.

    The real value of benchmarking only comes after performing hundreds of tests under hundreds of variants of enviromental conditions.

    Doing so will increase your *human* wisdom about how certain things perform. Performance is a *value* judgement, and any human who abrogates their value judgment to a machine probably gets what they deserve.

    Too bad that sort of shit usually rolls downhill though.

    KFG

  • The guy is ignorant about Java (Score:3, Interesting)

    by mike_malek ( 82254 ) on Thursday January 02, 2003 @12:57PM (#4999070)
    First off, I have interned three times at Sun, working on virtual machines. So I know a fair bit about VM's, and their run-time and processor-specific optimizations.

    "it has always been clear that Java is inferior to native code applications in terms of raw power. Computational performance depends on the effective use of a target processor's instruction set; such optimization can not be expected of Java given its goals of universality and portability."

    This statement is simply not true! The portability nature of Java does not conflict with a virtual machine that does processor-specific optimizations. Take a look at Sun's HotSpot VM source code (it's publicly available!) In the IA32-specific code, you'll see lots of run-time switches to enable specific P4 optimizations, for example.

    "Perhaps Java's Just-in-Time compiler could be enhanced to perform processor-specific run-time optimizations; on the other hand, doing so would require different JVMs (Java Virtual Machines) for different architectures, or a single bloated JVM with run-time architecture detection."

    This already exists in the current HotSpot VM. There's an IA32 binary, which includes optimizations for several versions of IA32. It does not include PowerPC or SPARC code, as that's in a different binary.

    " The "ia32" world is already fragmented between Pentium III, Pentium IV, Itanium, Athlon, and Opteron architectures, each having unique requirements for optimization;"

    That's the challenges a VM writer has to deal with. And the HotSpot team did a great job in managing this complexity.

    In the future, if you (or anyone else, for the matter) takes the time to write a paper, you should do more research. Some of the statements above are simply misleading.

  • A Few Other Fast Ones (Score:3, Informative)

    by Lucas Membrane ( 524640 ) on Thursday January 02, 2003 @06:21PM (#5002018)
    OCaml -- if you can do tail-recursive list processing instead of array processing, this is just about neck and neck with anything else. Arrays are slower.

    Corman Lisp and MLton -- Other fast functional languages.

    GNAT Ada -- Runs pretty close to C/C++ speed. Sometimes faster if the compiler can do expeditious compile-time optimizations.

  • Article Updated; My Comments on Your Comments! (Score:3, Informative)

    by ChaoticCoyote ( 195677 ) on Thursday January 02, 2003 @09:56PM (#5003491) Homepage

    The original article generated an exceptional collection of interesting and helpful of responses. In this section, I'll run down the important points people made.

    Note: This is a duplicate of a new section I added to the review; I'm posting it here for posterity.

    An overwhelming number of people suggested that I include results for IBM JDKs -- and I have. In fact, I've added results for Sun JDK 1.3.1_06, IBM JDK 1.4.0, and IBM 1.3.1 RC3. Adding these JVMs made a significant difference in the results, cutting runtimes in half. On the other hand, almabench runs as an infinite loop with Sun's 1.3.1 JVM (it starts, but never finishes). Note that I recompiled almabench with the corresponding javac for each JDK, so the JVMs were executing code generated by their corresponding compiler.

    The problem with Java's performance is not my code or my lack of Java skills -- what real problem is that Java 1.4 is slow. Both the Sun and IBM JVMs lost significant performance in the move from 1.3.1 to 1.4, whether due to a new language requirement or other factors. My faith in Java is severely shaken when applications lose significant performance by upgrading to the current release of Java.

    Java 1.4 added many new features to the language and packages; however, changing from version 1.3 to 1.4 should not double run-times! Nor am I comforted by the problems of Sun's 1.3.1 server JVM.

    Given the nature of the problem, my conclusions about Java stand (albeit slightly softened). By Sun's own definition, JDK 1.3.1 is obsolete; the fact that it performs better than the most current JDK is indicative of a serious problem with Sun's improvements to their language. Since Java 1.4.1 is what Sun is promoting, so that is what I base my conclusion on. I can say that IBM's product is superior, and have already set it is my default JDK. It's no wonder Sun is upset about IBM usurping Java -- IBM is producing better tools.

    Some people asked if my Java results were biased by the amount of time it takes to load the JVM. I've tested several empty and near-empty applications; a Hello, world program, for example loads and runs in less than 2/10ths of a second -- hardly significant. The start-up time increases with the number of imports -- but almabench.java imports only one small class package, java.math.*, which (in my tests) does not impose measurable overhead.

    I did not include any commercial Java compilers. Most, like the oft-cited Excelsior JET [excelsior-usa.com], are Windows-only; this article is about Linux. I don't benchmark Visual C++ or C# for the same reason (although I will look at Mono [go-mono.com] and C# some time in the future). The free version of Borland C++ does not include a complete optimizer, so I don't think it fits in this review.

    How do I know that the programs are producing the correct output? Each program includes code to display results; I run the programs with I/O to ensure that all calculations are being performed, then I comment out any header inclusions, imports, and print statements for actual testing.

    How am I timing the results? With the Linux time command. Table 3 reports the real value reported by time (the elapsed time of execution.) Embedding timers in the actual code is fraught with problems; for example, each language implements different time scales and abilities. I'm sure someone will tell me that time is full of problems too, but it works for me and is consistent across all programming languages.

    Amid the barrage of Java-related comments, a few people actually noticed the Fortran code. I am looking at other Fortran compilers for future updates. As for GNU g77 -- I wrote the code in Fortran 95 because I find Fortran 77 to be annoying. I wrote piles of Fortran 77 back in my CDC and VAX days, but these days I'm writing for environments where Fortran 95 is more appropriate. Believe it or not, Fortran 95 is a very clean, orderly language that eliminates many Fortran 77 idiosyncrasies while adding features important for high-performance coding.

Slashdot Top Deals

You've been Berkeley'ed!

Close