Forgot your password?
typodupeerror
Programming IT Technology

Another J2EE vs .NET Performance Comparison 533

Posted by michael
from the mine-is-bigger dept.
Starting yesterday, we received a bunch of story submissions about a performance comparison between J2EE and .Net. It didn't seem all that exciting, and we sort of ignored the story. But as usual, it appears that some people take issue with the methodology and conclusions.
This discussion has been archived. No new comments can be posted.

Another J2EE vs .NET Performance Comparison

Comments Filter:
  • by Faggot (614416) <choads@ga[ ]om ['y.c' in gap]> on Wednesday October 30, 2002 @01:43PM (#4565613) Homepage
    The beast most of us have sitting on our desk these days is so fast as to make language performance not such an issue. What should be focused on to support the future of computing is a well-typed, well-structured language to allow programmers to think at a higher level of abstraction than previously. That's why I love Mac's standardization of Objective C so much -- it allows high-level control of programs. Performance only matters if it sucks.
  • Save your time (Score:1, Interesting)

    by Captain Pedantic (531610) on Wednesday October 30, 2002 @01:44PM (#4565614) Homepage
    And read The Register's [theregister.co.uk] write up.

    Basically, nothing to see here.

    Oh, one interesting fact, "the .NET version required 14,004 lines of code, while the Java version featured 2,096."
  • Re:Save your time (Score:3, Interesting)

    by Yankovic (97540) on Wednesday October 30, 2002 @01:53PM (#4565712)
    you got that backwards. Java required 14k lines of code and .NET required 2k.
  • Infringement.... (Score:1, Interesting)

    by Tsali (594389) on Wednesday October 30, 2002 @01:54PM (#4565717)
    I didn't read the article (naturally), but doesn't the EULA for most Microsoft stuff explicitly forbid benchmarking .Net without prior consent from Microsoft?

    Can someone explain this so I can read the article later? :-)

    I'm just looking for Python/Qt benchmarking. :-)
  • J2EE, EJBs vs "JSPs" (Score:3, Interesting)

    by kisrael (134664) on Wednesday October 30, 2002 @01:55PM (#4565727) Homepage
    I've heard some word (admittedly not many datapoints) that some companies are still embracing Java/J2EE, but are going for "JSPs" (hopefully a euphamism for good use of regular java objects, maybe some wrapped JDBC) instead of the fullbore EJB. In my experience, this is a very smart thing. I've had successes with using a lot of the same patterns recommended for EJB with the lighter-weight stuff, and have heard of at least 3 really collosal EJB failures.

    EJB makes it easier to have physically seperate tiers, and adds enough systems-needed overhead that you'll probably need 'em...
  • by Art Popp (29075) on Wednesday October 30, 2002 @01:55PM (#4565728)
    ...are interesting when well researched, but basically useless to anyone who would actually have to choose between these two development environments. If you work for a company that designs applications of this kind there will be a host of more important things to consider than raw transactions per machine. The simple fact of e-commerce is that if a user is actually going to buy something at your site, you can waste tremendous processing power making them happy. If you make 2 dollars profit on a transaction and had to use 20% of the CPU on a 2Ghz processor for 40 1 second bursts (like you will if they shop using RH interchange), it's still worth it. What this benchmark argues well is that the MiddleWare product is probably worth buying if you have processor constraint problems. No amount of increased performance would warrant changing a staff of experienced Java programmers into a staff of inexperienced .net programmmers. Extra processors are just too cheap....
  • Lies, damned lies (Score:3, Interesting)

    by PhysicsScholar (617526) on Wednesday October 30, 2002 @01:55PM (#4565734) Homepage Journal
    OK, first off, I don't care how many lines of comments or exception-handling routines you take out, the Microsoft solution was still 7 times smaller. If a sub at two different stores costs the same $5.00, I would definitely buy the 7-inch one over the 1-inch version for the same price; essentially, it's better no matter how you cut it (no pun intended).

    Furthermore, if Yahoo moves from C++ to PHP for the majority of their Web applications, I think that's saying something. Perhaps J2EE and .NET are irrelevant at this stage in the game, and a PHP vs. ASP review would be more relevant.
  • Regus Reporting? (Score:3, Interesting)

    by Yankovic (97540) on Wednesday October 30, 2002 @01:58PM (#4565776)
    Anyone find it particularly hilarious that the Register couldn't even report [theregister.co.uk] the results correctly? Fine that they get anti-MS people to put in quotes, but the facts of the case (namely 14k lines of code for java v. 2k lines of code for .NET) were reported in reverse? Ugh... how these guys have a website is beyond me.
  • by gdeciantis (570658) on Wednesday October 30, 2002 @02:04PM (#4565837)
    One point about the refuting article is that it talks about the merging of data and business logic layers stretches the idea of object oriented. Although the code be less reusable, merging the two layers is in fact a very intuitive way to piece an application together and doesn't overload a project with excessive classes. As well the GUI will tend to dictate what functions can be performed. If the code is just gonna be used in that one page I don't think code should be anywhere than in that page.
  • by SirSlud (67381) on Wednesday October 30, 2002 @02:05PM (#4565848) Homepage
    Patently untrue. Microsoft is FUD-dy, not stupid. They wouldn't be touting performance reports if it didn't matter to purchasers and adopters.

    Whether or not it _actually_ matters is a whole other ball of wax, but I contend it still does. We're not a big business by any stretch of the imagination, but we need 20 servers to handle the load we do (400-600 'requests' per secondwith each request resulting in anywhere from 2 to 4 additional connections made for each request generated) .. if we ever moved to .NET or J2EE, you can bet your ass performance would be a big issue in making our selection.

    You might try rewording: In the *majority* of cases where people are selecting platforms, performance is not always the #1 issue.

    That might approach the truth, but to say performance only matters if it sucks assumes you can afford the hardware to meet your demand with room to spare on the first day you go live. In enterprise applcations (lots of customers) and high load applications (less customers but each customer generates tons of load - like an ad server), performance is _exactly_ where you're going to make or break your ability to supply the demand without buying the Noah's Ark of hardware.
  • by axxackall (579006) on Wednesday October 30, 2002 @02:06PM (#4565852) Homepage Journal
    Run-time performance is really a concern for system administrators, integrators and IT managers. The difference in run-time performance should be compensated by faster hardware, which gives a difference in cost of ownership. I expect such difference will be significantly less comparing to the difference in cost of software production, which is in essence a difference in performance of programmers, which in own turn time is very expensive.

    Therefore, it is much better to compare how both technologies help individual programmers as well as their teams to work faster and to produce a code with less errors (debugging time and QA resources). That would be a function of how API is structured, how concerns could be separated, how customizable code can be and will programmers tend to hardcode "business logic" riles.

    Does anyone know such comparison of J2EE and .NET?

  • by Anonymous Coward on Wednesday October 30, 2002 @02:10PM (#4565903)
    The problem is TheServerSide was incompetant in their testing and optimization.

    Any new EJB programmer knows that BMP EJBs perform poorly and require a lot more code than CMP EJBs.

    Any new EJB programmer would have used a current version of the PetStore as a base which uses CMP EJBs.

    Any new EJB programmer would have used JDK 1.4 instead of JDK 1.3 for the tests.

    Any rational programmer would not compare an old version of the Java EJB API with the next Beta version of .NET.

    This is just scratching the surface.
  • by nobodyman (90587) on Wednesday October 30, 2002 @02:18PM (#4565986) Homepage

    Did you intentionally reverse the figures? The .NET version was 2000 lines of code, and the Java version was 14,000 lines of code

    Why not go to the source [middleware-company.com] and draw your own conclusion. I looked at the report and it seemed more than fair. This was a straight up "best practices vs. best practices" competition, using Sun's recommended coding standards.

    It is helpful to note that this is the second such test that The Middleware Company performed. The Java folks squawked because the .Net version used stored procedures instead of dynamic SQL statements (Sun advises dynamic sql for easier portability and because they are idiots). This time, they addressed the Java camps' gripes and J2EE still lost!

    In my opinion you can pin the blame squarely on EJB's. They are bloated, the environments are a royal pain to configure, and they are S-L-O-W. Sun recommends that people use them, so it's totally fair that it was used against them in this comparison.

    Hate Microsoft if you want (I do), but you can't wear blinders and ignore the competition. J2EE blows. Get over it.
  • Error rates (Score:3, Interesting)

    by Animats (122034) on Wednesday October 30, 2002 @02:20PM (#4566014) Homepage
    What went wrong with "J2EE App Server B"? After two hours, it choked completely. But they don't say why. Even "J2EE App Server A" got 40 errors. The .NET servers supposedly completed the 24 hour load test without errors.

    Incidentally, all this stuff was run on Windows 2000. Somebody should try it on Linux.

  • I'm just waiting. (Score:3, Interesting)

    by glh (14273) on Wednesday October 30, 2002 @02:22PM (#4566030) Homepage Journal
    There will probably be another J2EE implementation that can "beat" the .NET benchmark. However, I think there is some degree of truth to this particular one. At our .NET User Group Meeting [nwnug.com] last night, we had someone from Microsoft actually talking about this benchmark. He didn't go into much detail on the J2EE side, but said that the MiddleWare company spent 10 weeks trying to get the J2EE implementation tweaked. So either these consultants are really incompetent on the Java platform, or there really is a significant performance difference.

    It's even more convincing in reading the article posted in the link (the "review" that is). Basically it was bringing up how the lines of code count was not correct because J2EE could have done a better job. Bah, that's a silly argument. LOC can't just be brushed off because it really does have something to do with the cost. More lines of code isn't just for "lazy programmers", it's also a factor when you have to think about MAINTAINING that code.

    However, I do buy the argument about not using the "latest and greatest" J2EE. So, I get back to my original point.. I'm just waiting for the next benchmark.

    So since the author complains about the PetStore app as being such a bad design, how about coming up with a new one and then comparing those? It seems to me like, no matter what, the author of the article doesn't believe .NET could *ever* be faster/better/etc. than J2EE. So really, it's a religious thing and I don't think any amount of proof will convince him. And I'm sure there are certainly others out there thinking that way. Of course the other camp also believes .NET is "all that".
  • Re:Paid for by MS (Score:1, Interesting)

    by Anonymous Coward on Wednesday October 30, 2002 @02:23PM (#4566047)
    so is this a denial of the results or a lame attempt to invalidate them because of the people that got them? how come i dont see any sun sponsored tests?
  • Jackal Feed (Score:2, Interesting)

    by waltal (128183) on Wednesday October 30, 2002 @02:31PM (#4566145)
    When a report is as complete and direct as this benchmark, it is interesting to see what the jackals go after. What body parts are worth flaming/roasting? In this case, ya'll missed two notable points:

    1. Two J2EE platforms by major vendors were tested. One really sucked for air (yeah, tell me performance doesn't matter!). Who were the vendors, all you Java experts reading between the lines? Would you buy that system?

    2. I understand that the .NET versions did not leverage stored procedures on purpose (apparently the first round of benchmarks got criticized for "non-portable" use of the DB!). I'd like to see that performance, because we are talking performance and not portability.

    Chew on this for awhile...
  • by rnd() (118781) on Wednesday October 30, 2002 @02:42PM (#4566276) Homepage
    Rigged demo? Please tell us what was rigged? Microsoft sent in an engineer, and a number of well-trained enginners were brought in (for a lot longer period of time than the 2 weeks spent by the Microsoft engineer) to optimize the A and B servers running Java.

    Of course Microsoft will be happy about the results, but think of it this way: Why would Microsoft have spent the money to build .NET if it was going to be slower than Java? Of course they know it's faster and wouldn't have ever done their original Pet Store comparison if they didn't have a margin of victory.

    So let me ask you this: Where is the comparison that shows Java being faster?

    Let me also ask you: Who would decide not to use Stored Procedures if they were available?

  • by toriver (11308) on Wednesday October 30, 2002 @02:54PM (#4566403)
    Any new EJB programmer would have used a current version of the PetStore as a base which uses CMP EJBs.

    Except that their "appserver B" - read IBM's feet-dragging WebSphere AS - doesn't support them yet. So they didn't use them for "appserver A" - read BEA WebLogic Server - either.

    Rather, they should have changed the Java PetStore implementation to closer match the .Net version. By replacing entity beans with far simpler "data access objects" they would have removed that bottleneck.

    Or used JBoss 3 + Tomcat 4 (+ Axis 1.0?) instead of WebSwear...

    Any new EJB programmer would have used JDK 1.4 instead of JDK 1.3 for the tests.

    Except that their "appserver B" - read IBM's feet-dragging WebSphere AS - probably doesn't run on it yet. And WLS 7.0 probably has "issues" with it.

  • Re:Some of us (Score:3, Interesting)

    by Laura_007 (621429) on Wednesday October 30, 2002 @03:12PM (#4566586) Homepage
    Absolutely ridiculous. Note the fact that Slashdot ignored submissions up until the point that someone rebutted it, and then suddenly it had validity. Everytime you take a single study as proving something, without doing your own research, you risk falling for advertising (not "false advertising", but advertising. Car A might be a piece of junk, while Car B is superb, but if Car A has 20HP more you can be damn sure they'll mention that in their ads).

    As Microsoft stated with the first one: Before all of the anti-Microsoftarians (of which Slashdot offer a tremendous number) slam this, GO AHEAD AND MAKE A SUPER OPTIMIZED PET SHOP APPLICATION ON THE PLATFORM OF YOUR CHOICE. The Java community, in this case, seems amazingly capable of criticizing this review, while failing to offer alternative proof.
  • by mosch (204) on Wednesday October 30, 2002 @03:16PM (#4566614) Homepage
    Could you try responding to points that I actually made? The fact of the matter is that servers, even db servers, aren't expensive compared to developer time. I can buy a mid-range SunFire 4810 for about the same financial cost as one or two man-years (a couple hundred grand). So if that 20 man development team can cut 2 weeks off of the project, but it'll require a SunFire 4810 to run afterwards, that's a good deal for the bottom line.

    I look forward to seeing what words you point into my mouth in your next response.

  • by elmegil (12001) on Wednesday October 30, 2002 @03:24PM (#4566700) Homepage Journal
    If you'd read the criticism, the Microsoft app claimed to be a compartmentalized 3-tier system. Apparently it was not. That much is at least under MicroSoft control.

    The other part that seems rigged, but less directly attributable to MicroSoft, is what sounds like really poorly coded J2EE. I'd really never consider paying TMC for any services, given their claim of expertise and the mistakes made in "fully optimizing" the J2EE version of the app.

  • Fix it then! (Score:2, Interesting)

    by jdevons (233314) on Wednesday October 30, 2002 @03:28PM (#4566753) Homepage
    If there is a problem with the Java version, can someone fix it?

    Everyone here is all bothered that MS was able to work on their version, but that Sun wasn't asked to do the same for its version.

    Sun can take both the code used in the .NET version and the code used in the J2EE version and compare them and THEN FIX IT.

    If they choose not to do so, or are UNABLE to do so, then we know which is better FOR THIS PARTICULAR TYPE OF WEB APP.

    Middleware did GREATLY improve the performace of the J2EE app, but there are obviously quite a few flaws in their design (from the comments). So someone who is pissed off about this should stop wasting their time ranting about it and correct the problem, then publish the code and the benchmarks.

  • by the eric conspiracy (20178) on Wednesday October 30, 2002 @03:43PM (#4566913)
    Oracle did a study showing the reverse with their app server and database.

    http://otn.oracle.com/tech/java/oc4j/pdf/9ias_ne t_ bench.pdf

    In Oracle's study, they achieved results that were as much as 22x faster than .Net

  • by Junks Jerzey (54586) on Wednesday October 30, 2002 @03:47PM (#4566959)
    It's not about loving Linux. It's about loving Freedom (TM).

    Realistically, though, we're talking about internet application development platforms here, not basic human rights. Being all high and mighty about not uing .NET is trifling at best, to say the least.
  • Re:Some of us (Score:2, Interesting)

    by Laura_007 (621429) on Wednesday October 30, 2002 @04:05PM (#4567125) Homepage
    It's an *example* app that demonstrats how to implement certain java features. They did not allow or ask anyone to make a new app they did it themselves then asked MS to make a competing one.

    Sun has promoted the PetStore application as an example of Java best practices, and how to implement a Java ecommerce site. Whether they are really trying to make it so burdensome that extra Sun servers are required is beyond the point: It's an example of how Sun recommends that an ecommerce site be implemented in Java. It seems entirely relevant to me.

    Even if there exists a "UPER OPTIMIZED PET SHOP APPLICATION" it's doubtfull they would have used it and optimising the current app would kill it's usefullness as a teaching aid.

    You have got to be kidding.... If Sun, who hold their own responsibility for putting their platform in the best light (the whole BS on here as if Microsoft should go to great lengths to polish Sun's code is just hilarious), could make a optimized PetStore application, you don't think they would have? They would have it slathered all over the net, and every anti-Microsoftarian would be screaming it from the highest mountaintops (indeed, I'd wager that many of the same busily tried to make just such an example after the last hoopla...strange that we never heard from them). Mind you of course they'd put haughty disclaimers about how they'd whored down the design to Microsoft's standard, but they'd readily do it.
  • by carlfish (7229) <cmiller@pastiche.org> on Wednesday October 30, 2002 @04:49PM (#4567703) Homepage Journal

    Here's the basic story.

    Once upon a time, Sun wrote a sample application, called PetStore, as a demonstration of various capabilities of the J2EE platform, and various techniques that might be helpful when writing J2EE applications. As such, it was deliberately over-engineered. A tiny shopping site doesn't need all the techniques they threw at it, it was just a context in which to deliver examples of coding pratices that might be useful in other situations. It was example code.

    Speed wasn't a goal. Keeping the LOC low was counter-productive to an application which is basically an example of different coding techniques.

    Microsoft saw this, and realised they had a cheap marketing opportunity. By rewriting the Pet Store in .NET, with completely different goals (speed and low LOC), they could score points just by issuing press releases. It's the marketing equivalent of saying "Hey! Our car is smaller and faster than your truck!" It's true, but meaningless.

    No matter that it was an apples to oranges comparison. No matter that the Pet Store could be rewritten in Java [ibatis.com] using Open Source frameworks with about the same number of LOC by one guy in his spare time. This is marketing, not reality.

    Charles Miller

  • Re:Final Conclusion (Score:3, Interesting)

    by truth_revealed (593493) on Wednesday October 30, 2002 @05:02PM (#4567857)
    If Sun's Pet Store application is not highlighting J2EE "best practice" then what the hell is it for, then? Are you saying that Sun with a billion dollars in cash in the bank could not find a suitable expert to write a decent J2EE example? They invented J2EE, afterall!
    First Entity Beans were good - and now they are bad. What other surprises can we expect? Who should an average Java programmer believe?
  • by j3110 (193209) <samterrellNO@SPAMgmail.com> on Wednesday October 30, 2002 @05:07PM (#4567916) Homepage
    Which brings another point to the table actually. J2EE is a spec, not a product. It makes no sence to say that J2EE is slow. There are no implementations of J2EE using Java's new asyncronous IO (NIO) libraries. Someone implemnted a webserver using NIO and it blew away Apache performance as well at raw serving. Of course it was so fast that it had to be run on a loopback interface so it doesn't matter at all. The big performance hit this takes is to the database. The DBMS in this Java version requires an SQL statement to return all the keys, and then one to retrieve each line from the DB. It doesn't have to be that way, it just is because they are dumb appearantly :) The SQL server has to parse the SQL for every single statement. That's not exactly CPU friendly either :)
  • by cmholm (69081) <cmholmNO@SPAMmauiholm.org> on Wednesday October 30, 2002 @05:13PM (#4567989) Homepage Journal
    Using Redhat 7.1 with jdk 1.3.1, the experience of our project team has been that the Linux jvm defaults to FORKING a new process for each java thread. If that's the behavior of the Linux jvm TMC was using, it could have skewed their results.

    What I'd really like to know is why TMC didn't test J2EE on the same Windows OS they tested .NET on. Were they getting paid to bash two opponents at once (Linux, J2EE)? If they really wanted to test J2EE, why not JUST test J2EE?

  • by mosch (204) on Wednesday October 30, 2002 @06:00PM (#4568522) Homepage
    Your original post states:
    So performance is very important in situations where the size of the application user base needs to scale dependably.
    It doesn't take a genius to understand that scalability and initial performance have nothing to do with each other. In fact, highly scalable solutions are often slower because they're designed with the understanding that 'perhaps there wil be a load balancer in front of this application someday, and I will not be able to handle session state in the simplest way'.

    Originally I said web servers aren't expensive because most performance problems can be solved by throwing hardware at them, which tends to be an appropriate solution for web-based applications, thus the current popularity of the 'rack of 1U webservers, and a fast db box' configuration, or some similar variant.

    P.S. If you want somebody to refrain from referring to you as a fucking retard, try to engage in a debate, instead of just flinging shit around like a retarded monkey.

Forty two.

Working...