Catch up on stories from the past week (and beyond) at the Slashdot story archive

 



Forgot your password?
typodupeerror
×
Java Programming

Who is Using Tomcat or Jetty in Production? 488

JettyCatReady queries: "Ok, my company (a rather large, global financial institution) has recently blessed Linux for production use (woohoo!). Their position is that it will save them hardware costs to run on Intel machines instead of big IBM or Sun iron. No mention at all has been made of their position on open source. I'm part of a team that wants to make the case that the real savings are to be made by making use of things like Tomcat in place of BEA where we can (if all we want is JSP why pay a huge cost per server?). I even have a boss's boss who said in front of me, 'So I'm thinking, am I missing something by not using Tomcat? Do I have anything to lose?'"

"These are all excellent signs. The next step is to get an open source server into production. Tomcat is the natural choice because it's got the name recognition among Java app servers. Here's where I'm a little stumped. Whenever I mention the words 'Tomcat' and 'production' together, performance junkies come out of the woodwork and tell me that Tomcat sucks for production (what with it being a reference implementation and not optimized for speed). They say use Jetty (except for the ones that say to use Resin). The counter argument is that if my managers have heard of Tomcat, and seen vendors that will support Tomcat, and have never heard of Jetty, then there's no way they're going to bless it over Tomcat. (The same boss who praised Tomcat above also made a face when I mentioned JBoss. And I'm sure it has nothing to do with his personal experience with either.)

My question is, does anybody have some real world numbers of large institutions actually using these servers in a production environment? If somebody can tell me 'Company X uses Tomcat exclusively' then we would have no problem contacting company X and saying, 'So, what have your experiences been?' In other words I need leads, not actual white papers (although those would be nice, too). I need some real experiences, not just people who like Jetty over Tomcat because they don't like Sun."

This discussion has been archived. No new comments can be posted.

Who is Using Tomcat or Jetty in Production?

Comments Filter:
  • Production Tomcat (Score:4, Informative)

    by Anonymous Coward on Tuesday August 20, 2002 @08:24AM (#4103413)
    Can't give our company name but we're using it in production for an ASP-type senario serving apps to large financial institutions off of WinNT boxes. Compared to the previous IIS builds (ugh) it's wonderful, stable and a nice advert for taking the whole show over to UNIX.
    • Re:Production Tomcat (Score:3, Interesting)

      by rwinston ( 602519 )
      I work for a major Telco, and recently we have made some sweeping and far-reaching decisions: 1) Our internal Web servers (including some servers that exist solely to serve Perl CGI scripts) are being migrated from NT/2000/IIS/ActiveState to Linux/Apache/mod_perl/Perl CGI. 2) Instead of using Oracle/SQL Server by default, we are beginning to use MySQL by default, and only use the big iron when its necessary. 2) We are using Tomcat for at least one production site, and perhaps more in the future. A lot of internal apps are being ported to Tomcat 4/mod_jk/Apache 2 This is not just a cost-saving exercise - this is going to solve some long-term reliability and security issues
  • by Tet ( 2721 ) <slashdot AT astradyne DOT co DOT uk> on Tuesday August 20, 2002 @08:25AM (#4103422) Homepage Journal
    Like the subject says. It seems to work OK for us. Startup times are annoyingly slow. If we need to deploy a new context, then restarting tomcat brings with it a 30-45 second outage. But other than that, it's fine. Performance testing showed that increasing the number of threads the connectors can handle, and increasing the memory size (we use -Xmx500M) helps enormously.
    • by flipperboy ( 153511 ) on Tuesday August 20, 2002 @08:44AM (#4103543)
      My experience is almost exactly the same. We started using tomcat during development because it was free, and found that it performed well enough that we were confident moving into production with it. Restart times are not an issue for us; we can schedule resource drops for times when system use is minimized.

      During load testing, we mucked about with the same tomcat parameters mentioned above, specifically thread count (starting and max) and heap size.

      One last note: with versions 4.x of Tomcat, we've found that Tomcat is quick enough at serving up static content that we don't need to put Apache into the mix.
    • by Hard_Code ( 49548 ) on Tuesday August 20, 2002 @08:54AM (#4103604)
      "If we need to deploy a new context, then restarting tomcat brings with it a 30-45 second outage."

      Remember, in 4.x, a command-line admin tool to insert/reload contexts at runtime has been added. A GUI is planned to follow.
    • We just went back to Tomcat. Basically my company is putting a "web application" thing out and when they first started they where using Tomcat. Went with HPAS for a while but now that that is dead we are going back to Tomcat. The main reason is because there is nobody to kill it. :) In any case I like it and hope we will be able to contribute back.
    • The prospect of restarting the whole server just to update some JSPs or to redeploy a web app is, frankly, a complete non-starter for most large sites.

      A lot of WebLogic shops update their content regularly, often using separate content management systems like Vignette (I know...), so if the original enquirer has requirements like this then Tomcat can be ruled out right now.

      To wax on WebLogic's virtues a bit (hey, gotta restore some balance!) it allows you to redeploy a whole WAR (as a single file or as individual ones in 'exploded' format), to update JSPs automatically just by writing them to the source directory, and to update other servlets using the refresh tool.

      [Disclaimer: poster is an independent BEA consultant type]
  • Novell (Score:5, Informative)

    by Scutter ( 18425 ) on Tuesday August 20, 2002 @08:26AM (#4103427) Journal
    Novell's Groupwise version 6 runs on Tomcat with Apache. It's actually set up to run on Netware, of course, but I've gotten it running quite nicely on linux as well.
  • JBoss (Score:4, Informative)

    by Anonymous Coward on Tuesday August 20, 2002 @08:28AM (#4103442)
    Take a look at JBoss, we replaced BEA with it for commercial product deploys and have been thrilled. It can also be integrated with Tomcat or Jetty.
    • Just wondering, version 3.0 or 2.x?
    • I must agree 100% with you. I work as a consultant and I have helped 3 companies to switch from WebLogic to JBoss.

      Everyone here should check the 3.x version, is one of the best software tools ever developed.
  • I know that you are probably looking for an open-source solution, but Sun has promised to release a free version of their application server this fall.

    Sun "basic" application server [sun.com]

    It will run on Linux.
  • by MSBob ( 307239 ) on Tuesday August 20, 2002 @08:29AM (#4103447)
    I'm no fan of tomcat myself but if you're doing servlets then it is probably your best option (and cheapest). Being in a situation similar to yours I've evaluated JRun, WebSphere and Tomcat and liked Tomcat the most. It was most up to date with the J2EE spec and wasn't trying to be everything I didn't want it to be. It simply got its job done. Having said that, Tomcat on the back end means Apache on the web tier and I'm no big fan of Apache (or its configuration nightmare specifically).

    Tomcat 4.x series is designed and built for production use unlike the 3.x series which was a reference implementation donated by Sun.

    Anyway if you're not doing EJB tomcat is a reasonable choice. If you ARE doing EJB work you can pick up JBoss which integrates well with Tomcat. Pick up GLUE for web services and a decent persistence layer (OJB for example) and you're all set for enterprise level development with $0 spent on infrastructure software.

    • by Anonymous Coward on Tuesday August 20, 2002 @08:39AM (#4103517)
      Having said that, Tomcat on the back end means Apache on the web tier

      Why? Tomcat can be used stand-alone and it can be integrated with other webservers, even IIS!
      • because using Tomcat to serve static html and images is like using a an suv to deliver 2x4 and drywall to your construction site. Sure, they fit in the vehicle, but its not exactly optimal, not by a long shot.
        • TUX! (Score:4, Informative)

          by ttfkam ( 37064 ) on Tuesday August 20, 2002 @12:26PM (#4105483) Homepage Journal
          If speed your concern for static content, put TUX in front of tomcat. No config changes are necessary for tomcat and TUX can saturate gigabit ethernet adapters easily and with comparatively little CPU overhead (more CPU free for tomcat to handle the dynamic stuff).

          You can read more about TUX here [redhat.com].
    • I'm no big fan of Apache (or its configuration nightmare specifically).
      Sigh, its amazingly simple, just because you don't have a pretty GUI, does not make it a nightmare to configure, more to the point, there are many programs that offer GUI configuration of apache, have a look around before dismissing apache because you can't set it up right
      • Maybe he's a bit rude, but I've never thought that Apache's configuration was a nightmare. Especially compared with another major free offering, which is configured with GUI boxes...
    • by Anonymous Coward
      And just what web server do you prefer? There isnt a product out there that can touch the versatility, reliability and stability of apache.

      -ax
    • Try Resin [caucho.com], a non-free open-source servlet/JSP server. It can run standalone or as an Apache plugin and absolutely screams. It works great with the IBM JDK under Linux and has very cheap licensing fees and incredible developer support. I myself am not partial to the whole Java phenomena, but if I had to use a web server for serving up such code, I wouldn't hesitate to use Resin.

      Sometimes, one has to step back from the plethora of big-name projects and realize that people are making considerable effort righting the mistakes made by the early pioneers of that medium.

      And sometimes, paying a little for a server engine ain't such a bad thing. Most companies with budgets can afford cheapo licenses.
    • I've got no experience with Resin and not too much with Tomcat. (I've used JRun and Websphere in the past and I like them both.) Anyways, there is a company at my local Java Users group that really speaks highly of Resin - I think this is the link [caucho.com].

      Obviously it's not open source which isn't exactly what you're looking for. On the other hand, what I hear is that it is fast, stable, and inexpensive. ($500 per deployed server.)
    • I'm no big fan of Apache (or its configuration nightmare specifically).

      So the majority of webserver admins in the world are having a bad dream ?

      Look, trying to use logic and something arguable to express your views is OK. But when you claim that the majority is stupid without any backing for your claims. It realy looks the other way.

    • If you dont like tomcat you can check out Jetty. Its true that tomcat can be used as a standalone webserver, but is not designed that way.

      Jetty instead is a full-featured webserver with all the functionality built in. And now that its integrated in the JBoss 3.x series you can use JBoss instead and take advantage of the WAR deployment/redeployment.

      You know, you can use JBoss and if you dont need the EJB, JMS, etc functionality you just disable it and create a custom configuration.
    • by Codex The Sloth ( 93427 ) on Tuesday August 20, 2002 @10:05AM (#4104084)
      Resin is significantly faster than tomcat. Catalina (Tomcat 4.x) has close the gap somewhat but if they still have a long way to go. OTOH, if cheap / stable is all you need then Catalina is a great way to go. FYI, Resin comes with all the source but is not free. Any of the EJB server will be total overkill and the overhead will soak you. And Websphere (at least the servlet side) is based on Tomcat (as is JBoss).

    • Apache on the web tier and I'm no big fan of Apache (or its configuration nightmare specifically).

      What "Configuration nightmare"?! Out of the box it'll run on a *nix machine (maybe Windows, but I've never tried it on a desktop OS). Really, there's just one well commented text file to edit, unless you intend to add user authentication to directories (which itself isn't difficult) - it's hardly brain surgery.

      Now sendmail.cf on the other hand....
  • by codepunk ( 167897 ) on Tuesday August 20, 2002 @08:30AM (#4103451)
    We use a BEA app server at work for our order processing system. Generally it works ok, but serious bugs in it cause us a lot of greif and downtime. First off it has serious memory leaks in the performance pack (trading speed for stability). We have to boot the BEA app server at least once a week least it runs out of memory and crashes. We are currently looking at JBOSS as our new production application server due to it's stability. If you code smartly you can move the code back and forth so you really have nothing to loose....

    • by JThaddeus ( 531998 ) on Tuesday August 20, 2002 @09:22AM (#4103788)
      Amen! I have been forced to run BEA and it has been agony. Plenty of things that run under Tomcat won't under WebLogic. IMHO, this just plain wrong! Afterall, isn't Tomcat the *reference* implementation? Examples:

      6.0SP2 would not honor VariableInfo.NESTED in custom tag libraries

      6.1 requires the weblogic.xml file in your WAR. Huh? Why in the WAR?

      6.1 will hang for 30 seconds on your servlets if you open and close the stream without sending anything on the stream

      6.1SP2 to set the proper application CLASSPATH

      6.1SP3 fails to handle code that translates a SAX2 event stream to HTML using Xerces (SAX2) and Xalan (XSL); I'm dead in the water with this because our application depends on SAX2 streams

      Honestly, I think I have spent more time tryiing to make WebLogic work than it took to write the application in the first place!
    • Ahh, so you guys are the ones that BEA talks about that have the "Godzilla Bean". Yes coroner, I did put 300 methods in my Godzilla bean, is this the cause of death?
    • [An independent BEA consultant writes...]

      Yes, releases can be buggy - I never liked 6.0 - but after a couple of service packs it settles down. 6.1 is a fine J2EE 1.3 setup, and I like where 7.0 is heading, really pulls together all the complex bits.

      On to specifics, if you're hitting memory leaks with the performance pack enabled, you are using some old version - no question! The only problems I've hit with the perf pack on current releases are to do with relatively obscure things like HTTP pipelining (the latter is a buggy spec if you want my opinion...). Turn on INFO level logging and check the report of java.library.path - you may be picking up an unintended version of the perf pack DLL/.so libs.

      BTW Could be that now is exactly the wrong time moving an order management-type application from BEA to JBoss - WebLogic 7 has the Web Services and workflow management pulled together and when the Workshop tool caters for both (end of year?) constructing complex flows should be a breeze.
  • We use it (Score:2, Informative)

    I won't give out the company name, but the shop I work for has several server-side software products we deploy on Solaris that use Tomcat & Apache, for use in IP telephony. A solid combination IMHO (although my personal preference on Unix is Apache & mod_perl).

  • Is Tomcat crap? (Score:3, Informative)

    by jukal ( 523582 ) on Tuesday August 20, 2002 @08:31AM (#4103460) Journal
    I don't know, but I archived this article [weblogs.com] when I saw it. The article contains some benchmarks made by an obvious geek, he also talks about the price.

    "In conclusion, yes - in my book Tomcat is crap. I haven't actually really touched on the problems with Tomcat here (other than it has bad performance and bad developer productivity) and I apologise for that. Perhaps I'll get to them another day. For now, consider the other alternatives until Tomcat improves (which I hope - but doubt - it will)."

    • Bad developer productivity?!? Not at all. We develop using JBuilder, which comes with tomcat built in and allows for very effective development.

      Since I like the development and production environment to be as much alike as possible, we also run tomcat for production (on an important 1000-user intranet application for a large bank). We run it stand alone to avoid problems with the connectors. I see no reason not to, since all our pages are dynamic so Apache would not bring any speed advantage, it would only be an extra relay/bottleneck between the servlet engine and the browser.
    • Re:Is Tomcat crap? (Score:4, Insightful)

      by _xeno_ ( 155264 ) on Tuesday August 20, 2002 @12:38PM (#4105574) Homepage Journal
      There are issues with that article that he doesn't mention. For example, what is his test page? Is he serving static content? Does he have Tomcat in "reload" mode (where it checks for updated code every time a servlet is executed)?

      I can't help but think that this article is just poorly written. It doesn't really paint a clear picture of what he's using Tomcat for. He mentions nothing about the various configurations tested. It's way too easy to just write him off due to an overly evident bias against Tomcat from the beginning. (Hint: when attempting to persuade people, calling a benchmark test "Is Tomcat Crap?" reveals your bias...)

      I use Tomcat at work as the development platform we use. We're probably going to be using Oracle Portal for the production system (not that I know what that is or what it uses for it's JSP engine, but...). It works fine for a development platform.

      I haven't done any performance testing on it (yet), but when I get the chance, I might look into it. It'd be interesting to find out what my results are. I have a suspicision that Tomcat performed poorly in his "tests" because the other servlet engines came in an optimized for speed setting while Tomcat comes "optimized" for development.

  • JBoss ! (Score:5, Informative)

    by FullClip ( 139644 ) on Tuesday August 20, 2002 @08:31AM (#4103465)
    JBoss [jboss.org] is an excellent fullfledged J2EE application server.

    They even offer consultancy if you cannot get it right the first time.
    Excellent award winning server, excellent support, what do you need more ?

    It has Jetty integrated and gives you the full J2EE stack.
    You can get it to work with Tomcat too: no problem.

    Check it out, the design is awesome for the techies.
    The support option is great for the management.
    Everyone's happy :)
    • JBoss == Tomcat (Score:3, Informative)

      The servlet engine used in JBoss is Tomcat. Since he said he didn't need to do EJB (Servlets / JSP only) there is no value in using Tomcat with an EJB server added. That having been said, if I had to actually use EJB for something (shudder) I would use JBoss.

  • don't get too cheap on the Intel HW, I mean quality hardware - no software runs well when the tin memory contacts get flaky or the fans seize up after a week. Use a percentage of sw saving for quality stuff and rest is punching buttons with minimal finger pointing at the resident screwdriver jocky & board swapper.The last production box we got even for Msft 2K server was just a 1.1Ghz P3 w/ PC133 ram, and 2 40Gb SCSI disks (low voltage differential, new one on me). Giving untrusted hw a weeks thrashing under simulated production conditions builds confidence immensely and avoids em-bareass-ment.

  • Apache is nice and fast for serving up static pages, and really nice with php pages.
    But in my own personal experience, and this was 2 years ago, Tomcat was really slow. It seemed to be just average with jsp pages, and then the more towards the j2ee route you went, the more worthless it was.
    we were mainly using it as a quick and dirty testing/training server system, so I would assume that perhaps it has either come a long way since then, or it is only really meant to server jsp pages.
  • Tomcat (Score:4, Insightful)

    by PacketMaster ( 65250 ) on Tuesday August 20, 2002 @08:36AM (#4103500) Homepage
    We've been using Tomcat in a production environment for 1 1/2 years and before that we were using Tomcat's predicessor JServ. It's been rock solid. 4.0 brought a lot of nice changes (like not overwriting the logs on startup!) and 4.1 is a refactoring release for performance. The one thing to keep in mind about Tomcat is that you have to write your own wrapper script/program to make Tomcat start up as a non-root user. If you're going to use it in conjunction with Apache, Apache2 will only work properly with the ajp13 connector. The webapp/warp connector doesn't seem to work properly yet.

    If you're going to replace BEA though, consider looking at JBoss which is a true J2EE server unlike Tomcat which is just a servlet container. To replace a commercial product such as Weblogic, WebSphere or iPlanet, you want to look at JBoss for a complete J2EE/EJB solution.
  • by TheICEBear ( 536953 ) on Tuesday August 20, 2002 @08:39AM (#4103516)
    I come from a similar situation and have managed to do what you want to do. To sound a little zen don't try to change their minds just show them the benefits. In my case I drew on my knowledge on the lack of vendor lock-in combined with the economics of the situation and the inclusion of support in our seperate support contract (really cheap support at that).

    As for support that was never really and issue with us so I have no argument there. Now Tomcat has some flaws (most in the JSP compiler Jasper and their live redeploy area), but is otherwise a very sweet little servlet engine (don't call it an appserver it isn't one in the J2EE sense of the word and that is the game you're playing when you use things like servlets).

    Once it has compiled your JSPs it works just fine and the sweet things and the selling argument for our projects was redundancy of providers. You have a change of enviroments like going to another servlet engine. With a very minimal amount of care in your coding and everything is portable in fact if you stick to the Servlet/JSP api then you're good to go.

    In fact we had some time one evening and switched between Tomcat, Resin and Jetty with only a few minutes spent making the configurations fit and the files unpack and install.

    On a sidenote if you can delay any lock-in on a specific version of Tomcat, try and see if you can get your system over on the upcoming Tomcat 4.1 I am loving the improvements it brings esspecially in speed.

    You should try to change his opinion on jBoss though. jBoss has been the most loved thing about that recent projects (and EJB writing is in combination with a good Ant script and XDoclet http://xdoclet.sourceforge.net not that big a pain). It is probably the most stable thing about this entire project with hot redeploy (great for development), good performance and great ease of use and install on top. In fact the new 3.x version is even greater with clustering, failover and some very interesting innovations in the area of control over which parts of the server to actually run via SARs and JMX. But enough about all this.
  • by rjkimble ( 97437 ) on Tuesday August 20, 2002 @08:39AM (#4103518) Homepage Journal
    running on Linux for all our clients. We build and deploy customized web apps for our growing client list. We have been running Tomcat for more than a year, and its performance has been superb. Of course, our clients don't have high volume web sites. And we're not a large company.
  • Tomcat is fine (Score:3, Informative)

    by Hard_Code ( 49548 ) on Tuesday August 20, 2002 @08:43AM (#4103539)
    We use Tomcat pretty extensively over here (major league northeastern university). I have heard that Jetty and Resin are much faster. I have also heard TONS of praise for Resin (faster, easier to configure, deploy, etc.), so you might want to look into that.

    That said, Tomcat is perfectly adequate. Unless you are running Ebay or Amazon.com or something, your main bottleneck will probably be your database IO. Typically Tomcat (and any servlet engine, in general) is set up with mod_jk hooked into Apache, so that Apache is the frontend that serves all static files, and *only* those paths which are servlet/jsp get forwarded to Tomcat. In the recent past there seems to have been some flakiness in the Apache->Tomcat connector, but I presume that has been solved by now. Also, until 4.x, the configuration file format, and class loading mechanism were changing each release, but I believe that has settled down.

    Like many Apache (or maybe Open Source in general) projects you pay for not having the depth of features a commercial product would, but you get in return breadth of features, and the comfort of a de facto standard with tons of inertia and support behind it. Besides, the J2EE specs are written sufficiently well, that any servlet engine implementation is basically a dime a dozen. You won't lose with going with Tomcat - and you can always switch to a commercial product if/when you feel you need richer/deeper features (I know people who develop on Tomcat, but deploy on Resin).

    I must still be naive because I still can't fathom the absolute craptacular $$$,000 amount companies spend on commodity software. Unless there is something you *really* need in a commercial product, it is usually not worth the hassle chaining yourself in.
  • by imr ( 106517 ) on Tuesday August 20, 2002 @08:44AM (#4103545)
    to tune your answers.
    he doesnt want to know what he can gain by using either of them, he wants to know he wont lose anything.
  • UK local authorites, via the Accessible and Personalised Local Authority Websites [aplaws.org.uk].

    This is a web toolkit based on the Arsdigita (of Phil Greenspun fame) Community system.

    Their setup is *nix, Apache, Tomcat/Resin and Oracle.
  • by Epesh ( 2854 ) on Tuesday August 20, 2002 @08:47AM (#4103572) Homepage
    I've used Tomcat for testing against the Sun specs, and I find that it's slow and not worth the money you spend on it.

    Yes, I know it's free. Pay attention.

    It does a relatively poor job of implementing the spec itself, and the spec is supposed to be its reason for being. It's gotten faster over time, which is nice, but it's still not very good at handling things. Tweaks abound, but running a custom version of a servlet container isn't likely to bring comfort to you... I hope.

    I'd suggest spending some money on the container, myself; Jetty [mortbay.org] is okay, but I personally prefer Orion [orionserver.com], which is fully J2EE, fast as all get out, and very, very easy to administer. Installation of an Orion instance takes three steps: unzip, copy tools.jar, java -jar orion.jar. Done. It's also free for development, so there's no per-seat license cost for you to use it to write code.

    An aside: Oracle recently posted ECPerf numbers which were very good, and Oracle licensed the Orion codebase... and Orion costs thousands less. Since ECperf yields numbers based on dollars per transaction, you'd think Orion would kick butt on ECPerf.

    I find Tomcat to be acceptable only for compliance testing, because so many people think it's the best that out there (because of the price point). I've spent a lot of time having to work around Tomcat; I'd hope you didn't feel like doing the same.

    • I had heard bad things about Tomcat's performance (especially with a load - and that has improved over the last year or so), but not much about compatibility. I personally have to deal with Websphere and Weblogic, and would take BEA's product over IBM's in a heartbeat (that's my opinion, not my company's opinion, although many developers here share that opinion).

      Truth be told, I haven't seen many compatibility problems, though. Usually something that doesn't work on Tomcat doesn't work on Weblogic or WebSphere, either. If we do find a bug in the JSP engine, it usually is fixed quickly by the vendor (except, AHEM, in IBM's case, but I'll let bygones be bygones :) or worked around. Most of our code just provides a client form-based interface, though, so I doubt we're pushing the limits of the J2EE spec.
    • by Turmio ( 29215 ) on Tuesday August 20, 2002 @09:58AM (#4104026) Homepage
      What the hell are you talking about?

      You didn't mention what spec you mean but I think you meant that Tomcat implements Servlet, JSP etc. specs poorly?

      I wonder what is your standard for poor but you can't get any better compliance than what you get with Tomcat namely it happens to be the official, Sub-blessed reference implementation of these very specs. If you're in doubt, then check these URLs: Java Servlet technology - Implementations ja Specifications at java.sun.com [sun.com] and Front page of Tomcat site [apache.org]. Thank you.
  • by potcrackpot ( 245556 ) on Tuesday August 20, 2002 @08:52AM (#4103592) Homepage

    From my experience, Tomcat 4.x is faster than Apache and JServ.

    Don't know how it compares to other servers (at least, from experience I don't), for example IIS, Resin, JRun etc.

    Tomcat 3.x WAS very slow - for example, who had to combine Apache and Tomcat to get anything reasonable - using Tomcat for JSP and servlets, and Apache for static pages. This was in itself a bit of a nightmare. Tomcat 4 is miles better.

    Comparing JRun to Tomcat for performance, see here [macromedia.com].

    Compared to Orion and Resin, Tomcat also lost comprehensively [weblogs.com]. The arguments raged for a while over performance (for example [metronet.com])- but not many about whether it "did what it said on the tin".

    A more serious point here is that your bosses care more about the name and image than the quality. I'd think about trying to convince them that this is Not A Good Idea. For someone who IS using Tomcat in production, just do a google search; you'll get quite a few, for example [eapps.com].

  • Primavera's [primavera.com] Expedition v8.0 ships with the following.
    • Jetty Web Server v3.1.1
    • JBoss Application Server v2.43
    • Expedition database running on Sybase Adaptive Server Anywhere v7.0.3
    • Sybase JConnect v5.5 JDBC driver
    • Expedition 8.0

    I just installed the Jetty Web Server the other day so I don't have any real data to provide, sorry. I know of a few people that use it and have been happy with it. The only complaint that I've heard is that the pages take long to load. The person that said this thought it might have to do with the page being Java, but I think it might just be the database itself causing the slowdown. Just my $.02 worth.
  • by nevermind ( 19336 ) on Tuesday August 20, 2002 @08:53AM (#4103598)
    We have migrated to Linux, Apache, and Tomcat over the last year-and-a-half. We use it both in development and in production, across 100 or so boxes. As with everything, there are issues, but for the most part we are very happy. Even most commercial vendor's idea of a "big" site doesn't come close to what we do, so we have found very little difference between problem solving in the open-source and closed-source worlds.

    For what we do, you can't beat the price... And yes, that includes the price of our time.
  • ...in the catalog server. It wasn't serving up HTML per se:, but it was still used to run the application.

    Since the catalog server was a major piece of the procurement software, it had to perform. If people can't populate their cart, then what's the point? This software was sold to various 'largish' companies without any complaint on the performance of the catalog....

    I'm not sure what they're using now, since I bailed just before the .bomb meltdown...i2 bought the company, and we all know (!) where i2's stock price is - 0.71 yesterday. I'm not even sure if they're still selling the product, which would be a shame.

    If the performance isn't good enough, throw a few caching proxies in front of the web servers. You may want to do that regardless of what web server software you run in the back end...
  • Blogger (Score:3, Informative)

    by mkelley ( 411060 ) <slashdot@[ ]lley.net ['mke' in gap]> on Tuesday August 20, 2002 @08:58AM (#4103636) Homepage
    I believe Blogger has moved from the ASP-based code that runs free Blogger to Jakarta & Tomcat....the Blogger API page [blogger.com] is a plain Jakarta/Tomcat page. According to some of the comments Evan Williams made recently, about "moving away from ASP [blogger.com]" and some of the discussion over at Blogroots [blogroots.com] point to it.
  • by standards ( 461431 ) on Tuesday August 20, 2002 @08:59AM (#4103640)
    My former employer, a very large areospace company, was at one time very very much against any software that wasn't back by a "stable corporation".

    The excuse was that if something went wrong, my company could sue the pants off the software provider. Of course, they almost never did that - instead, they just wouldn't pay the bills until the provider complied with company demands.

    Enter terminal emulator software. The popular 3270 emulator cost about $500+ per desktop. And with 10,000's of desktops, that was... um, expensive. So I started my own little cost/benefit analysis. We could buy a shareware product for $5 per seat, and it was very robust and served 99+% of the users (except for mainframe sysadmins, of course!).

    And the savings was amazing. We rolled out the product slowly. Everyone was happy. In the end, everyone used the product.

    This one little step put us on the road towards purchasing more shareware. Soon afterwards, we did the same kind of argument with freeware - and won.

    Conclusion: Start with something simple that you can back away from ... just in case it doesn't work out. Perform a cost/benefit analysis. Purchase a product if it's the right decision - don't let "free" blind you. Write white papers for management. Counter industry FUD "reports" ... as they're often BS that are easily attacked.
  • the big question begging moving from a simple jsp/servlet engine like Tomcat/Jetty/Resin to a full blown J2EE app server like WebSphere/BEA/JBoss is Do you want to run EJBs?
    i'm not an expert on EJBs by any means, and i'm trying to ask this same question of my own projects, but what i keep hearing is this: EJBs allow me to run much more scalable than servlet/javabeans.
    i dont know what your prospective usage numbers are, but if they are large scale (aka site on the internet that loads of people will hit hard) then you want to use an EJB architecture because you will be able to scale up with lots of big servers. given that you are working on PCs, my guess is that this is not the case.

    Also, i keep hearing that utilizing an EJB App Server will bring with it database connection architectures like Container Managed Persistance etc. BUT... there are some great examples of utilizing other data access patterns like Data Access Objects (see the jpetstore example [ibatis.com])

    i think it comes down to proper application architecture. make sure your applications have good design, and keep in mind scalability (especially with the data access bottleneck) and you should be ok.

    oh yeah... in favor of tomcat ... Tomcat is used by IBM WebSphere to run all the jsp and servlets in webSphere, which in turn can then utilize the EJBs...
  • We're using Tomcat on servers here to replace PHP on Apache. Eventually we're going to phase out Apache and use just Tomcat. It works great except for one tiny thing for us, there's no equivelant to htaccess. Which makes setting up restricted parts of our website a tiny bit annoying. But once you get the hang of the xml config files it's pretty easy. As for robustness, one of the sites we run it on gets a few hundred thousand hits to the same 3 or 4 webapps. I think it's great. But I can't give out the company name.

    Kintanon
  • Their position is that it will save them hardware costs to run on Intel machines instead of big IBM or Sun iron.

    You mean AMD Athlon MP machines, right? :-)
  • Navy (Score:3, Funny)

    by jonasson ( 224996 ) on Tuesday August 20, 2002 @09:15AM (#4103738)
    I've heard good things that the Navy has been using Tomcats in production use for quite a while now. They even made a movie [imdb.com] about it.
  • I work for Watchfire [watchfire.com] a leading maker of website quality management software. One part of the suite is FeedbackXM [watchfire.com], a user feedback survey system (of the survey button on the web page / pop-up survey kind.)

    FeedbackXM uses Tomcat as an application server. This information is in the customer documentation.

  • http://trafac.chmcc.org

    It is a bioinformatics web site for finding promoters in DNA sequences.

    I did not design it, I work on it though (use and develop it).
  • Won't put our company name on here(it is a large UK health insurance company), but we use apache and tomcat for all our large scale web apps. For example we saved huge amounts of money by fronting our legacy middleware product with an app that you call using XMLRPC and it sits on Apache and tomcat on a Sun E10k domain. For performance it handles about 30 requests per second and about 1.2 million calls a day.

    The reason for this combo? It works, we tried other products and they couldn't handle the load, and when you dig deep then the parts we were using were old versions of Apache and jserv or tomcat! :)
  • At NetDoktor A/S we use Tomcat/Weblogic currently to run our Java platform. We're deseperatly trying to gain time to look at JBoss to see if we can kill off Weblogic.

    So I'd like to stand forward and say, yes we use Tomcat, and yes it's been in production so long we'd never consider Weblogic as a JSP/Servlet container.

    /Ian
  • by Simon Brooke ( 45012 ) <stillyet@googlemail.com> on Tuesday August 20, 2002 @09:31AM (#4103844) Homepage Journal
    ... and so do many of my customers. This may not be as good an endorsement as it sounds as none of the sites concerned is particularly high traffic and performance isn't really a big issue. However I've also tried JRun (fundamentally broken and useless), BEA WebLogic (huge, over-complex, bloated) and Jigsaw (very nice for small installations but last time I used it didn't yet support Servlet 2.2).

    I would have no hesitation in recommending Tomcat for low and medium traffic sites; I don't really know enough to recommend it for very high traffic sites.

  • Try Resin (Score:5, Interesting)

    by peterdaly ( 123554 ) <{petedaly} {at} {ix.netcom.com}> on Tuesday August 20, 2002 @09:31AM (#4103847)
    I run two production Servlet Containers. One is Tomcat 4.X, the other is on Resin. While Resin is not open source, the cost is only $500/server, which is quite low by J2EE standards. I believe it is free for development, but I could be wrong.

    I tried Resin since I have heard "buzz" about it in message forums, and now can't say enough about it myself. Tomcat has a lot of quirks with reloading updated war files, reloading modified JSP's, etc. Resin does not have these problems, and I believe is much better suited for a non-stop production envirnment.

    In Tomcat, it is not uncommon for me to have to restart the container when rolling out updates where certain things have changed. In Resin, I can even add or remove a JDBC Connection Pool from the resin.conf file and have to pool rolled out or back without any additional intervention from me. In short, it just works. Not only does it work well as far a few (if any) glitches, it is VERY fast as well.

    For a commercial envirnment, I suggest you try Resin just to see if you find the value it adds over Tomcat worth it for you. I did.

    -Pete
    • Re:Try Resin (Score:2, Informative)

      by AndyMan! ( 31066 )
      I agree about Resin. We run it exclusively on our production machines.

      It is "sort-of" open source, you can see the code, make changes, just not redistribute it. The makers of resin (caucho.com) are very good about taking good code, though...

      Development licence is free, and production licence is $500 / server.

      Resin blows both webLogic and tomcat out of the water.

      _Am
    • We just finished some benchmarking on our internal app-server. we found that tomcat ran 2-4 times slower than BEA/resin. Before we knew about resin, the cost of buying the extra hardware for tomcat was greater than paying for the extra BEA licences. That said.. we found resin, it is just as fast as BEA in our tests, and we are looking at using it and saving some $$$
  • We are using JBoss and Tomcat in a production system, deployed in the government offices of 9 different governments.

    For more information, see www.trackernet.org
  • They say use Jetty (except for the ones that say to use Resin).

    Here are some good reasons to use Resin [caucho.com]:

    • Fast, reliable, scalable
    • Easy configuration
    • Full source code provided under a "Developer Source" license [caucho.com], which makes it competitive with "free" products, at least from a commercial perspective
    • Development use is free - no licences need to be purchased for development and testing purposes. Runs well on Linux as well as Windows; latter can be useful for test deployments on developer machines
    • Inexpensive to deploy: $500 per server for the JSP version, $1000 for the EJB version; additional for priority support options.
    • Commercial support - see their pricing page [caucho.com].
    Here's a testimonials and customer list [caucho.com] which includes some pretty big names.

    Despite any appearances above, I'm not associated with Resin, except as a customer. I first heard about it here on Slashdot, a couple of years ago. We evaluated it and eventually switched a number of applications from both Tomcat and commercial servers and have never looked back.

    Other servers that might be worth looking into are Orion and JBoss, but I don't know how they compare on some of the points above. The availability of commercial support from the vendor can be a clincher, and the source code provides some insurance against the vendor disappearing.

    I agree with all the cautionary comments about Tomcat, although we haven't worked with it for some time now.

  • Cisco is working towards moving all their web based applications to tomcat. Just about all their new applications are being written there.

    Tomcat works for us. Regarding speed, if everybody says it's slow then maybe they're right. However, in my experience it's the database interaction that's slow, not the java code. Tuning the query can cut seconds, tuning the java cuts milliseconds. It's hard to fuss too much about the milliseconds when there is low hanging optimization fruit to be found in the sql.

    Vanguard
  • by md17 ( 68506 ) <james@@@jamesward...org> on Tuesday August 20, 2002 @09:47AM (#4103951) Homepage

    I manage a few servers...
    1 Apache box on an Ultra 5 (Slow sun box typically used as a workstation)
    1 Tomcat box on an Ultra 5
    I use mod_jk and hide the tomcat box behind the web server. This adds a nice layer of security and lets Apache process .html pages.
    In total I have 5 instances of Apache, ~100 instances of tomcat, and ~150 web sites. The apache box sustains about 2MB/s and about 400k/s gets sent to the Tomcat box to deal with. I have had very few problems with Tomcat 3.3.
    If you need some redundancy I would recommend using the mod_jk load balancing [ubeans.com]. It works very well and is simple to setup.

    My advice: Don't litsten to all the Slashdoters who gripe about anything to do with Java, give Tomcat a try. It works for me!

    BTW: If you want to get into J2EE stuff, absolutly use JBoss [jboss.org]!!! It rocks!
  • with "servlet container" and-or "JSP container". both tomcat and jetty are fine examples of servlet and JSP containers.

    an app server, i.e., a server which serves up J2EE applications, is more generally thought of as a combined servlet, JSP, and EJB container. BEA, IBM WebSphere, etc, and JBoss [jboss.org] are the real players in this space, but Oracle has their own app server rolling along now too. FYI, JBoss ships both a Jetty and Tomcat distribution.

    Tomcat is the reference implementation for the Servlet and JSP container APIs. This means it is absolutely the standard. While Jetty is great for being "lighter" weight and more easily embedded, Tomcat would have to be my choice for "open source" servlet and JSP container.
  • The problem seems to be that it is extremely difficult to benchmark such servers. Greg Wilkins [mailto], one of the primary authors of Jetty [mortbay.com], explains the issues fairly clearly in a response [yahoo.com] to the Jetty Benchmark thread [yahoo.com] on the jetty-discuss list [yahoo.com].

    In addition, experience shows that J2EE application optimization is not as straight forward as other Java applications, so it is easy to get radically different performance results from a servlet with only minor tweaks. There was a wonderful presenation at JavaOne 2002 San Francisco about servlet optimization (link for atendees only [sun.com]). Among other things, the author demonstrated. a simple 6 line "Hello World" servlet that is written in standard style, yet can be made to run 3x faster with only minor tweaks. He also shows that testing under load reveals that servlets can behave much differently under load and that the only way to really write fast and reliable servlets is to write them as you normally would and then test them mercilessly.

    My conclusion is that you can't believe any of those published bechmarks, they're mostly biased marketting crap (everybody's benchmark seems to show their product is fastest). What you really need to do is load up multiple servers and configure them to do what you need them to do and test them under load to see how they perform in your environment. I know it's not what you want to hear, but since there are so many features that have varying performance, it's the only way to really find out.

  • But just last week at our local Java users group [denverjug.org] James Duncan Davidson, the original developer of Tomcat (and Ant), and he really didn't recommend using Tomcat by itself in a large enterprise type setting where speed is an issue. He said using Apache to help serve up static pages/images will certainly help but it is important to keep in mind that Tomcat wasn't built for speed, it was built to be a reference platform.

    On a side note Davidson is a good speaker and worth seeing if you get a chance to.
  • We may have been on a conference call a few weeks ago. Call it a hunch.

    Here is my take...

    I suspect you did a big cost analysis of the different Application servers out there. Looked at IBM, BEA, and a few of the other smaller players. Tomcat was listed, but written off. Things moving at the speed of business - the developers started building with Tomcat. Now there is lots of code and tweaking to Tomcat that may (may not) prove difficult to migrate. Additionally, it looks like your not going to worry about an EJB container for the first couple phases and focus on STRUTS instead.

    Tomcat works great for development. It works amazingly well for fewer than 100 concurrent users - and that really works out to a lot of people. The cost factor is less of an issue. The full blown EJB container / CORBA ORB / kitchen sink tend to cost a pretty penny.... but a strait JSP / Servlet container do not. Weblogic Express goes for a couple grand - and contrary to what folks my say, you can get decent support from them. (Just make sure you are the contact rather than someone on the "business" side). I guess I view the cost as irrelevant at that point. Take the time to benchmark and load test on a production box (sparc?) a couple different solutions. . Some of the commercial Servlet engines are very fast - at the expense of some bleeding edge features you may get with Tomcat. Figure out if runtime or development time is more important.

    That being said, yes... there are folks using Tomcat in production. I have seen tons of departmental stuff out there. Enterprise gets a little rougher - I've seen a few companies wedge it in.
  • Anyone running Vignette's V/6 Enterprise Application Platform is running Jakarta Tomcat. It's a prerequisite. You can of course make custom connections to another servlet engine, but it isn't supported.

    I don't have a customer list or anything, but there it is.
  • Complementary (Score:5, Informative)

    by Martin Spamer ( 244245 ) on Tuesday August 20, 2002 @10:18AM (#4104220) Homepage Journal

    There seems to be a lot of confusion about what Tomcat, Jetty, JBoss and J2ee App-Servers. They are not really competative but complementary products. A Java AppServer is composed of [at least] three main components. The HTTP deamon, a Servlet/JSP container and a EJB Container.

    Jetty is a primarily HTTP deamon, it is designed to handle HTTP request in a scalable manner.

    Tomcat is a Servlet/JSP container, it implements the Servlet API it provides limited HTTP handling and no EJB support. Tomcat is highly reliable more so than most commercial 'industrial strength' App Servers. On the performance side; the Tomcat 3.x architecture is not hot but is adequate for many applications, all but the heaviest loads. Tomcat 4.x is significant better in this regard, because it includes an enhanced HTTP deamon.

    JBoss is an EJB container which uses Tomcat 4.0 as it's HTTP deamon and Servlet container.
  • Check out Computer Associate's portal product, CleverPath [ca.com]. It uses Tomcat as its application server. My company is testing CleverPath right now for deployment as a B2B portal for our customers.

    If I were you I would let somebody else do the heavy lifting on benchmarks, where it's in production, etc. Contact CA, tell them you are thinking about deploying their portal and you want to know where it's in production and what the benchmarks are. Since CleverPath can be deployed with a third party app server (BEA, WebSphere, Sun ONE) you need to specify a native deployment for the reference customers. Since you know that the app server architecture is built on Tomcat you will have good references for Tomcat that you can use to demonstrate its abilities, or lack thereof.

  • Ok guys, that's my experience. We had bad results with tomcat, our group of linux boxes (4 biprocessor pentium class machines RH 7.1) had big problems only with 60.000 pages on a day. What I appreciate less was the Tomcat ungraceful behaviour under stress. One moment all is Ok, after some seconds it starts trashing. We changed the code, installed Resin and achieve to serve 500.000 pages on a day with only 2 servers!!! Tomcat version was 3.2.something... Good luck!
  • Check out AnalystScan [analystscan.com], a DB and JSP intensive site running exclusively on Tomcat and PostgreSQL. It's survived several mentions in the mainstream press, as well as a mini slashdotting.

    The only caveats I have for you are to be sure that you use the groovy built in connection pooling for your DB resources, and that you tune your JSPs before going into production.

    Good luck!

  • We use Jetty 4.1 with JBoss 3.01 in production for our sites, which are pumping out about a million pages per day.

    Jetty is very well integrated into the JBoss stack, and all calls between components are optimized, which is an attractive bonus for us.

    Support for Jetty has been great on the lists. Very fast responses.

    We used to use Tomcat, but it crumbled under heavy load. I hear these problems have been fixed, but I'm not going back.

    http://www.liberationmedia.com
  • I was volunteered to evaluate application servers to replace what we had been using (New Atlanta's ServletExec).

    Their main interest is in speed and scalability... and the $0 pricetag of Tomcat is also of interest to them (though not critical).

    I've been testing Tomcat as well as Caucho Resin (www.caucho.com) and have found Resin to be much faster and feature rich, but my bosses are actually pushing for Tomcat because it's open source and free. I think mostly also do to the (hopefully) larger installed base so peer to peer support will be more of a reality.

    BTW: Does anyone know a decent... free package that can script through a complex website (logins cookies, javascript) and load it down to test for performance? They want to see numbers of both of these and I have till friday!! eek.
  • by eyefish ( 324893 ) on Tuesday August 20, 2002 @11:03AM (#4104684)
    We've tested both Tomcat and Resin, and decided to go with Resin for several reasons.

    First of all it is very stable and very fast. And secondly, it has a very comprehensive way to do clustering, fail-over, and distributed sessions management.

    In just a couple of minutes you can set it up to cluster with several copies of Resin, each residing on a separate machine, on the same machine, or even in the same VM. You can even set up a Resin container to be a backup of another Resin container in the same machine, so you get both inter-machine and intra-machine failover.

    You can also do distributed sessions in several ways (with TCP messages, database storage, etc), and you can even force a user session to stay within the same Resin container out of a clustered group.

    As for Web Services, we heartly recommend GLUE from The Mind Electric. It's bar-none the absolute best (in terms of speed, stability, and easy of use) Web Services toolkit available for ANY platform. It puts Microsoft's .Net to shame, and it's way easier than offerings from IBM, Sun, Bea, Borland, or the Apache/Tomcat efforts. It's so easy to use that already you can make your *existing* applications be Web-Services compliant without re-writing or re-compiling them!!! You just tell GLUE which classes and methods will be exposed as Web Services and it automatically generates WSDL and starts listening for SOAP clients!!!

    As for a database, try the latest non-beta version of mySQL. It supports row-level locking, full transactional support using innoDB, and it is fast (specially considering its price). (Note that postgress is also a good alternative).

    Note that like many here, I also agree that Tomcat and JBoss are great solutions to your needs, so if your boss definitelly cannot be convinced otherwise, I think you'll be fine with Tomcat at least. I only advice you to design your applications in a way that they can cluster, so that you can increase performance easily by adding more Tomcat servers to the mix.
  • haha your blood pressure doubled and pulse rate jumped 15 points, call it your exercise for the day.
  • JBoss/Jetty (Score:3, Informative)

    by GOD_ALMIGHTY ( 17678 ) <curt DOT johnson AT gmail DOT com> on Tuesday August 20, 2002 @12:38PM (#4105571) Homepage
    We've been porting an app from SilverStream (complete pos!) to JBoss. Originally we used JBoss/Tomcat, but have moved to JBoss/Jetty since the Jetty guys have been much better at supporting features via JBoss.

    I would recommend against straight servlet/JSP development. Using EJBs, you get portability to different user interfaces, data source pooling, transactional integrety, and a larger choice of security options a la JAAS.

    Since we're working on JBoss, I can write message beans for JMS systems, I have a built in timer mechanism, I can hot deploy by copying my ear file to a directory.

    I can federate enterprise wide Directory Servers (LDAP via JNDI) and Databases, integrate with MessageQueue systems (MQSeries), tie in with CORBA apps and manage everything via custom JMX apps.

    Jetty was also easier to work with in the development cycle, we didn't want to unpack the ear and war and redeploy the EJBs every time we changed a single HTML tag in a JSP, so I wrote an Ant target that copies the JSPs and associated stuff to the Jetty temp dir where Jetty does a great job of finding it and recompiling it.

    Tomcat's temp dir structure was too dynamic and unpredictable to do this. I've also found more options when configuring Jetty via JBoss than Tomcat (you don't use the std config xmls, they have JBoss specific ones that JBoss parses and passes on to the Web Container).

    The other beautiful thing about JBoss is the JMX. JBoss is really a JMX 'spine' with the EJB Container and Servlet Container (Jetty or Tomcat) as interchangable JMX MBeans. You can provide your app way more in the way of services.

    Also Jetty supports clustering, real session clustering in JBoss.

    JBoss has also integrated Apache AXIS so you can expose your EJBs via SOAP if needed. (I still hate SOAP though) Using EJBs I retain the flexibility of my user interface, since the data model and business logic are in EJBs, I can write a GUI client with relative ease, or expose my EJBs to a CORBA client via JacORB (also integrated with the default JBoss install).

    Some things to also look at if choosing the J2EE path:
    Apache Struts or Jade for web user interface development

    Xdoclet for generating your EJBs and maintaining all those XML files in your source code (web.xml, jboss.xml, struts-config.xml, ejb-jar.xml, etc.)

    Ant, become one with Ant, you'll thank yourself later.

    http://sf.net/projects/middlegen
    Middlegen, point app at database, generate CMP Entity Beans and basic CRUD ops in struts, write business logic, then user interface, done with new J2EE app.

    ArgoUML and UML2EJB
    Create a UML diagram, generate EJB code. Still a work in progress, but very promising.

    With all the development in code generation tools, I'm in danger of becoming a point and click programmer on Linux ;-) Never thought I'd see the day ;-)

    Downsides, XDoclet and Middlegen are lacking in docs, Ant has a lot of useful, poorly documented tricks, JBoss could use some more docs too, or at least better organized ones... (I even have the subscription docs)

    Believe me, get into the J2EE swing with all the loving Open Source tool goodness, you'll never want to touch Perl or PHP again. It just works so much nicer, and the pace of development is blinding fast. Also most of the J2EE open source projects deliver, and deliver on time.

    The community is great. Mailing lists are good, IRC not as good. Sites like The ServerSide and JavaLobby have a lot of good info as well and their forums are really lively.

    With JBoss and the other open source tools it's the feel of a well supported commercial environment with all the source goodness you can read, and it scales up to enterprise class systems and development methodologies, try that with Perl/PHP!

Top Ten Things Overheard At The ANSI C Draft Committee Meetings: (3) Ha, ha, I can't believe they're actually going to adopt this sucker.

Working...