Forgot your password?
typodupeerror
Java Programming

Using Java In Low Latency Environments 371

Posted by Soulskill
from the shaving-milliseconds dept.
twofishy writes "Something I've noticed amongst financial service companies in London is a growing use of Java in preference to C/C++ for exchange systems, High Frequency Trading and over low-latency work. InfoQ has a good written panel discussion with Peter Lawrey, Martin Thompson, Todd L. Montgomery and Andy Piper. From the article: 'Often the faster an algorithm can be put into the market, the more advantage it has. Many algorithms have a shelf life and quicker time to market is key in taking advantage of that. With the community around Java and the options available, it can definitely be a competitive advantage, as opposed to C or C++ where the options may not be as broad for the use case. Sometimes, though, pure low latency can rule out other concerns. I think currently, the difference in performance between Java and C++ is so close that it's not a black and white decision based solely on speed. Improvements in GC techniques, JIT optimizations, and managed runtimes have made traditional Java weaknesses with respect to performance into some very compelling strengths that are not easy to ignore.'"
This discussion has been archived. No new comments can be posted.

Using Java In Low Latency Environments

Comments Filter:
  • Thanks Dice!! (Score:2, Insightful)

    by Anonymous Coward

    This Slashvertisement brought to you by: Dice.com

  • by Anonymous Coward on Friday August 02, 2013 @10:17AM (#44456017)

    I find java helps me with my latency :)

  • Huh? (Score:2, Insightful)

    by TWX (665546)
    Why would the language of choice matter terribly much if the programmer has skills with the language?

    Wouldn't a C++ programmer generate an applicable program effectively as quickly as a Java programmer? It's not like compiling is the time-consuming part anymore...
    • Re:Huh? (Score:5, Informative)

      by bunratty (545641) on Friday August 02, 2013 @10:22AM (#44456071)
      Programs written in some languages run more quickly than those written in other languages. Some languages allow you to write programs more quickly than other languages. Therefore, the choice of language is always a tradeoff. There is no perfect programming language, and no, not all programming languages are alike as you seem to believe. If I want to write a program quickly, I write it in Python even though I know the program will run quite slowly.
    • Re:Huh? (Score:4, Funny)

      by i_ate_god (899684) on Friday August 02, 2013 @10:26AM (#44456111) Homepage

      well I am a master of PHP, so let's find out...

    • by medv4380 (1604309)
      Their referring to the lag you get in launching a Java App. There are ways around those issues, but not ways that are native to the language or the platform. Up until the hotspots are found and compiled a Java App is pretty slow. I'm not sure if this should be an issue with HFT though. The Algorithm should be persistent in memory for the trading day. The host spots should be found and compiled within a few minutes. Unless they are repeatedly launching it there shouldn't be a problem.
    • "It's not like compiling is the time-consuming part anymore" needs a sarcasm tag attached to it. Some people still deal with large compile times.
      • by h4rr4r (612664)

        That is a problem money can solve.

        A TB of ram and all PCIE SSDs can do wonders.

      • Re:Huh? (Score:5, Informative)

        by LostMyBeaver (1226054) on Friday August 02, 2013 @10:52AM (#44456379)
        Some people also work on multi-million line projects which compile in a minute or less. It's a trade-off. Most compiler issues are related to obscene (and generally unnecessary) amounts of header dependencies. 1000 lines takes an almost immeasurably small period of time to compile, but a 2 line file can take a minute if it includes the wrong headers. The ISO C/C++ library or boost is pure evil in these regards.

        A well formed program can compile extremely rapidly. A poorly formed program can often compile extremely quickly with enough CPUs working in parallel across a fast enough network. However a decent program will take time for the initial compile but take almost no time for each consecutive compile when only a file here or there changes.

        Also, to make things politically incorrect on Slashdot, good tools like Visual C++ 2012 can even take poor code and make it compile quickly if the projects files are designed well. Do your coding there and compile it later on GCC.
    • Re:Huh? (Score:5, Interesting)

      by elfprince13 (1521333) on Friday August 02, 2013 @10:42AM (#44456295) Homepage
      Manual memory management is a whole class of bugs that Java doesn't have to deal with. A good C/C++ programmer shouldn't have *that* much difficulty, but it does add to debugging costs and detracts from mental focus on the algorithmic aspects of the problem.
      • Re:Huh? (Score:5, Insightful)

        by parlancex (1322105) on Friday August 02, 2013 @11:29AM (#44456789)
        Garbage collection creates a different class of problems, namely that the performance characteristics of your program become non-deterministic. This is a Big Deal for certain classes of applications such as video games and in particular HFT. Would you like to be the person explaining why the GC caused your program to stall at an inopportune time for 50ms and lost somebody a few million bucks?
        • Re:Huh? (Score:5, Informative)

          by bws111 (1216812) on Friday August 02, 2013 @11:44AM (#44456961)

          Any JVM that would run this code will be tunable, including the ability to tune the GC so it becomes more deterministic. The fact that your desktop app runs in 'use all memory then GC' does not mean that is the only way the JVM can work.

        • Re:Huh? (Score:5, Informative)

          by Infiniti2000 (1720222) on Friday August 02, 2013 @11:50AM (#44457025)
        • Re:Huh? (Score:5, Interesting)

          by smcdow (114828) on Friday August 02, 2013 @12:48PM (#44457841) Homepage

          The article suggest that one solution is to simply not do GC until the end of the trading day. I have to admit that's a good pragmatic solution, certainly so for HF prop traders.

          Not enough memory on your servers? Add more. Still not enough? Add even more. The cost would be a pittance for any prop trading company worth its salt.

        • Re:Huh? (Score:4, Insightful)

          by Darinbob (1142669) on Friday August 02, 2013 @02:07PM (#44458991)

          Garbage Collection with a bad implementation this is true. But garbage collectors do not need to cause non-determinism; they can be used even in real time systems. The problems is that people re-invent the wheel and don't learn from all the research in the past that sped GC up. Or more likely, the system was originally designed to be a simple quick-and-dirty scripting language but its scope has grown so that now the hastily written GC is unsuitable for the new requirements.

          To be fair, good GC is hard, and it is extremely difficult to do it portably. Some good GC techniques need hooks into the operating system (ie, you want to mark pages as clean or dirty). Doing this portably sometimes means you see implementations that sit GC up on top of malloc, so that GC is merely a way to eliminate manual allocating and freeing, and you end up with performance headaches with memory fragmentation.

          Yes it's true, if you use a malloc and free directly and no GC, you can still end up with non-deterministic allocation performance! Especially with a naive malloc I see used a lot on small systems, where you just have a linked list of available regions so you have a linear search to find regions big enough. And you end up with fragmentation on any system that does not have relocatable memory (ie, most traditional non-interpreted systems), so many embedded systems prefer to use memory pools to fix those problems.

    • Re:Huh? (Score:5, Insightful)

      by Xest (935314) on Friday August 02, 2013 @10:47AM (#44456329)

      "Wouldn't a C++ programmer generate an applicable program effectively as quickly as a Java programmer?"

      No, some languages simply are slower to develop with and debug. The problem is also made worse depending on the frameworks and IDEs available. As an example, you're going to get your work done way quicker writing an application manipulating dates and times using C# and Visual Studio than you are Java and Eclipse because until Java 8 Java's date time functionality is shit and Eclipse is a dog slow IDE. With Java 8 and say NetBeans or JDeveloper though things will be pretty similar.

      At the end of the day with C/C++ you have to deal with memory management and that's just one additional piece of work that you don't have to be so concerned with with Java. C/C++ give you more scope for optimisation and more control over memory management as a benefit of that though. It's about trade-offs and figuring out what matters.

      But it's possible to be great at both C++ and Java without having to descend into petty arguments as to which is better and know when to use each in response to a specific task, and that's the sort of great programmer these institutions will be looking for and this is really what they're talking about - both have their place but in some cases getting a trading application to market a day earlier than the competition even at slight latency trade-off may be enough to net your company a few million dollars advantage. In other cases, the latency improvements of a highly optimised C++ application may instead be the key to scooping up those extra millions.

      • Re:Huh? (Score:4, Insightful)

        by 0123456 (636235) on Friday August 02, 2013 @02:43PM (#44459429)

        At the end of the day with C/C++ you have to deal with memory management and that's just one additional piece of work that you don't have to be so concerned with with Java.

        You're funny.

        I think we have about a dozen calls to new and free in a few hundred thousand lines of code in our server. The vast majority of memory allocation in C++ is hidden in libraries like STL, which we presume have been debugged.

        In Java, on the other hand, you have to be very careful with memory management or you'll either end up pausing for long periods for garbage collection or crashing with out of memory errors. Instead of worrying about whether people remember to call free after allocating something, you end up putting in caches so you can reuse objects rather than reallocate them. We've used some Java libraries designed for high-performance financial uses and they try very hard not to allocate any objects that they'll later need to clean up.

        Programmers believing that 'you don't have to worry about memory management in Java' is the reason so many Java programs are a slug-like mass of memory bloat.

        • by Xest (935314)

          But that's the point isn't it? Sometimes those hits from the garbage collector simply don't matter and if they don't matter because performance in this respect isn't the greatest concern then you don't have to bother yourself with it, whilst in C++ you have to bother yourself with it whether performance is a concern or not - in other words C++ always has an inherently slower time to market because of this inherent underlying trait and that's the point of TFA.

          If optimisation is more important than time to ma

      • by devent (1627873)

        What hinders you to use Joda Time if you think that Java's data and time are insufficient?
        Any non-trivial application have dozens or more dependencies, one more or less really is not important.
        I find the strength of Java is the enterprise grade open source libraries, one for every possible scenario.

    • by gl4ss (559668)

      for this kind of scenario I would say to write and tweak it in java.. once it's been stable in what it actually does, then rewrite it in java, then do it in c++ or c.

      though I suspect part of the problem with doing these battle algos is the need to evolve them constantly.

  • by carou (88501) on Friday August 02, 2013 @10:34AM (#44456185) Homepage Journal

    Challenge Accepted!

    • by iggymanz (596061)

      Java benchmarks always avoid the thing that makes java slower, mandatory use of heap and all the indirection for reference-semantics for user types. In other words, as soon as you start using many objects, java's performance goes into the crapper, especially using the standard EE libraries of the major vendors,

  • by sg_oneill (159032) on Friday August 02, 2013 @10:35AM (#44456193)

    The chest beating about Java vs C is kinda sad. Look, I've spent the past 20 years hating java with the fire of a 1000 suns, but having been kinda forced to use it lately I've realised its actually not a bad language, in fact its quite neat and well thought out (But god help me, somehow its date handling is even more broken than javascript)

    The problem is all the verbose cruft that goes with it. The giant overly complicated frameworks that require configuring 50 different XML files fed through a labrynith undocumented build process that allows you to write terse and insane pattern-madness code ....or alternatively just fire up JDBC and write a bloody SQL query instead.

    I think JAVA could shine if people just threw out about 15 years of insane and overly complicated frameworks and took some tips from the python and perl people and replace them with some simple but effective libraries that do one thing and do it well.

    But hey, at least its not bloody javascript!

    • by hedwards (940851)

      Java has its issues, but ultimately, it's good at what it does. Trying to use the wrong language for the job is always going to end poorly.

      From what I've seen, most of the hatred for Java comes from people that are either fanboys or have been using programs that would have been better written in some other language.

    • by Threni (635302)

      Why don't you just use c# instead?

    • by Darinbob (1142669) on Friday August 02, 2013 @02:17PM (#44459109)

      Yes, Java at its core is a simple language, with well understood concepts and features that people have been using for decades. But it's saddled with a huge amount of framework. To be a good Java developer these days doesn't mean knowing how to use Java the language, it means knowing all about the infrastructure and how to quickly tie together pre-existing functions to do what you want.

    • by codealot (140672)

      Wow, this is new. A rational, reasonable argument from a self-described Java hater... not just another "Java is slow because it's interpreted" or "Java is insecure" bullcrap post.

      I've seen legions of developers grow frustrated with the gigantic frameworks and libraries that have grown up around Java, and react by abandoning Java and building something insanely simple in another language.

      I wish just one of those guys would have instead tried abandoning the bloated frameworks and instead build something insa

  • Java does NOT perform anywhere close to as efficiently as C/C++. You might be able to get message transmissions to take the same time, but the Java environment will undoubtedly take more system resources. The same happens with any through the kitchen sink of libraries at it interpreted language. .Net, Java, Ruby. In my experience Perl runs faster than those 3 but managers have been led to believe that Perl has a slower time to market, thus is slipping from the mainstream. The closer you get to stripped down

    • Java does NOT perform anywhere close to as efficiently as C/C++. You might be able to get message transmissions to take the same time, but the Java environment will undoubtedly take more system resources. The same happens with any through the kitchen sink of libraries at it interpreted language. .Net, Java, Ruby. In my experience Perl runs faster than those 3 but managers have been led to believe that Perl has a slower time to market, thus is slipping from the mainstream. The closer you get to stripped down, just what you need, compiled language, the faster and less system resources the code will take to execute.

      The big issue with language decisions these days is that they tend to be driven by perceived market value. People are the most expensive cost to most businesses these days. So the marketing battle between languages focuses less on performance and more on how experienced and expensive your developers need to be. What I see being missed with this marketing is that by lowering the people quality and marginalizing your language and code quality, you are setting yourself up for maintenance, improvement, and performance costs down the road.

      Never trust the assertions of people who say "undoubtedly", "obviously", "just plain common sense", etc.

      A lot of "undoubtable" things are wrong. Especially when they're based on how things "should" be. Doubt. Measure.

    • by bws111 (1216812)

      The closer you get to stripped down, just what you need, compiled language, the faster and less system resources the code will take to execute.

      Which is exactly why modern, JIT-compiled Java can, and often does, outperform C.

  • So now the polls are allowed to have comments, they are for data mining only, not discussion.

    Why?

    What is the reason we can't talk about the subject of the polls anymore?

  • by Anonymous Coward

    Java runs in a virtual machine, and is compiled "just in time".

    For "real time" applications, that need guarenteed latency, this is a no no.

    Java's garbage collection, running is also a no no.

    Real time applications need hard deadlines, that are deterministic. A function must always return X mSec, regardless of any other things like GC, etc.

    Just imagine if say a pacemaker ran Java.

    "I detected an unusual event, but I have to run GC first, then process the event..oh, what the patient is dead?"

    Why don't we have f

  • Step 1) Don't.
    Step 2) See step 1.

    C developers often get accused of trying to use their particular hammer to drive in every screw they see. This very much counts as the other side of that equation.
  • Java vs. C/C++ (Score:3, Informative)

    by Anonymous Coward on Friday August 02, 2013 @10:42AM (#44456285)

    For simple applications that don't deal with a lot of dynamic memory, Java is almost as fast as C/C++. Unfortunately, trading systems are not such - I know because I worked in that domain for some time, in the 1980's and again in the 2000's. Java's biggest issue is garbage collection - it is decidedly non-deterministic. IE, it can have SERIOUS impacts upon performance, and at times that cannot be pre-determined. In my experience (30+ years), if you need consistency and speed, then you do not select Java for your environment - and I do a LOT of Java development. I just don't use it when I need the software to have a small footprint, be fast, and stay out of the way of other system processes.

    • In my experience (30+ years), if you need consistency and speed, then you do not select Java for your environment

      You have 30+ years of Java experience? Wow. BTW, if you do have a lot of Java development under your belt, how does Zing compare to other solutions in your application domain? Still inconsistent and slow? (I mean, the GC implementations started converging to what can be described as "good performance" only in the last few years, so unless you focus solely on the last few years, of course your experience is going to be horrible on the average.)

  • by sunking2 (521698) on Friday August 02, 2013 @10:52AM (#44456371)
    Low latency is not measured in the millisecond, but in the microsecond. I'm so tired of hearing about how special the HFT think they are.
  • by WOOFYGOOFY (1334993) on Friday August 02, 2013 @10:57AM (#44456425)

    Uh since around 1.3 the JIT optimization for java has led to blindly fast code containing optimizations which are not even available to C++ . Dynamic compiling allows for branch prediction to be more accurately, unlike malloc, the GC knows where to look for free memory and returns it from the last bit of memory you just requested, if you know where pointers are pointing at compile time, you can put them in registers. C++ and other statically compile languages don't have this information, so it stores them in cache, but the JIT can acquire this information and store it in a register. It's the difference between a register to register test and reading from disk.

    There are tons of other stuff like this. I don't have it committed to memory, and compiler technology is not my thing but if you look around you'll see that actually GC and JIT are theoretical advantages in terms of speed and those advantages are being realized. It can even figure out what chip it's being run on at runtime and optimize the code for that chip.

    The Java is Slow Meme is left over from 1995 before there was even HotSpot.

    Not bashing any other language here. C# could also avail itself of these advantages.

    • unlike malloc, the GC knows where to look for free memory and returns it from the last bit of memory you just requested,

      Ah, the old memory allocation canard. Java can allocate from the heap faster than C++. However C++ can allocate from the stack for free which java cannot do nearly as readily.

    • Java continued to carry the stigma of being slow, and rightfully so, because it was horrendously slow for desktop use until Project Mustang (Java 6). Java 6 marked a point where Java not only no longer sucked for desktop application use, but actually became quite good at it.

      Printing went from completely, unusably slow to blazingly fast.
      GUI components went from being painfully slow and difficult to program to some of the fastest and most flexible GUI components I've used on any platform.

      Java was considered

    • by phantomfive (622387) on Friday August 02, 2013 @12:21PM (#44457411) Journal
      All that is nice theory, but in practice C still runs faster, for various reasons. The problem you have is that you're only looking at facts that support your position. That is the problem with the world.

      Consider that every single array access in Java is checked, for starters.
  • by LostMyBeaver (1226054) on Friday August 02, 2013 @10:58AM (#44456437)
    Leaving aside my political beliefs that high speed trading needs to be banned, let me say this.

    Switching to Java should yield similar results to C++. What matters is whether the programmer understands the memory architecture of the run-time environment well enough to not have issues. Generally, you'll find that even the best programmers in either language will overlook things like garbage collectors and memory fragmentation issues. It's a time-to-market thing. When working with large dynamic data sets, it doesn't matter if you're using Java or C++, the developer needs to be able to adapt their code to perform well on the system.

    Code written without considering the processing time of memory management will probably work much better in C++ than Java. That said for huge data sets, Java could perform better since the memory itself is location independent and it is highly probable that you're gain a great deal of performance from being able to defragment memory. Consider however that the garbage collector and the defragmenter will have unpredictable times which can cause multi-millisecond hiccoughs during processing.

    I recommend if you take this route, you hire a compiler geek to work on staff optimizing the memory operations.
  • First I am not saying that Java has no place.

    To me this is not a case of Java being fast enough as good enough. Keep in mind these are people who are building their own microwave towers from one exchange to another to shave microseconds. But also keep in mind that this is is also the era of big data. So you now have a situation where you have to process insanely large amounts of data in near as is possible to real time in order to make trades "Now!"

    So if I write a test program in Java that connects to a
    • by bws111 (1216812)

      You lost all credibility as soon as you used Eclipse as an example. I use Eclipse almost every day. It does exhibit some of the behaviors you say. BUT, and this is a very large BUT, Eclipse is a desktop application, running on whatever you happen to be running, with absolutely NO tuning of the JVM. Applications like HFT are NOT running on some random desktop, they are running in servers with sufficient resources and, more importantly, proper tuning of the JVM to meet their needs.

      There is no reason Java

      • There is no reason Java can not run just as fast and predictably as C/C++, given people who know what they are doing.

        Yes, yes there is. Well, more than one actually. Here we go again:

        (1) Unbounded stop the world of the JVM causing thread stalls (GC being the canonical example)
        (2) Other stop the world's in the JVM due to code profiling causing a newly compiled version to be swapped in (maybe they've switched to atomics here, I'm not sure)
        (3) Java's internal profiling and housekeeping absorbs CPU cycles
        (4) Everything is allocated on the heap. If you have an array of objects - it's an array of pointers to things in the heap.

  • by johndoe42 (179131) on Friday August 02, 2013 @11:09AM (#44456563)

    I'm not a Java user, so I've never directly tuned for things like GC, nor do I interact with it directly. Warm up is a different story.

    I interact with quite a few exchanges (over all kinds of protocols). Most are, unsurprisingly, written in Java. Almost all of them perform terribly at the beginning of the week. The issue is a standard one: the JVM hasn't JITted important code paths, and it won't until several thousand requests come in. For a standard throughput-oriented program, this doesn't matter -- the total time wasted running interpreted code is small. For a low-latency network service, it's a different story entirely: all of this wasted time happens exactly when it matters.

    The standard fix seems to be to write apps to exercise their own critical paths at startup. This is *hard* when dealing with front-end code on the edge of the system you control. Even when it's easy, it's still something you have to do in Java that is entirely irrelevant in compiled languages.

    If JVMs some day allow you to export a profile of what code is hot and re-import it at startup, this problem may be solved. Until then, low-latency programmers should weigh the faster development time of Java with the time spent trying to solve annoying problems like warm-up.

  • I built a real-time interface to some hardware in Smalltalk 20-years ago! (Yikes!) Since Java and Smalltalk have the same GC theory (at least, simplistically), some of my experiences would translate. I also spent 5-years running high-performance, low(ish) latency stuff in Java too. The simple rules of low-latency Java (or Smalltalk) are: - Create less garbage. Don't tell me that the GC is an issue when you are creating huge garbage. If your application is running in a tight loop (small amount of code

"Tell the truth and run." -- Yugoslav proverb

Working...