How and Why Wall Street Programmers Earn Top Salaries 791
msmoriarty writes "Given the level of interest in the recent highest-paid programmers discussion, our reporter decided to do a follow-up looking into the languages and skills needed to work on high-frequency trading systems. There's actually a pretty wide range of languages/tools used, but Linux is the 'default' OS and, not surprisingly, the 'ability to work under pressure when the traders are screaming at you' is a must-have skill."
Stress (Score:5, Interesting)
I used to co-locate in the same building as the local stock exchange. One day, very late, I took the elevator down to the car park, which was where the computers were. There was a guy in the elevator who looked absolutely wrecked. He was sweaty, shaky and not taking things in. He got off at my level and stumbled off towards a porsche which appeared to be similarly young and in equally bad condition.
The thing is, I work in air traffic control, where the stakes are even higher. The difference is that the operational people have an absolutely obsessive approach to managing their workforce. Traffic controllers are just not allowed to get upset or stressed. In many environments they have unlimited sick leave.
The difference, I suppose, is that traders personally stand to take home a lot of money. You could do this in any field: offer ridiculous compensation for ridiculous effort. But if you work it out, I doubt the long term returns justify what this does to people.
I am an HFT programmer (Score:5, Interesting)
I'll be posting anonymously, but I think many here have a very poor understanding of what we do. Most of that is because we do tend to be a very secretive group, but if you were to sit down with some of us, you would see that we really do very normal (and useful!) things in the market.
I work on the algo and core infrastructure. I wrote price feeds that take 1/5th of a microsecond in C++ and (a little slower) in Java. I understand in fine detail how cache and the the PCI-e bus works. I have a very good understanding of algorithms and the constant-time tradeoffs. I know when to make something simple, when to use and avoid threads, and I can debug in minutes and push out a new version in the seconds before market open (not many people can handle that level of stress well). I read the C++ and Hotspot assembly, and know how to program for superscaler architectures specifically. If you really need me to, I can even crank out some VHDL code.
On top of that, I understand market microstructure and derivative pricing. I work 12 hours a day on average and do 100 hour weeks. I am on call during Asian hours and need to come in sometimes on holidays when other markets are going nuts and we need to plan.
I also hope to make $500,000 this year.
You always hear about Google programmers being the best in the industry, but I've been to a couple Google interviews and turned them down both times because the engineering quality just isn't there. I'd put the average HFT programmer up against the best in Google anyday.
Ask away, and I'll answer to the best of my ability.
Re:Traders (Score:5, Interesting)
It's not even luck, either. It's down to your ability to hob-nob with other psychopaths. My friend's brother-in-law was a trader five years back (pre-recession) and lost something like $800 million (of other people's money) in bad speculations. He was fired for it, but hired into a new firm thanks to his "connections" a few weeks later, at a higher salary than his previous job. Oh, and of course he kept the bonuses he made at the first firm before his investment went bust.
Wall street is the world's greatest argument against the notion that capitalism rewards people in proportion to their skill and hard work.
Re:I am an HFT programmer (Score:4, Interesting)
I also hope to make $500,000 this year
That's all? Demand more, if your skills are what you say. You should be pulling in $2M/yr minimum.
Re:Traders (Score:5, Interesting)
"Capitalism", in general, refers to making capital investments in goods that in turn will make a profit. Often, that means indirectly and over time: you spend $1M to buy a machine that makes thousands of widgets a day, and sell the widgets for $5.00 each.
However, with the current obsession with turnover and a quick dollar, what you end up with on Wall Street is really much more pure money speculation, which doesn't have much in the way of capitalist underpinnings. In fact it is a lot more like gambling, which has been around a lot longer than capitalism has.
We simply haven't been seeing the long-term investment in capital goods, which in turn power a robust economy, and at one time made the US the greatest economy in the world.
Until we see a return to something resembling real capitalism (and it still does happen, just not so much on Wall Street anymore), we will continue to have economic problems. A corollary to that is something I have been saying for a long time: Washington needs to stop concentrating on viewing Wall Street as some kind of driver for the economy. That's a false economy. Reasoning: anything that can lose a large percentage of itself overnight is not a stable thing on which to base a real economy. Keynesian "created" fiat dollars can disappear just as fast as they are created, as we saw quite clearly in 2008.
And for similar reasons, I believe HFT should be outlawed. It isn't even a real market anymore; it's simply a contest to see who can spot a discrepancy in perceived value the fastest.
Re:I am an HFT programmer (Score:4, Interesting)
Re:Perversion of Capitalism (Score:5, Interesting)
This guy nails it - http://boards.straightdope.com/sdmb/archive/index.php/t-601887.html [straightdope.com]
"It's not about acting on market information. It is purely arbitrage. A mis pricing allows one to buy and sell simultaneously and lock in the difference minus trading costs.
In the old days, traders used to do this in the trading pit. Now it's computers closest to the exchange feed.
Tied in with this is the automatic trading. In the case of that big intra day fall, a wrong trade was entered. I forget the details but it was big enough to push down the market xx amount, which triggered automatic sell programs from non-arbitrage automated computer selling, which triggered a market sell off, which in turn triggered more selling until the market circuit breakers kicked in.
During the mandatory no trading period, the original bad trade was discovered and reversed. This IIRC also triggered automated buy programs and the whole thing went in reverse. The bad thing is that the market whipsaw really hammered some real end investor trades as collateral damage.
I remember watching the Hang Seng Index the day that Soc Gen announced Jerome Kerviel's fraud and liquidated the positions. It was a full trading day of massive market swings for big losses to big gains several times throughout the day. It was almost all computer generated programmatic trading."
Re:Traders (Score:2, Interesting)
When people talk about 'return to a capitalistic base'. They are talking about the sort of thing where 500 people build things on assembly lines making things. That sort of thing only exists in very low wage countries these days. In higher wage countries those jobs are automated.
I was watching a 'how its made' show. They showed a factory that was picked up 100% brick by brick and moved to another country from the US. Why? The factory was build for building things using humans. The labor here costs too much. It is that simple. You can buy 2-3 workers in another country for the same as here. Most of the shows they have 20 people working in a small factory that pumps out 100k in items a day. Most of the things they show are nearly automated are so close its not even funny.
Wall street is not a real economy but they have *MAJOR* effects on our economy. To minimize them is not right either. But the fed is made up from people that come from there. So they do what they know...
Also the dollars disappearing was a result of bad loans unfolding. It was also an effect of 15 years of unwinding of the 1930s laws that protected the rest of us from those silly antics. In the name of 'simpler paper work'. It was an effect of low hold rates placed upon the banks to create the ability of banks to loan money on the cheap. In many ways it was an echo of the 2000 crash. Which was an echo of the 1992 crash. Which were a direct result of the 1986 crash and wanting to 'free up cash to drive the economy'. They unhinged the very things reigning in wall street. To keep wall street from creating a 1930s crash. Which once those reigns were unshackled in 1997 they did. The 2008 crash is almost exactly the same as the 1930s one.
HFT is just a game to see who can screw each other faster. Making millions off the backs of others all in the name of 'liquidity'. They could put and end to it by doing 3 things. Making people take delivery of a % of commodity goods they buy. Also by limiting the number of times per day you can put in an order for a good say start with 3. Creating a minimum owned time. You buy something you must keep it for at least 1 day.
HFT is creating demand that doesnt really exist and in effect creating liquidity that doesnt exist. As my econ teacher said on the first day talking about government regulation or lack of it. You can cheat the market but eventually the market fixes itself. 2008 was a major correction. There is another bubble building in the commodities market. Whoever gets caught on the other end of that one is really going to get their shorts burnt.
This 'false' liquidity pushes prices up and creates cash that does not exist. It is why you are seeing 100+ oil, and 1600oz gold. Never mind the printing spree the gov went on. Plus the 'possibility' of default. The last 2 we can not really gauge the effects because of the first one. I read in the WSJ of 1 guy controlling 75-90% of the gold market 2 years ago. With only 1% of the cash needed to do so. Something is very wrong there...
Re:Traders (Score:5, Interesting)
"I agree HFT is rubbish, but where do you see justification for long-term investment in capital goods in the US? It seems that industry is being moved to where it can be operated more cheaply - there is no economic justification for more investment here."
Industry WAS being moved to where it can operate more cheaply. And there is plenty of justification. If you're American.
First, let me say this and get it out of the way: China's economy has been slowing. It's not exactly something they advertise, but it's true nevertheless. Further (and I won't claim this is directly related): there is a new Chinese "consumer class" that scarcely existed before, and they are hankering for goods that are not made at home. But having said that, I'm not going to belabor it, and instead I will concentrate on the U.S.
We have learned some things over the last couple of decades. One of the things we have learned is that regardless of whether it results in cheaper goods, outsourcing is bad for the economy. We know for a fact (it is not a guess or just a theory anymore) that it costs jobs at home, and those jobs are more valuable to our economy than the cheaper goods are.
Further, outsourcing is not "fair trade", because it artificially bypasses exchange rates. It is easy enough to say that "U.S. labor can't compete with foreign labor for the same amount of pay". Which is a true statement... but it's only half the story. It's a lie by omission. Because we aren't really competing on an equal basis. Let me construct a simplified example to illustrate the point (numbers just pulled out of the air to make the point):
Company X wants to hire people to manufacture widgets. It can hire unskilled Americans for $10 per hour, or Crotobaltoslavonian workers for $2 per hour. But here's the thing: In America, that $10 buys about 5 pounds of rice. In Crotobaltoslavonia, that $2 buys 10 pounds of rice. So in actual purchasing power, you're paying that Crotobaltoslavonian the equivalent of $20 per hour in America. Americans literally can't compete for that amount because it doesn't pay them enough to eat... in THEIR economy.
And that's the whole point. Economies are different. That's just the reality of the situation. And that's why we have exchange rates... to keep trading FAIR between countries. But hiring labor in a foreign country, in effect, bypasses that exchange rate, and so it is not a "fair trade". Americans are hurt in the process... even if rich Company X gets to keep a few more dollars. We are exporting even more dollars than what they keep as a result... and that doesn't help our economy, either. So it hurts in several different ways, not just one.
Savvy companies actually realize this, but they just don't care. Which is pretty unethical, if you ask me. They are willing to sacrifice your $10 so they can make an extra $1 in eventual profits. And you should be angry about that.
Plain and simple: we need to bring the manufacturing jobs back home. Partly because of the direct monetary imbalance that outsourcing causes, and partly because we need to keep that capital investment (in machines and buildings and other capital goods that can actually MAKE things) here at home, rather than letting it be done overseas at our expense.
"In fact there's a surplus of "capital," at lest in the sense of invested money seeking good returns. ... So we keep getting bubbles due to over-investment, first in .com, then housing... what is the "next big thing" where we could invest to bring real growth?"
The bubbles aren't due to a surplus of capital, per se. I mean they are, but the proximate cause of that surplus is indisputably government monetary policy. They have been keeping interest rates artificially low to "stimulate" the economy (BEFORE the 2001 and 2008 bubble burstings). Easy money means more capital. But it wasn't invested in capital goods, intended to
Re:Perversion of Capitalism (Score:5, Interesting)
This is from the Wikipedia article http://en.wikipedia.org/wiki/Definitions_of_fascism [wikipedia.org]: Umberto Eco defines fascism with the following features:
Re:Perversion of Capitalism (Score:5, Interesting)
Look, what am I supposed to do.
I'm a chip designer. I used to work on beautiful hardware, you might be using some of my products to play games even today. Turns out, ASIC design is screwed as a business in so many ways, and it's not because of outsourcing. People just don't want the hardware all that much, and they don't pay a lot for it. The chips are priced by cost of manufacturing, not cost of R&D, and so R&D is basically cut to the bone on salaries.
Outside of financial companies (and FPGA vendors, who have been trying to sell this idea unsuccessfully to everyone on the planet for the last 10 years), nobody seems to have recognized that bringing stuff into hardware (FPGAs, for instance) can make money through performance. So that's where I work.