How and Why Wall Street Programmers Earn Top Salaries 791
msmoriarty writes "Given the level of interest in the recent highest-paid programmers discussion, our reporter decided to do a follow-up looking into the languages and skills needed to work on high-frequency trading systems. There's actually a pretty wide range of languages/tools used, but Linux is the 'default' OS and, not surprisingly, the 'ability to work under pressure when the traders are screaming at you' is a must-have skill."
Perversion of Capitalism (Score:5, Insightful)
"In high-frequency trading (HFT), programmers eke out every last incremental tick in performance to build algorithms that battle other algorithms for computational supremacy and millions in profits -- and earn a lot in the process."
Skimming money off billions of micro-transactions. Ahh, yes... forget investing in ideas and backing well managed companies... this is the way capitalism was envisioned.
Re:Perversion of Capitalism (Score:2, Insightful)
Oh, admit it; this has been every computer programmer's dream since they saw Superman III as a child.
Traders (Score:5, Insightful)
"the 'ability to work under pressure when the traders are screaming at you' is a must-have skill.""
Yeah, most of the 'traders' I've met were complete assholes, probably because they realised their high salaries were more down to luck than skill.
Fortunately when I screw up air traffic controllers just have to start rerouting airliners with hundreds of people on board so they don't crash into each other, so I clearly couldn't justify being paid a Wall Street salary. I can't imagine the stress level of having a 'trader' screaming at me.
yeeeeeah (Score:5, Insightful)
I'm past the whole "a-holes can yell at me if they pay me enough" phase.
from an HFT developer's view (Score:2, Insightful)
I'm a developer at an HFT firm and can say that it is some of the most interesting work I've done in my career. You need to wear many hats for the job.
On a daily basis I multitask between making low-level kernel modifications to reduce system latencies, to refactoring our high level marketdata->prediction->execution engine. It's a never ending balance/conflict between making things as fast as possible while trying to maintain good design principles and not take shortcuts in the sole interest of efficiency.
To those that say HFT is an anathema to the economy, I think there is some merit, but there are two sides to every story. I've definitely met some other HFT shops that focus on some really shady and morally questionably tactics (essentially DDOSing markets to gain an advantage). However, the amazing advancement in network technology and all these crazy RDMA and 10G user-space network stacks is literally due to the HFT community, and I'm already seeing these new technologies start to penetrate other markets. This is capitalism at work.
Also, believe it or not, HFT has reduced spreads due to competition, which means you pay less when you want to buy or sell a stock RightNow instead of using a limit order (and while spread reduction is good for investors, it sucks for us because it is not as profitable as it used to be a few years ago).
The "Billions" in profits going to HFT firms right now previously went to the big investment banks and the old classic broker taking orders over the phone and charging a ridiculous market spread.
Re:I am an HFT programmer (Score:4, Insightful)
enough lies please (Score:5, Insightful)
we all understand what 'arbitrage' is. when the synthetic CDO market calls their deals 'arbitrage' we all know its fucking bullshit.
when the sales guys in the brochures talked about the 'AAA' ratings on these pieces of 'arbitrage', it was all bullshit.
when Lloyd Blankfein calls it 'hedging, not betting', its fucking bullshit. ]
there is absolutely nothing, whatsoever, 'valuable' behind a credit default swap. it is a bet. that is a fact, and its not rocket science, and its not a conspiracy theory, and its not "the ignorant and alarmist" decrying some nefarious boogey man. its the basic fucking fact of what fucking happened.
I beg of you. stop lying. nobody believes you anymore. this is like the scene in Shattered Glass when Peter Saarsgard has to finally explain to Hayden Christiansen that the whole charade has ended.
the financial industry has no clothes. we all know it. there is no point in pretending.
Re:I am an HFT programmer (Score:3, Insightful)
how much cocaine do traders use? (Score:2, Insightful)
and how many prostitutes do they kill, on average, per year?
Re:Traders (Score:5, Insightful)
Re:Perversion of Capitalism (Score:5, Insightful)
We pay our taxes to these guys as much as we pay them to the government.
Re:Perversion of Capitalism (Score:5, Insightful)
"Perversion of Capitalism"
No it;s exactly what you get in the real world where market values (profit) drive everything. We see it all the time in offshored jobs, destroyed lives, rolling back of the welfare state, the election of extreme right wing conservatives like Stephen harper.
This "there is some pure capitalism we have to get back to" bullshit is just that - bullshit. The left was born from captialism killing workers, it caused two world wars and then then there was the cold war. This idea that is some 'benign' capitalism we have to get back to is just utter american ignorance.
Re:I am an HFT programmer (Score:5, Insightful)
This, from the people in charge of trading amazingly large amounts of money in a market which influences the global economy in a big way. (remember that billion vs. million mix-up awhile back that caused some pretty big problems until it was fixed)
You call it risky, I call it reckless. You try what you're doing in any other field and you'd be fired pretty damn quick.
That said, I'm not so much angry at you as I am at the people who ask you to do this.
Re:I am an HFT programmer (Score:4, Insightful)
Yet another HFT programmer here. Let me address your concern about the risk to the market. HFT systems have tons of controls which govern our ability to trade. We have what we call 'doors', which act like circuit breakers. When a door is closed, it prevents any further trading. Doors can be triggered by any number of conditions (abnormally high P&L, too much exposure, etc.), and there are strict compliance policies governing how and when they can be reopened. We also have throttle-style controls ("speed bumps") which effectively limit how many trades we can make in a given time window. These controls can be configured per security, per name, per exchange, or globally. They are extremely robust, and we maintain them meticulously.
We take these matters very, very seriously. We are at the mercy of the exchanges: if we screw up and don't contain the damage, they will revoke our ability to trade, effectively shutting us down.
Wall Street (Score:5, Insightful)
I worked for a Fortune 100 financial company in Manhattan. Some of the nastiest people I ever worked with were there. I also never worked anywhere where the company made more clear to me that I was a disposable cog. Not that that isn't the case elsewhere, but management usually tries to at least conceal it. I worked 60 hours a week, including weekends, and I was on the low end compared to others who were working to move up in the hierarchy.
There was a limited and fixed bonus pool, so if you got less money, others got more. I'm sure this was a plan by management - the firm was so wealthy, they felt it better their workers be divided against one another than working together. They did this in a variety of ways - staff versus contractors, contractor firm versus rival contractor firm and so forth. This encouraged people trying to rip one another apart during weekly meetings, code reviews and the like.
The office politics can be strange too. I used to be on conference calls with a programmer with a huge ego, and who people deferred to because the program suite he wrote was important for the firm. The program was shoddy though, it had massive memory leaks and the like. But he and the team under him were able to throw it together quickly and the business people were happy with him so he was a golden boy. It wouldn't have been so bad if he wasn't always denigrating everyone else's work with a holier-than-thou attitude. I wasn't in a position to say anything though. Lots of stuff like this happens. A lot.
It was not all bad. It was a large environment. Stuff I would have done maybe once a year at a smaller company I was doing every day there. I was surrounded by dozens of people who were sharp and knew what they were doing. There was a feeling of camaraderie among some of us. It is hard to explain the change in quality due to the sheer size of the company, with its ability to spend massively if needed.
One thing I will say - unless you are there to work 24/7/365 and try to make it to be one of these "highest paid programmers", there is no reason to be there. Some of my co-workers joined straight of college, which seemed dumb to me. Anyone who takes a job for less than six figures is a fool - if you don't have the skills to make at least $100k, there is no reason to work there. It is not the place to come in at a low level and hang around.
To repeat a point - anyone who takes a job on Wall Street right out of college is a fool. They recruit heavily on campuses for the same reason Microsoft and Intel and Electronic Arts do: so they can take advantage of suckers with no work experience who don't know any better. Thankfully I didn't make this mistake. I was able to put everything I saw there into perspective. Going from college straight to Wall Street as a programmer is a dumb move for most people.
Re:I am an HFT programmer (Score:5, Insightful)
Just because you're good at what you do doesn't mean that what you do is good.
Microseconds? Why not nanoseconds? How about femtoseconds? Why wouldn't that make just as much sense?
What could possibly change in the underlying value of a corporation made up of flesh and blood humans and capital with decades of depreciation in a fucking microsecond? Here's a hint: nothing. You are not investing, or trading, but simply racing other gamblers. Investing doesn't benefit from microsecond response times, and trading doesn't need it either. People could buy IPO shares just fine over the phone. Nobody ever needed a microsecond response time to buy a thousand bushels of wheat, and never will, because bread is baked daily, not a million times a fucking second.
If politicians had two braincells to rub together, they'd enact a law to prevent trades faster than some tick, say, an hour. Your 'trading' company would go out of business in a week, and nobody would care. Farmers would still sell their wheat, and bakers would still buy it, but without you leeches skimming off the top.
Re:Traders (Score:4, Insightful)
Q: How can we bring jobs back home?
A: By directly -- and heavily -- taxing companies that outsource labor and manufacturing. This avoids the pitfalls of tariffs: we would not be blocking trade from coming into or going out of the country. However, we would be demanding compensation for our economy, from the companies that have been so harming it by their practices. That's about as fair as it gets.
(It's also not a subsidy by the government... so it avoids those problems, too. Instead, it is a source of revenue TO the government.)
Re:enough lies please (Score:5, Insightful)
there's a strong moral risk: if you can both buy a company's debt, and insurance in case they don't repay it, why not keep doing that endlessly ? you make money both ways, nobody is really interested in the underlying debt quality, and bankers make fat commissions on both sides of every deal.
oh, wait...
Re:Perversion of Capitalism (Score:5, Insightful)
Re:I am an HFT programmer (Score:5, Insightful)
I am most likely better than you at each and every aspect of software (and HDL) development you have mentioned. Except, of course, "debugging in minutes" -- that kind of irresponsibility would get me fired. I also have to work long hours, and have to have clear understanding of complex issues unrelated to software.
Except I do embedded software and FPGA development for professional audio equipment. Each device I worked on, each firmware release, each line of code, does something useful for many, many people. Some of those people don't even know that audio equipment, leave alone software, is involved with what they are hearing. Large fraction of my work ends up being free/open source, too -- platform, drivers, etc.
I also don't have any problems with posting here under my real name. Or with telling you, and people like you, to die in a fire.
Re:Perversion of Capitalism (Score:5, Insightful)
Patience, my friend. Soon we'll pay more of our taxes for these guys. It's all a matter of time and things will get sorted out.
Then we'll finally have the small government everyone dreams of. Because there won't be any money for anything but a small government left after the locusts had their share.
Re:Liquidity (Score:4, Insightful)
Liquidity is a commodity of diminishing returns. If I put in a sell bid on stocks and you say it will take one month, the team than can sell in a day instead is indeed providing a valuable service. And going from taking a day to trade down to an hour or say five minutes is quite valuable too (though not nearly as much so). But the moment your liquidity is faster than my ability to be informed about it, additional liquidity has ZERO value. It takes me several seconds to click sell on a website, and watch as the site refreshes to inform me the transaction has occurred.
In the 1990s, these people were creating value. Today, they are exclusively leeching money from the rest of us.
Re:I am an HFT programmer (Score:5, Insightful)
Do you ever buy or sell stock? Perhaps indirectly, through a mutual fund or 401k type plan? If so, then you benefit from high liquidity in the market. HFT and other Wall Street shenanigans do skim from the top, but they also provide liquidity. It's almost certain that the liquidity benefits small market players more than the skimming hurts them. In other words, the money they're skimming comes from the banks and brokers rather than you and me.
For example, take the stock of Red Hat (http://finance.yahoo.com/q?s=RHT). Yahoo Finance right now shows that, as of the last time the market was open, I could buy 100 shares for $42.56 (that's the "ask" or best current asking price), or I could sell 300 shares for $42.09 (the best available "bid"). That's a bid-ask spread of about 50 cents. That spread is a hidden cost to either buying or selling stock: If you buy and then sell RHT, you will have paid about 50 cents per share just for the privilege, even if nothing in particular happens to the company. Let's split that 50-50 and say that every stock transaction in RHT (buy or sell) costs you 25 cents per share in implied fees.
Those bids and asks are set by individuals and companies who are competing. They want to get a good deal for either buying or selling the stock, but they also know that if they set asking price too high or their bid too low, they'll never make any trades. The more competition there is, the tighter the bid-ask spread will be. HFT and other algorithmic approaches allow firms to set prices on tons of stocks without requiring human attention for each one, which dramatically increases the competition and thus tightens the bid-ask spread.
In this example, if you outlaw HFT and similar trading strategies, maybe RHT will have a spread of $1 intead of 50 cents. Maybe you'll be happy that HFTers aren't making ther 5 cent skim off the top anymore, but it'll be cold comfort when you're paying 25 cents more on each transaction and it's just going to a different Wall Street firm.
If you think I'm exaggerating the effect of computerized trading of the spread, have a look at slide 8 (page 4) of this study: http://fisher.osu.edu/~diether_1/b822/trading_costs_2up.pdf [osu.edu]. Starting in 1960, the average bid-ask spread has ben dropping steadily every decade to a small fraction of what it used to be.
Background: I am an actuary trained in quantitative finance. I've never worked in Wall Street or done any HFT or other algo trading.
Re:I am an HFT programmer (Score:0, Insightful)
I am most likely better than you at each and every aspect of software (and HDL) development you have mentioned. Except, of course, "debugging in minutes" -- that kind of irresponsibility would get me fired.
Sounds like you are much better at being a dick
Re:I am an HFT programmer (Score:4, Insightful)
Losing money is also defined as not making as much as possible. If you were making $1000/minute on Monday doing something and only $100/minute on Friday (many bots only run for the first 5 to 10 minutes of the market); then that is also defined as a loss of $900/minute.
I find that statement quite disgusting, and also indicative of why we're in the trouble we are as a country.