Forgot your password?
typodupeerror
Businesses Bug Programming The Almighty Buck

Wall Street and the Mismanagement of Software 267

Posted by Soulskill
from the of-barn-doors-and-horses dept.
CowboyRobot writes "Last week, a bug in high-frequency trading software from Knight Capital Group resulted in erroneous trades costing almost a half-billion dollars. So, what went wrong and how can they, or any other software developer, prevent something similar from happening again? In hindsight, it's clear that the developers did not verify the code under enough conditions. But the real issue is how these high-frequency trades work in the first place. Robert Dewar at Dr. Dobb's suggests the financial industry needs to take a page from the avionics rulebook, which has very strict guidelines about what code can be implemented due to the high cost of failure in that field. 'High-frequency automated trading is not avionics flight control, but the aviation industry has demonstrated that safe, reliable real-time software is possible, practical, and necessary. It requires appropriate development technology and processes as well as a culture that thinks in terms of safety (or reliability) first. That is the real lesson to be learned from last week's incident. It doesn't come for free, but it certainly costs less than $440M.'"
This discussion has been archived. No new comments can be posted.

Wall Street and the Mismanagement of Software

Comments Filter:
  • That's What We Did (Score:3, Insightful)

    by crrkrieger (160555) on Friday August 10, 2012 @08:51AM (#40944573)

    Back in the late 90s when I was system admin for a trading company, they recruited me from a place that did 911 computer aided dispatch software. My shop, at least, recognized that some of the same reliability issues were at stake, so some people get it.

  • by Anonymous Coward on Friday August 10, 2012 @08:52AM (#40944589)

    First 100 trades in a day: free
    Next 1000, taxed at 0.02%
    Next 1000, taxed at 0.1%

    And so on.

    This would do wonders for the problem.

  • by knapper_tech (813569) on Friday August 10, 2012 @08:59AM (#40944653)
    It's already such a waste that so much talent is getting thrown at problems that seek to make money while producing absolutely nothing. HFT is cleverly sanding in the middle of a river in an eddy and dipping your hand in to tap power without getting pushed downstream. What does Wall Street actually produce? What is their product? Why should we care that they periodically lose their minds and shirts? If anything HFT should be taxed into oblivion so that excellent minds aren't recruited to deliver nothing of social value.
  • Warning to execs (Score:4, Insightful)

    by jerpyro (926071) on Friday August 10, 2012 @09:02AM (#40944679)

    Maybe it should serve as a warning to executives to not release buggy software. I know a lot of shops that push things out the door before they're fully baked.

    In terms of the stock market, I don't see a problem. The long-term market wasn't affected, no value was created or destroyed, and those who played the game improperly lost out big time. Short term trades on the exchange are gambling. Anyone who tells you otherwise just wants your money. Don't forget, there's always a buyer *and* a seller. Just because Knight lost $450m doesn't mean other people didn't gain $450m.

  • Re:Not comparable (Score:5, Insightful)

    by MickyTheIdiot (1032226) on Friday August 10, 2012 @09:04AM (#40944693) Homepage Journal

    Is it me or are we continuously using "profits" as a excuse for bad *anything* and pushing that idea to an extreme?

  • by sycodon (149926) on Friday August 10, 2012 @09:08AM (#40944745)

    Outlaw HFTs They just pervert the market.

  • by dkleinsc (563838) on Friday August 10, 2012 @09:11AM (#40944781) Homepage

    In theory, what Wall St is supposed to produce is investment directed at useful activity - for instance, if making solar panels is useful, and making fake cold fusion devices is not, Wall St is supposed to ensure that the solar panel company gets investment capital to make more solar panels while the cold fusion company does not.

    In practice, this doesn't happen as well as it should because many investors are stupid and believe the hype (e.g. Facebook IPO), and even more try to profit off of other people believing the hype by successfully selling securities for more than they're worth.

  • by Anonymous Coward on Friday August 10, 2012 @09:18AM (#40944845)

    You probably aren't thinking enough about the 'problem area' here. Note that I personally think HFT should be taxed into obscurity as it produces nothing and has the potential to cause real damage. But setting that aside, the whole point of these algorithms is to be HIGH FREQUENCY. Ie: fast. If you can get a quote (order) into the market, one ms faster than your competitor, you can make the profitable trade and your competitor won't. Any checks and balances you put in - will result in slower algorithms. It is a constant balance to squeeze out even a few more nanoseconds of performance versus putting in checks. You can make it perfectly smart and safe - and you will never make any money because its slower than the rest. Put in too few checks, and you risk losing a lot of money when some exceptional circumstance occurs that wasn't covered.

    Clearly Knight erred on the wrong side of the balance here, but within the context of HFT, its not just a matter of being 'too lazy' to write the checks.

    Disclaimer: I worked for a while in a HFT firm.

  • by Anne Thwacks (531696) on Friday August 10, 2012 @09:30AM (#40944961)
    You seem to assume that they want to stop the "accidents". You misunderstand the senario - the whole thing is a distributed Ponzi scheme. It works like a sort of "reverse pass the parcel". Each transaction takes a small percentage in brokerage fees as the underlying "commodity" gains or loses value. However, over time, the brokerage fees exceed the true worth of the commodity has in real life.

    The spectacular "losses" are the path by which the money leaves investors, and is syphoned off to pay the disproportionate brokerage fees.

    As a point of reference see gold - the value of gold traded between speculators each day is 1,000 times the value actually sold into industry/jewelery/etc - so a 0.25% brokerage fee on both buyer an seller is worth 5* the value of the gold that enters or leaves the market each day. Obviously, the additional margin actually paid by real life users for the benefits of a liquid market is a tiny fraction of the value of the gold - sometimes there has to be a "software problem" or "rogue trader" or, to use the colloquial term "scapegoat", to provide the money distributed on brokerage fees.

    I am not suggesting any one person is responsible for this - the banks have colluded to look the other way while this system has developed in an ad-hoc manner. In all probability, many of the so called "masters of the universe" are too stupid to understand what is going on, as it requires an understanding of the laws of maths, which they generally don't have. The few "whizz-kids" who could understand are "highly motivated" to look the other way.

  • by 140Mandak262Jamuna (970587) on Friday August 10, 2012 @09:33AM (#40944981) Journal
    They will sue the unemployed coder for 400 million dollars. Some CXO will certify in good faith he hopes to collect the money. S&P will accept the certificate and rate the credit worthiness of the company AAA. Goldman Sachs will use S&P rating to sell the company to some poor smuck, your 401K mutual fund or my pension fund or our municipalities long term fund at 400 million over the true worth. Everyone involved in the racket will award themselves huge bonus, consultation fee and commissions. That is how 400 million dollars of profits are created, transferred to private individuals and the corresponding 400 million dollar loss for the counter parties are socialized. Either small investors, or government institutionalized investors, or straight forward government bail out. We are always the counterparties who are on the losing end of every such gigantic whoppers.

    America once had a great capitalism. Now we have the system where no matter what risk the rich insiders take, all the profits are theirs and all the losses are ours. A system where the ruling elites are protected from the consequences of their actions, where they can rig the game so that they win no matter what, is how societal collapse begins. Jared Diamond's book "Collapse" discusses specific case studies showing how it collapses. Greenland colonization from Iceland, Pueblo Indians, Easter Island were what he discusses in great detail.

  • by Remus Shepherd (32833) <remus@panix.com> on Friday August 10, 2012 @09:37AM (#40945025) Homepage

    The purpose of avionics is to get a plane from one point to another without incident.

    The purpose of automated stock trading software is to make as much money as possible while screwing the other guy if you get the opportunity.

    You'll never make automated stock software 'safe'. Its purpose is inherently risky and combative. You're not up against the laws of physics and the occasional thunderstorm; you're up against other people who have similar software and an urge to hurt you. This is Wall Street PvP (that's Prick-versus-Prick). It's unsafe by its nature.

    You cannot make competitions entirely 'safe'. What you can do is pen them in so that they do not hurt bystanders. Just like putting crash walls around a NASCAR track, we need to put up regulations around Wall Street so their blood combat does not spill out and harm the larger economy. Re-implementing Glass-Steagall is the least that we can do to keep Wall Street's fiery crashes from hurting the common people. There are probably more reforms we could make to wall them off properly.

  • by jedidiah (1196) on Friday August 10, 2012 @10:30AM (#40945731) Homepage

    As someone that actually stands to lose from Obama's tax policy, I understand the need to do my fair share in tough times. I also freely acknowledge that giving young families and recently graduated students more money will more likely cause money to move around the economy.

    Not everyone with money is a narcissist jackass.

  • by somersault (912633) on Friday August 10, 2012 @10:52AM (#40945983) Homepage Journal

    I'm one of the least business interested type techies you can have, but even I would know that the most important thing when doing stock exchange software is making sure that it's not doing anything retarded, and cutting it off immediately if it does. That's the type of thing that terrifies the hell out of me, and why I wouldn't really want to get involved in financial programming without a lot of supervision. Then again, maybe that means that some of the programmers who go to work for these places are idiots who don't think about consequences..

  • by timeOday (582209) on Friday August 10, 2012 @11:17AM (#40946363)
    I find the comparison to the 911 hotline or avionics (in the summary) almost offensive. This $400,000,000 "loss" is just ownership changing hands in a zero-sum game; nothing was actually destroyed. Even individual investors (who spread their bets) were probably on both the winning and losing sides.

    It's NOT the same as airplane full of people being destroyed or an ambulance failing to show up. In fact, all the money the summary suggests pouring into perfecting HFT software would be a waste and a loss to the economy overall. The real question is how can we fix the incentives to get those HFT developers back working on avionics or 9/11 call centers or something else with real value?

  • by somersault (912633) on Friday August 10, 2012 @11:35AM (#40946651) Homepage Journal

    But the real problem issue here isn't even the buggy software IMO. It was the way the software was put into place, not monitored, and nobody was ready to just shut it off if it started going haywire. According to the article I read, Knight hadn't even noticed a problem themselves - it was pointed out to them by the stock exchange that they were doing a very high amount of trades compared to usual, and it still took them half an hour or more to shut everything down. There should have literally been a big red STOP button in place to shut things down if they went wrong.

  • by iplayfast (166447) on Friday August 10, 2012 @02:43PM (#40949513)

    High-frequency automated trading are destroying the stockmarket. With transactions on a stock happening at thousands of times a second, when things go bad they go bad very quickly. When things go well, the normal trader gets reamed.

    IMHO High-frequency automated trading should be outlawed, stocks should be bought and sold at a rate comparable to human interaction. Say 1 per second. Then if things go bad, it goes bad over the course of a day and people can react. Brokers normally buy and sell bulk amounts of stock, so this would be no different.
    It would level the playing field, and put the normal investor at less of a disadvantage compared to the big companies. If a stock is particularly hot, then some trades won't get made by the end of the day. This is in other words reverting to the stock market of old, where the market could be looked at, and expected to stay the same over the course of a minute.

Receiving a million dollars tax free will make you feel better than being flat broke and having a stomach ache. -- Dolph Sharp, "I'm O.K., You're Not So Hot"

Working...