Slashdot is powered by your submissions, so send in your scoop

 



Forgot your password?
typodupeerror
Bug Transportation Software The Courts

Stack Overflow Could Explain Toyota Vehicles' Unintended Acceleration 664

Posted by timothy
from the go-ahead-ask-your-car-a-question dept.
New submitter robertchin writes "Michael Barr recently testified in the Bookout v. Toyota Motor Corp lawsuit that the likely cause of unintentional acceleration in the Toyota Camry may have been caused by a stack overflow. Due to recursion overwriting critical data past the end of the stack and into the real time operating system memory area, the throttle was left in an open state and the process that controlled the throttle was terminated. How can users protect themselves from sometimes life endangering software bugs?"
This discussion has been archived. No new comments can be posted.

Stack Overflow Could Explain Toyota Vehicles' Unintended Acceleration

Comments Filter:
  • Wow (Score:5, Funny)

    by rsilvergun (571051) on Friday February 21, 2014 @07:01PM (#46307735)
    Is there anything Stackoverflow [stackoverflow.com] can't do?
    • Re:Wow (Score:5, Interesting)

      by snookerdoodle (123851) on Friday February 21, 2014 @07:02PM (#46307745)

      I have to admit, that was my first thought as well. :)

    • Re:Wow (Score:5, Funny)

      by Hsien-Ko (1090623) on Friday February 21, 2014 @09:24PM (#46308681)
      Toyota::MovingForward encountered an unhandled exception
    • Re:Wow (Score:5, Funny)

      by wickerprints (1094741) on Friday February 21, 2014 @09:33PM (#46308731)

      Gives a new meaning to "race condition," doesn't it?

  • by RevWaldo (1186281) on Friday February 21, 2014 @07:02PM (#46307763)
    ..."We'll I'm sure somebody on there could!"

    .
  • Go Amish? (Score:5, Insightful)

    by dbc (135354) on Friday February 21, 2014 @07:03PM (#46307769)

    "How can users protect themselves from sometimes life endangering software bugs?"

    Amish buggies typically don't have software throttle failures. Run-away horses can be an issue.... and actually having to share the road with dipshit drivers who don't understand the number of slow moving vehicles (not just buggies) that there are out in farm country are a real danger.

    Seriously, software has bugs. Mecanical throttle linkages can stick, too. Life has risks.

    • Water's wet.

      Skies are blue.

      Software and cheaper accommodations have bugs.

    • by mjwalshe (1680392) on Friday February 21, 2014 @07:18PM (#46307893)
      typical stack overflow high rated answer - totally ignoring the question at hand
      • Sometimes the question is dumb and asking to do the wrong thing. There isnt a good answer to "how do I avoid software bugs" other than "avoid software", and not the answer that is wanted.

    • Re:Go Amish? (Score:4, Insightful)

      by Anonymous Coward on Friday February 21, 2014 @07:22PM (#46307905)

      Coming from the aerospace industry, you cannot have software that has bugs. And if there was the possibility of a software bug, you have to prove that you can mitigate the effect in hardware. So just to say "software has bugs...life has risks" isn't an acceptable answer (in my opinion). We have to remember this is not an apples to apples comparison. Just because traditional consumer software always has bugs in it (which are acceptable) doesn't mean they are acceptable in other industries. Considering that the failure puts someone's life at risk, I would think it should be considered unacceptable in automotive industry as well.

      • Re:Go Amish? (Score:5, Interesting)

        by CodeArtisan (795142) on Friday February 21, 2014 @07:59PM (#46308135)

        Coming from the aerospace industry, you cannot have software that has bugs. And if there was the possibility of a software bug, you have to prove that you can mitigate the effect in hardware. So just to say "software has bugs...life has risks" isn't an acceptable answer (in my opinion). We have to remember this is not an apples to apples comparison. Just because traditional consumer software always has bugs in it (which are acceptable) doesn't mean they are acceptable in other industries. Considering that the failure puts someone's life at risk, I would think it should be considered unacceptable in automotive industry as well.

        If you want your cars to be as expensive as a 747, then you can attain that goal. I used to work in the automotive industry designing embedded software for engine management systems. At that time, no automotive company would pay more than $100 for the Engine Control Unit. Probably 60% of the code was written to manage failures (both software and hardware), and there were other electronic fail safe mechanisms. But you can't mitigate every possible failure event without introducing costs that would have made the unit orders of magnitude more expensive.

        • I'm not so sure cost is the biggest problem. Once upon a time ABS was an exotic tech used only on aircraft. I was impressed when they became a car option.

          My biggest concern is whether a driverless car can be smart enough to reliably handle all the situations that arise. Probably someday, but I don't know when. Google's over-hyped toys can only handle clear weather, and even then they often have to kick out and go to manual control.

          • Re:Go Amish? (Score:5, Interesting)

            by jrumney (197329) on Friday February 21, 2014 @10:20PM (#46308921) Homepage

            Once upon a time ABS was an exotic tech used only on aircraft. I was impressed when they became a car option.

            In other words, ABS software (and hardware) was very expensive to develop, and only the development budgets of airliners was large enough to cover its development. Once developed, the companies that developed it realized that adapting it for automotive use would be within the budget of luxury car makers, and once it was working there, it became very cheap to adapt for standard cars, as the differences are very minor (in fact it is basically a case of the luxury car makers funding continued improvements, and standard cars getting the previous generation that already has its development paid for).

          • by jnana (519059)
            People can't reliably handle all the driving situations that arise. The actual target for driverless cars should be something more like handling situations that arise at the 95th percentile compared to human beings. When they are as good as the very best human drivers, that should be good enough, although at that point, it will probably not be too much longer until they're at the 99.999th percentile level.
      • Re:Go Amish? (Score:5, Informative)

        by dcw3 (649211) on Friday February 21, 2014 @08:08PM (#46308191) Journal

        The aerospace industry deploys bugs very frequently. Don't pretend like you don't. Yes, for some applications, we test the hell out of it, but bug free, hardly.

      • Re:Go Amish? (Score:5, Insightful)

        by arkhan_jg (618674) on Friday February 21, 2014 @08:15PM (#46308219)

        Even in the aerospace industry, there are software bugs. Very few, yes, because a lot more time and money is spent to track them down. There are mechanical failures too, despite the best engineering efforts. But anything we build has the potential to be flawed somewhere in the process. That's why we still put highly trained pilots in the things, for when something goes wrong - and even then, human error causes unintended faults, sometimes catastrophically.

        If a car cost as much as a jet, and drivers went through as much training as a passenger pilot - and had to have a co-driver at all times - we'd be far safer on the roads.
        After all, the vast majority of car crashes are driver error. And failure modes when you're at 30mph on wheels are a lot better on the whole than when at 30,000 feet. But if we built cars to the same standard, and held drivers to the same standard as aerospace engineering, only the rich could afford to.

        There's a trade off between acceptable risk, and cost. Even though the designs get safer every year, maybe we allow too much risk in drivers and their cars. But absolute flaw free cars? Impossible.

      • by jrumney (197329)

        Yes, aerospace software [wikipedia.org] has solved the problem of software bugs. We should all be following its perfect example. </sarcasm>

      • by Wuhao (471511)

        You're not supposed to, but you do routinely have software that has bugs even in aerospace, because there is no development process that can guarantee the prevention of 100% of defects, nor even guarantee that 100% of defects are detected and corrected.

      • by Mashdar (876825)

        Coming from a fault tolerant computing background, your non-trivial software could have bugs. There is a reason N-version programming [wikipedia.org] exists (and it's not for fun).

        Unless you have true N-version programming with totally separate teams writing on different platforms with different languages, different academic backgrounds, different cultural paradigms, etc, then you don't have perfectly reliable software. (And even then your application may lend itself to certain symantic commonalities.)

        When done properly, r

    • Until someone actually reproduces the bug I'd say the loose floor mat explanation is just as credible.
      • by bobbied (2522392)

        Until someone actually reproduces the bug I'd say the loose floor mat explanation is just as credible.

        I'm actually going for "operator error" but hey...

        Two cases come to mind. First, was short term unintended acceleration. Probable cause for that is hitting the wrong or both peddles.

        Long term acceleration on the highway? Nobody thought to just turn off the ignition switch? Turns off the fuel pump, care stops running, eventually you come to full stop.

        BOTH where operator error..

        • Re:Go Amish? (Score:5, Insightful)

          by amorsen (7485) <benny+slashdot@amorsen.dk> on Friday February 21, 2014 @08:06PM (#46308181)

          Killing the ignition also means killing power steering and power braking. There is a quite widespread belief that it can also engage the steering wheel lock, but AFAIK no one has been able to name a car where that happens so far. The next challenge is that in many modern cars the ignition switch is just a button which is handled in... software. You could throw the key out of the window and wait for the anti-theft device to kill the fuel supply, but that does not seem like something that most people would try.

          In most cars you can put the gear box in neutral. The car will likely have a rev limiter (possibly in software, but it might still work). Worst case the engine breaks, but in almost all cases that would not be fatal to the people in the car.

          However, in almost all cars, when not going down a steep hill, the brakes are actually more powerful than the acceleration. Just do not let off the brakes once you get the car slowed down and you think things are under control -- then the brakes overheat and you have a stuck accelerator combined with no brakes, and that has killed at least one driver already.

    • by FatdogHaiku (978357) on Friday February 21, 2014 @07:35PM (#46307981)
      I have just finished my patent application for the steering wheel mounted Ctrl, Alt, Delete button combination...
      Problem solved!**

      **Some users may experience complete lack of vehicle control while the system is rebooting.
  • by Skinkie (815924) on Friday February 21, 2014 @07:04PM (#46307783) Homepage
    How would a mandatory publication of all code as open source [not suggesting liberal licensing here] work out here? Might converge at a collaborative initiative and will most likely be reviewed by all sort of people.
  • One way would be to insist that automakers do not nickel and dime design vehicles. The critical components related to vehicle safety should be designed for safety first, cost second.

    These vehicles go for over $20 000, I should at least have the option to pay an extra $1000 to chuck the electronic crap.

    • One way would be to insist that automakers do not nickel and dime design vehicles. The critical components related to vehicle safety should be designed for safety first, cost second.

      These vehicles go for over $20 000, I should at least have the option to pay an extra $1000 to chuck the electronic crap.

      The electronics are very deeply embedded. Not sure how you're gonna dump them when there's no physical cable connecting your throttle to your engine. Impossible in the case of EVs or hybrids.

      Also although the article does a decent job of showing that a stack overflow is possible and might result in unexpected behavior, what's needed is a simulated failure scenario to see if that's what actually happens.

  • by Marrow (195242) on Friday February 21, 2014 @07:12PM (#46307837)

    No software. No seat belts. No automatic..anything.

    • No software. No seat belts. No automatic..anything.

      You'd have to restrict that to old motorcycles. My '98 has ABS and fuel injection, both of which used programmed electronics. Newer bikes include systems such as CAN Bus, traction control, fly by wire throttles and more. Except for things like air bags, seat belts and bumpers, motorcycles use a lot of technology found in automobiles.

    • by thegarbz (1787294)

      You better call Yamaha and tell them to stop putting drive-by-wire throttles, ABS, and stability control on their motorbikes.

  • by Anonymous Coward on Friday February 21, 2014 @07:14PM (#46307853)

    Idiot drivers hit the gas pedal instead of the brake and instead of owning up to their incompetence as a drivers, they blame the car instead. The Toyota sudden acceleration problem disproportionately affects the elderly and inexperienced drivers. It also a uniquely an American problem and it occurred during a deep recession where GM and Chrysler were going bankrupt and Americans needed some FUD against Toyota because supporting American car companies was the jingoism of the day. The toyota sudden acceleration is more of a case study of an American moral panic and mass hysteria perpetrated by the media than it was an engineering problem.

    • by phantomfive (622387) on Friday February 21, 2014 @09:03PM (#46308549) Journal
      I think it was actually two problems. There was the problem reported by Wozniak, where cruise control would start to accelerate, but tapping on the brakes would fix the problem. It was a bug, but it wasn't life-threatening at all.

      Then there is the problem of the car going wildly out of control, unable to stop even when the brakes are applied. That one seems to be a case of foolish driver syndrome.
    • by Attila Dimedici (1036002) on Saturday February 22, 2014 @08:17AM (#46310465)
      Actually, the best analysis of the situation I have seen came from someone who had investigated a similar problem that was reported for cars with purely mechanical acceleration linkages some years back. They had studied that situation and discovered that overwhelming majority of those experiencing the problem were in a particular age range (I forget now if that age range was 55-65, or somewhat older). Further analysis revealed that most people go through a kinesthesia change during that period (kinesthesia is the awareness of where a body part is based on the sensations you are receiving from it). When going through that change, people often believe that their feet are positioned a few inches from where they actually are. As a result, drivers in this age range are likely to be positive that their foot is on the brake when it is actually on the accelerator. The interesting thing is that once the person becomes used to the change, they are perfectly capable of being aware of where their foot is once more. The person who did that original analysis analyzed the Toyota data and found that the majority of reported cases involved drivers who were in the same age range. When he took out those data points which looked suspicious as to being part of this actual problem(drivers who looked to be cashing in on the publicity of this, either for money or to explain away their own bad behavior, accidents where no one in the vehicle survived, but this was used to explain irrational behavior on the part of the driver, etc) the overwhelming majority of cases were in this age range and most of the remaining were inexperienced drivers.
  • by 140Mandak262Jamuna (970587) on Friday February 21, 2014 @07:17PM (#46307885) Journal
    Suddenly Toyota lawyers sued this website http://stackoverflow.com/ [stackoverflow.com] and claimed they are victims too.
  • if you cant demonstrate it in lab conditions it is just awlawyer speculating about stuff "M'lud I postulate it was the alien space bats wot dun it"
  • Not much (Score:5, Interesting)

    by n1ywb (555767) on Friday February 21, 2014 @07:23PM (#46307909) Homepage Journal
    Honestly, not much, except perhaps demand better software. Better processes, better languages. I'm just hypothesizing here but it might not have happened if they had e.g. followed better development standards like the MISRA C standard, or don't use C at all, use Ada or something. Better QA processes might have caught it before it went into production, e.g. using a dynamic stack profiling tool, input fuzzing, whatever. Fundamentally a system like this should have an independant hardware watchdog timer to at least try and make it fail-safe in the event of a CPU crash. Finally any motor vehicle ought to have a manual cutoff switch wired into the fuel pump or ignition circuit so that when the CPU shits it's bits you can still turn the damn thing off before you crash crash.
  • Can != did (Score:4, Insightful)

    by 140Mandak262Jamuna (970587) on Friday February 21, 2014 @07:24PM (#46307921) Journal

    "We've demonstrated how as little as a single bit flip can cause the driver to lose control of the engine speed in real cars due to software malfunction that is not reliably detected by any fail-safe," Michael Barr, CTO and co-founder of Barr Group, told us in an exclusive interview. Barr served as an expert witness in this case

    Emphasis mine.

    Yes, a single bit flip can cause unpredictable behavior in any code. You could say that without any analysis. A single mistake in sign can get you a goose egg in the Algebra paper. So many people could have won the lottery if only one digit was different. These are well known. But can != did. Did that stack overflow? Did it actually happen? That is the question.

    • Re:Can != did (Score:4, Insightful)

      by Cochonou (576531) on Saturday February 22, 2014 @12:04AM (#46309245) Homepage
      Did it actually happen? That is the question.

      From an engineering standpoint, it's not really the question - if there is a design flaw so that the system can fail with a non-negligible probability, it will eventually fail. Bits flip everyday, everywhere, but there should be mitigation in place to take care of that (at least a watchdog).
      • Re:Can != did (Score:4, Insightful)

        by AmiMoJo (196126) * <[ten.3dlrow] [ta] [ojom]> on Saturday February 22, 2014 @09:11AM (#46310653) Homepage

        A watchdog isn't the best solution to this problem. You don't really want your ECU to reboot randomly due to a fixable error. Just use ECC RAM and keep redundant copies of critical values in memory.

        There is no way this issue could have caused the number of events people are claiming to have had. Such a bit flip has never been observed in real life (they used a debugger to simulate it) and the changes of that particular bit out of millions corrupting is extremely low. Of bit flips were common enough to cause this we would see the effects as other systems like the dashboard display and head unit crash, not to mention other parts of the ECU.

  • by EmperorOfCanada (1332175) on Friday February 21, 2014 @07:24PM (#46307923)
    Quite simply the absolute control should not be handed over to the computer. Basically doing something like pulling on the handbrake should basically physically cut the throttle. Or stomping on the brakes should activate a simple solenoid that cuts the throttle. This mechanism should be 100% separate from the computer and override most computer outputs.

    I see this as critical in a driverless car. There needs to be a way for people to pull the plug and there needs to be a way for people to phone in an emergency. So if someone is lying in a pothole being run over by car after car, or the bridge is failing, there needs to be a way for 911 to say that a stretch of road is now cut off. The key is that this cannot be ab abusable by officials. I do not want my car grinding to a halt because the police are looking for some runaway or a bank was robbed.
    • by StikyPad (445176)

      I do not want my car grinding to a halt because the police are looking for some runaway or a bank was robbed.

      GLWT.

    • by jcdr (178250) on Friday February 21, 2014 @08:19PM (#46308233)

      Actually the brakes alone are enough to stop the car even in case of a full throttle bug.

      • I have actually had to do this before. Had a 2002~ A8L that would full throttle on its own, and yes the breaks are more powerful than the engine. We spent months going around with Audi on the issue before at some point we took a regional manager on a ride and it did it to him. And no it wasn't the fucking floor mat. They took it back without a word and gave us a newer model with 20k less miles. The important part to note here is that stepping on the breaks will still stop the car.
    • That would be hilarious. Something goes wrong, and every car is automated, so you just get car after car calmly driving their passengers to their death in a giant sink hole, or something.

      • This will happen, and it will make national/international news, and there will be a bunch of asswipes all going, "I told you so, these automated cars were going to be the death of us all." But this will be in the face of driverless cars killing so few people that it might be single digits nationwide.

        Oh and I forgot to mention all the "experts" they will get on the news who will try to turn driverless car safety into an issue; when in fact any changes that they propose will probably kill even more people.
    • by vidarlo (134906)
      What you're asking for is basically an emergency stop. The problem is that in some cases this can be dangerous as well. What if there's a truck 30 feet behind you, and you suddenly by accident (or inherent fault) activate the emergency stop? Safety is complex, and I'm not sure emergency stop is a good idea here, as it introduces it's own problems.
  • How can users protect themselves from sometimes life endangering software bugs?

    Drive older non digital cars. Come to think of it I can get you a great deal on a factory standard model 1971 Ford Pinto.

  • coding standards (Score:5, Informative)

    by lkcl (517947) <lkcl@lkcl.net> on Friday February 21, 2014 @07:33PM (#46307965) Homepage

    ... you know... i worked for pi technology in milton, cambridge, in 1993. they're a good company. they write automotive control systems, engine control management software, vehicle monitoring software and diagnosis systems and so on. one of the things i learned was that coding standards for mission-critical systems such as engine control have to be very very specific and very very strict. the core rules were simple:

    1) absolutely no recursion. it could lead to stack overflows.
    2) absolutely no local variables. it could lead to stack overflows.
    3) absolutely no use of of malloc or free. it could lead to stack overflows.

    now you're telling me that there are actually car manufacturers that cannot be bothered to follow even the simplest and most obvious of safety rules for development of mission-critical software, on which peoples' lives depend? that's so basic that to not adhere to those blindingly-obvious rules sounds very much like criminal negligence.

    • absolutely no local variables. it could lead to stack overflows

      In which case you shouldn't have function calls or interrupts either. I know recursion and dynamic allocation are to be avoided, but even in the highest rel standards I've never heard of not being able to use local variables. Careful stack use analysis and testing, sure.

  • by Trax3001BBS (2368736) on Friday February 21, 2014 @07:38PM (#46308013) Homepage Journal

    For anyone else that's curious; at first I thought it was double speak, so not to sound as bad.

    Stack overflow refers specifically to the case when the execution stack grows beyond the memory that is reserved for it. For example, if you call a function which recursively calls itself without termination, you will cause a stack overflow as each function call creates a new stack frame and the stack will eventually consume more memory than is reserved for it.

    Buffer overflow refers to any case in which a program writes beyond the end of the memory allocated for any buffer (including on the heap, not just on the stack). For example, if you write past the end of an array allocated from the heap, you've caused a buffer overflow.

    http://stackoverflow.com/quest... [stackoverflow.com]

  • NO (Score:5, Informative)

    by confused one (671304) on Friday February 21, 2014 @07:58PM (#46308125)
    First rule of real time: No recursion.
  • by mveloso (325617) on Friday February 21, 2014 @08:22PM (#46308255)

    From the slides:

    "Toyota used dangerous recursion"

    Not like that safe recursion that other vendors use.

  • by GrahamCox (741991) on Friday February 21, 2014 @09:43PM (#46308779) Homepage
    Anyone who knows anything about coding, or just has an interest in this case can read a great summary here: http://www.safetyresearch.net/2013/11/07/toyota-unintended-acceleration-and-the-big-bowl-of-spaghetti-code/ [safetyresearch.net] which also has links to the testimony of the expert witnesses to the court.

    The testimony makes fascinating reading, and is based on analysis of the actual source code in clean-room conditions. If after reading it you still think this isn't a software problem, maybe you need to turn in your geek badge right now.
    • by AmiMoJo (196126) * <[ten.3dlrow] [ta] [ojom]> on Saturday February 22, 2014 @05:59AM (#46310137) Homepage

      That article demonstrates just how clueless the guys doing the testing were. For example they complain that there "thousands of global variables", but that is actually the normal way to write safety critical firmware since local variables can cause stack overflows. They couldn't read any of the source code comments which were in Japanese either, only get poor machine translations of them.

      Most damning of all though is that actually the skid marks the article claims are evidence of the bug are easily explainable, and indeed Toyota did offer an explanation. If the mat/carpet causes the brake and accelerator pedals to become linked pressing one with of course press the other was well. The driver could not explain why she didn't push the brake pedal hard enough to stop the car (even with max acceleration the brakes will always win over the engine), but Toyota could. The mat that was also pushing the accelerator was preventing her from fully engaging the brakes. The pedal could not be pushed down fully.

      The fix was to change the firmware to stop accelerating when both the accelerator and brake are heavily engaged. So, in actual fact, the supposedly lethally flawed firmware is now saving people from their own stupidity.

  • Simple (Score:4, Funny)

    by bigjocker (113512) * on Saturday February 22, 2014 @01:15AM (#46309453) Homepage

    Use java

  • by Ateocinico (32734) on Saturday February 22, 2014 @02:04AM (#46309541)

    Mission critical systems usually have a set of voting computers. Today electronics is cheap and the technology is not new. Maybe inertia prevents them of building something better. Very common in the automobile industry.

Science and religion are in full accord but science and faith are in complete discord.

Working...