Stack Overflow Could Explain Toyota Vehicles' Unintended Acceleration 664
New submitter robertchin writes "Michael Barr recently testified in the Bookout v. Toyota Motor Corp lawsuit that the likely cause of unintentional acceleration in the Toyota Camry may have been caused by a stack overflow. Due to recursion overwriting critical data past the end of the stack and into the real time operating system memory area, the throttle was left in an open state and the process that controlled the throttle was terminated. How can users protect themselves from sometimes life endangering software bugs?"
Wow (Score:5, Funny)
Re:Wow (Score:5, Interesting)
I have to admit, that was my first thought as well. :)
Re:Wow (Score:5, Funny)
Re:Wow (Score:5, Funny)
Gives a new meaning to "race condition," doesn't it?
Re: (Score:3)
I am not so sure, have you tried it yet?
Re:"Stack Overflow" not good for discussion site. (Score:5, Insightful)
Technically knowledgeable people often give very poor names to their efforts.
I thought "Stack Overflow" was great branding for a website aimed at helping programmers solve technical problems. It's a distinctive, cheeky in-reference understood by its intended audience. (And honestly, it didn't hurt that most developers enjoy being made to feel clever about themselves.) That's what a brand is suppose to do [about.com], and it partially* explains their overwhelming success [alexa.com]. And hey, much better branding choice than ExpertSexChange.com!
*Of course, branding is just one of many things they did right. They also filled a unique niche, understood their community (because it was started by programmers, for programmers), and made the site super-easy to use by (here comes the important part...) NOT crapping over the UI with a fake paywall that sought rent for years' worth good-faith user contributions. However, they are sort of starting to be dicks about subjective questions (such as help with API choices, etc.). That may provide a niche for a new competitor to fill...
Re: ieee floats (Score:5, Informative)
Division by zero results in positive or negative infinity, depending upon the sign of the numerator.
False. Division by zero is simply undefined. Sometimes you can determine the limit of some function as it approaches a discontinuity due to division by zero, but the limit will depend on the function (not just the numerator) and possibly on which side you approach the discontinuity from. For example, the function x/x has a discontinuity at x=0, but the limit isn't infinity, it's 1. The function 1/x approaches positive infinity as x goes to zero from the right, and negative infinity as x goes to zero from the left; since these are not equal, the limit does not exist at x=0. On the other hand, the limit of 1/(x*x) at x=0 is in fact positive infinity, because the function approaches positive infinity from both sides.
If X divided by zero were actually defined to be "equal to" positive infinity for all X > 0, then you could turn that equation around and say that 0 * infinity = X. Since 1/0 = infinity, 0 * infinity = 1. And 2/0 = infinity, so 0 * infinity = 2. Ergo 1 = 2. (Or not...)
The problem is that infinity isn't a number you can calculate with; it's more of a trend, or a way to express the absence of a finite boundary. An expression is never "equal to" infinity, though it may approach infinity as some other condition changes.
Did anyone else read that title and think... (Score:5, Funny)
.
Go Amish? (Score:5, Insightful)
"How can users protect themselves from sometimes life endangering software bugs?"
Amish buggies typically don't have software throttle failures. Run-away horses can be an issue.... and actually having to share the road with dipshit drivers who don't understand the number of slow moving vehicles (not just buggies) that there are out in farm country are a real danger.
Seriously, software has bugs. Mecanical throttle linkages can stick, too. Life has risks.
Re: (Score:3)
Skies are blue.
Software and cheaper accommodations have bugs.
Re:Go Amish? (Score:5, Funny)
Re: (Score:2)
Sometimes the question is dumb and asking to do the wrong thing. There isnt a good answer to "how do I avoid software bugs" other than "avoid software", and not the answer that is wanted.
Re:Go Amish? (Score:4, Insightful)
Coming from the aerospace industry, you cannot have software that has bugs. And if there was the possibility of a software bug, you have to prove that you can mitigate the effect in hardware. So just to say "software has bugs...life has risks" isn't an acceptable answer (in my opinion). We have to remember this is not an apples to apples comparison. Just because traditional consumer software always has bugs in it (which are acceptable) doesn't mean they are acceptable in other industries. Considering that the failure puts someone's life at risk, I would think it should be considered unacceptable in automotive industry as well.
Re:Go Amish? (Score:5, Interesting)
Coming from the aerospace industry, you cannot have software that has bugs. And if there was the possibility of a software bug, you have to prove that you can mitigate the effect in hardware. So just to say "software has bugs...life has risks" isn't an acceptable answer (in my opinion). We have to remember this is not an apples to apples comparison. Just because traditional consumer software always has bugs in it (which are acceptable) doesn't mean they are acceptable in other industries. Considering that the failure puts someone's life at risk, I would think it should be considered unacceptable in automotive industry as well.
If you want your cars to be as expensive as a 747, then you can attain that goal. I used to work in the automotive industry designing embedded software for engine management systems. At that time, no automotive company would pay more than $100 for the Engine Control Unit. Probably 60% of the code was written to manage failures (both software and hardware), and there were other electronic fail safe mechanisms. But you can't mitigate every possible failure event without introducing costs that would have made the unit orders of magnitude more expensive.
Re: (Score:3)
I'm not so sure cost is the biggest problem. Once upon a time ABS was an exotic tech used only on aircraft. I was impressed when they became a car option.
My biggest concern is whether a driverless car can be smart enough to reliably handle all the situations that arise. Probably someday, but I don't know when. Google's over-hyped toys can only handle clear weather, and even then they often have to kick out and go to manual control.
Re:Go Amish? (Score:5, Interesting)
In other words, ABS software (and hardware) was very expensive to develop, and only the development budgets of airliners was large enough to cover its development. Once developed, the companies that developed it realized that adapting it for automotive use would be within the budget of luxury car makers, and once it was working there, it became very cheap to adapt for standard cars, as the differences are very minor (in fact it is basically a case of the luxury car makers funding continued improvements, and standard cars getting the previous generation that already has its development paid for).
Re: (Score:3)
Re:Go Amish? (Score:5, Informative)
The aerospace industry deploys bugs very frequently. Don't pretend like you don't. Yes, for some applications, we test the hell out of it, but bug free, hardly.
Re:Go Amish? (Score:5, Insightful)
Even in the aerospace industry, there are software bugs. Very few, yes, because a lot more time and money is spent to track them down. There are mechanical failures too, despite the best engineering efforts. But anything we build has the potential to be flawed somewhere in the process. That's why we still put highly trained pilots in the things, for when something goes wrong - and even then, human error causes unintended faults, sometimes catastrophically.
If a car cost as much as a jet, and drivers went through as much training as a passenger pilot - and had to have a co-driver at all times - we'd be far safer on the roads.
After all, the vast majority of car crashes are driver error. And failure modes when you're at 30mph on wheels are a lot better on the whole than when at 30,000 feet. But if we built cars to the same standard, and held drivers to the same standard as aerospace engineering, only the rich could afford to.
There's a trade off between acceptable risk, and cost. Even though the designs get safer every year, maybe we allow too much risk in drivers and their cars. But absolute flaw free cars? Impossible.
Re: (Score:3)
Yes, aerospace software [wikipedia.org] has solved the problem of software bugs. We should all be following its perfect example. </sarcasm>
Re: (Score:3)
You're not supposed to, but you do routinely have software that has bugs even in aerospace, because there is no development process that can guarantee the prevention of 100% of defects, nor even guarantee that 100% of defects are detected and corrected.
Re: (Score:3)
Coming from a fault tolerant computing background, your non-trivial software could have bugs. There is a reason N-version programming [wikipedia.org] exists (and it's not for fun).
Unless you have true N-version programming with totally separate teams writing on different platforms with different languages, different academic backgrounds, different cultural paradigms, etc, then you don't have perfectly reliable software. (And even then your application may lend itself to certain symantic commonalities.)
When done properly, r
Re: (Score:3)
You can prove a program correct, but it's an exercise in academic wankery; it's at least as difficult and error-prone to prove a program correct as it is to write it in the first place.
Re: (Score:2)
Re: (Score:2)
Until someone actually reproduces the bug I'd say the loose floor mat explanation is just as credible.
I'm actually going for "operator error" but hey...
Two cases come to mind. First, was short term unintended acceleration. Probable cause for that is hitting the wrong or both peddles.
Long term acceleration on the highway? Nobody thought to just turn off the ignition switch? Turns off the fuel pump, care stops running, eventually you come to full stop.
BOTH where operator error..
Re:Go Amish? (Score:5, Insightful)
Killing the ignition also means killing power steering and power braking. There is a quite widespread belief that it can also engage the steering wheel lock, but AFAIK no one has been able to name a car where that happens so far. The next challenge is that in many modern cars the ignition switch is just a button which is handled in... software. You could throw the key out of the window and wait for the anti-theft device to kill the fuel supply, but that does not seem like something that most people would try.
In most cars you can put the gear box in neutral. The car will likely have a rev limiter (possibly in software, but it might still work). Worst case the engine breaks, but in almost all cases that would not be fatal to the people in the car.
However, in almost all cars, when not going down a steep hill, the brakes are actually more powerful than the acceleration. Just do not let off the brakes once you get the car slowed down and you think things are under control -- then the brakes overheat and you have a stuck accelerator combined with no brakes, and that has killed at least one driver already.
Got it Handled! (Score:5, Funny)
Problem solved!**
**Some users may experience complete lack of vehicle control while the system is rebooting.
Mandatory publication? (Score:5, Interesting)
How can drivers protect themselves.... (Score:2)
One way would be to insist that automakers do not nickel and dime design vehicles. The critical components related to vehicle safety should be designed for safety first, cost second.
These vehicles go for over $20 000, I should at least have the option to pay an extra $1000 to chuck the electronic crap.
Re: (Score:2)
One way would be to insist that automakers do not nickel and dime design vehicles. The critical components related to vehicle safety should be designed for safety first, cost second.
These vehicles go for over $20 000, I should at least have the option to pay an extra $1000 to chuck the electronic crap.
The electronics are very deeply embedded. Not sure how you're gonna dump them when there's no physical cable connecting your throttle to your engine. Impossible in the case of EVs or hybrids.
Also although the article does a decent job of showing that a stack overflow is possible and might result in unexpected behavior, what's needed is a simulated failure scenario to see if that's what actually happens.
Motorcycles! (Score:3)
No software. No seat belts. No automatic..anything.
Re: (Score:3)
No software. No seat belts. No automatic..anything.
You'd have to restrict that to old motorcycles. My '98 has ABS and fuel injection, both of which used programmed electronics. Newer bikes include systems such as CAN Bus, traction control, fly by wire throttles and more. Except for things like air bags, seat belts and bumpers, motorcycles use a lot of technology found in automobiles.
Re:Motorcycles! (Score:4, Informative)
Does it have an automatic transmission, and if not does it have clutch by wire?
Automatic transmissions are common (perhaps universal) on scooters and have been used on motorcycles in the past. The newest BMWs have ability to shift without pulling the clutch lever or reducing the throttle. From http://www.motorcycledaily.com... [motorcycledaily.com] "The BMW Gear Shift Assistant Pro, available as a factory option, represents a world first for production motorcycle manufacture. It enables upshifts and downshifts to be made without operation of the clutch or throttle valve in the proper load and rpm speed ranges while riding."
Re: (Score:3)
You better call Yamaha and tell them to stop putting drive-by-wire throttles, ABS, and stability control on their motorbikes.
Mental stack overflow of the driver is more likely (Score:5, Interesting)
Idiot drivers hit the gas pedal instead of the brake and instead of owning up to their incompetence as a drivers, they blame the car instead. The Toyota sudden acceleration problem disproportionately affects the elderly and inexperienced drivers. It also a uniquely an American problem and it occurred during a deep recession where GM and Chrysler were going bankrupt and Americans needed some FUD against Toyota because supporting American car companies was the jingoism of the day. The toyota sudden acceleration is more of a case study of an American moral panic and mass hysteria perpetrated by the media than it was an engineering problem.
Re:Mental stack overflow of the driver is more lik (Score:4, Interesting)
Then there is the problem of the car going wildly out of control, unable to stop even when the brakes are applied. That one seems to be a case of foolish driver syndrome.
Re:Mental stack overflow of the driver is more lik (Score:5, Insightful)
Re: Mental stack overflow of the driver is more li (Score:5, Insightful)
he was sitting at a stop sign waiting for a train to pass when the car inexplicably accelerates and lurched forward.
Why did he take his foot off the brake?
Re: (Score:3)
Driver error. When you're stopped you're supposed to keep the brake down (and the wheels straight). The usual reason is in case someone rear ends you. My car even helps out: if you press the brake all the way down it holds it on until you release it completely.
Sure, most people like to do the creeping forward thing. Most people are poor drivers.
Those wily Toyota lawyers .... (Score:5, Funny)
Likley the cause? (Score:2)
Not much (Score:5, Interesting)
Can != did (Score:4, Insightful)
"We've demonstrated how as little as a single bit flip can cause the driver to lose control of the engine speed in real cars due to software malfunction that is not reliably detected by any fail-safe," Michael Barr, CTO and co-founder of Barr Group, told us in an exclusive interview. Barr served as an expert witness in this case
Emphasis mine.
Yes, a single bit flip can cause unpredictable behavior in any code. You could say that without any analysis. A single mistake in sign can get you a goose egg in the Algebra paper. So many people could have won the lottery if only one digit was different. These are well known. But can != did. Did that stack overflow? Did it actually happen? That is the question.
Re:Can != did (Score:4, Insightful)
From an engineering standpoint, it's not really the question - if there is a design flaw so that the system can fail with a non-negligible probability, it will eventually fail. Bits flip everyday, everywhere, but there should be mitigation in place to take care of that (at least a watchdog).
Re:Can != did (Score:4, Insightful)
A watchdog isn't the best solution to this problem. You don't really want your ECU to reboot randomly due to a fixable error. Just use ECC RAM and keep redundant copies of critical values in memory.
There is no way this issue could have caused the number of events people are claiming to have had. Such a bit flip has never been observed in real life (they used a debugger to simulate it) and the changes of that particular bit out of millions corrupting is extremely low. Of bit flips were common enough to cause this we would see the effects as other systems like the dashboard display and head unit crash, not to mention other parts of the ECU.
This is a case of manual override (Score:5, Insightful)
I see this as critical in a driverless car. There needs to be a way for people to pull the plug and there needs to be a way for people to phone in an emergency. So if someone is lying in a pothole being run over by car after car, or the bridge is failing, there needs to be a way for 911 to say that a stretch of road is now cut off. The key is that this cannot be ab abusable by officials. I do not want my car grinding to a halt because the police are looking for some runaway or a bank was robbed.
Re: (Score:2)
I do not want my car grinding to a halt because the police are looking for some runaway or a bank was robbed.
GLWT.
Re:This is a case of manual override (Score:4, Informative)
Actually the brakes alone are enough to stop the car even in case of a full throttle bug.
Re: (Score:3)
Re: (Score:3)
That would be hilarious. Something goes wrong, and every car is automated, so you just get car after car calmly driving their passengers to their death in a giant sink hole, or something.
Re: (Score:3)
Oh and I forgot to mention all the "experts" they will get on the news who will try to turn driverless car safety into an issue; when in fact any changes that they propose will probably kill even more people.
Re: (Score:3)
Re: (Score:2)
I strongly suggest that you re-evaluate the needs of those public servants who had their blue lights on en route to their coffee/donut break a few months back.
Protecting yourself from bugs? (Score:2)
How can users protect themselves from sometimes life endangering software bugs?
Drive older non digital cars. Come to think of it I can get you a great deal on a factory standard model 1971 Ford Pinto.
coding standards (Score:5, Informative)
... you know... i worked for pi technology in milton, cambridge, in 1993. they're a good company. they write automotive control systems, engine control management software, vehicle monitoring software and diagnosis systems and so on. one of the things i learned was that coding standards for mission-critical systems such as engine control have to be very very specific and very very strict. the core rules were simple:
1) absolutely no recursion. it could lead to stack overflows.
2) absolutely no local variables. it could lead to stack overflows.
3) absolutely no use of of malloc or free. it could lead to stack overflows.
now you're telling me that there are actually car manufacturers that cannot be bothered to follow even the simplest and most obvious of safety rules for development of mission-critical software, on which peoples' lives depend? that's so basic that to not adhere to those blindingly-obvious rules sounds very much like criminal negligence.
Re: (Score:3)
absolutely no local variables. it could lead to stack overflows
In which case you shouldn't have function calls or interrupts either. I know recursion and dynamic allocation are to be avoided, but even in the highest rel standards I've never heard of not being able to use local variables. Careful stack use analysis and testing, sure.
Re:coding standards (Score:5, Informative)
This should be rule number one for this type of application.
Perhaps it should be rule number one, but actually it's Rule 16.2 of MISRA-C:2004 (Motor Industry Software Reliability Association, Guidelines for the use of the C language in critical systems):
Functions shall not call themselves, either directly or indirectly.
The rule actually appeared first in MISRA-C:1998. Each rule is accompanied by a detailed rationale that I will not reproduce verbatim here as the standard is not open; one must pay for the privilege. The rationale for 16.2 is that recursion may cause stack overflows. I only cite the rule itself because it appears in public testimony and also on the (first) page linked by this story ...... which you obviously did not read.
Because MISRA also disallows constructs such as function call indirection, self modifying code, etc. a compiler is entirely capable of detecting recursion and reporting the violation as an error. MISRA compliant compilers do exactly that.
Yes Virginia, the largest auto manufacturer on Earth ignores the very thing that was designed to prevent simple, common, easily predictable failures such as stack overflow despite the fact that the cost of compliance is much, much smaller than a rounding error for an outfit like Toyota.
Also, despite the fact that Industry dutifully identified this specific problem in a published standard at least 16 years ago, compliance is apparently not yet a requirement by government regulators. I suspect they're too busy investigating child seat manufacturers or Telsa batteries or whatever other politically high profile crisis that giant, engineer-free gaggle of NTSB lawyers fill their bankers hours with.
Re:coding standards (Score:4, Informative)
We are following all these rules and so we are safe. We can save a penny per 1000 units sold using a crappy MMU-less CPU.
First of all, following the stupid rules requires you to use baroque lint imitations which will go off on every line of idiomatic C. You need a paper trail to justify every line of code. Seems about right, people's lives are in danger, right?
Now consider that the controller system is hundreds of thousand of LOCs(for us it's more like millions). Most of that is crap boilerplate code required by the standards. This means if you follow that methodology strictly, you need hundreds of people going through mindnumbing lists of "You are not using this argument/This code assigning an argument to itself does nothing". Given that most software developers are inept and overworked, I can give you a certificate that there will be bugs.
It took me two weeks with the code to find a checksum function used all over the place that had been "fixed" to detect offset data after some earlier corruption bug was not detected.
Every 256 bytes "checksummed", a bit from the input would be left unaccounted(And it was actually used for data several times larger than that). I know for a fact that had to go through at least three source and design reviews and at least one more design review with some fat managers higher up.
Now tell me you feel safe.
Note to PHBs: Googleing up a fucking working CRC and getting a CS PhD to make a formal proof that it will work as intended would have cost far less.
Also, you see, the crappy CPU vendor stack measuring tools - that rules say we must use to guarantee safety - don't account for function pointers(they do show scary icons for recursive functions). They say foo(384) bar(uhhm... maybe 0?) I know to look for that when I add calls to function pointers, but I guess most people don't.
Now you add another rule. LOZRA 4092: You can't use function pointers at all.
Make my life more miserable, give the remaining work I will be unable to do to Dave, the monstera plant, or someone with the same programming aptitude.
I will give the crappy CPU/Compiler/RTOS vendors that should be sued free advice:
0- Add an MPU
1- Add canaries to every function call with any local variable at all(here it's not hackers it's programmers following LOZRA 396: cast the shit off everything so the compiler can't tell)
2- Add stack overflow canaries on every task switch. (add an MPU and align to page in the stack growing direction)
3- Add canaries to any memory pool allocation. (add MPU dead pages - You don't need RAM, just fucking address space of which you are using like 2%)
4- If any of the above traps, jump to a customer defined function(stored in ROM than can only be physically modified by outside hardware) that puts all vital hardware in a safe state, adds a record to the black box and reset the whole thing from scratch.
5- Forget about tasks and threads and move on to processes running on separate address spaces. If information must flow from a to b it better go through accepted channels. 6- Did I tell you to add a fucking MPU!?
Re: (Score:3)
Re:coding standards (Score:5, Informative)
> I wouldn't want to drive a car which was designed on a budget restriction.
That criterion will eliminate a lot of confusing choice from your purchasing decisions.
Re: (Score:3)
:) Which is why we had to spend the first 6 months after hiring any new grad, retraining them in development techniques that actually worked in our embedded near-real-time, real-world sitations. I still have no idea why colleges convince their graduates that they actually know anything. College is an opportunity to learn how to think and how to learn, not to learn what's needed to be an instant star. We all have to learn constantly our whole lives to stay on top of the technology.
That's not to say I've
Re: (Score:3)
Certainly most embedded code is terrible. I've worked on systems from the smallest to the largest (4K ROM + 128 bytes RAM to Google-scale) and it turns out that MOST code is terrible. There was one embedded project I worked on where most of the code was a horrible mess... and then there was this one little module that was beautiful. And not by appplication of any rules like "no function pointers"; no, it was
Stack overflow vs. buffer overflow (difference) (Score:5, Informative)
For anyone else that's curious; at first I thought it was double speak, so not to sound as bad.
Stack overflow refers specifically to the case when the execution stack grows beyond the memory that is reserved for it. For example, if you call a function which recursively calls itself without termination, you will cause a stack overflow as each function call creates a new stack frame and the stack will eventually consume more memory than is reserved for it.
Buffer overflow refers to any case in which a program writes beyond the end of the memory allocated for any buffer (including on the heap, not just on the stack). For example, if you write past the end of an array allocated from the heap, you've caused a buffer overflow.
http://stackoverflow.com/quest... [stackoverflow.com]
NO (Score:5, Informative)
Dangerous recursion! (Score:5, Funny)
From the slides:
"Toyota used dangerous recursion"
Not like that safe recursion that other vendors use.
Read this before you blame the driver (Score:3)
The testimony makes fascinating reading, and is based on analysis of the actual source code in clean-room conditions. If after reading it you still think this isn't a software problem, maybe you need to turn in your geek badge right now.
Re:Read this before you blame the driver (Score:5, Interesting)
That article demonstrates just how clueless the guys doing the testing were. For example they complain that there "thousands of global variables", but that is actually the normal way to write safety critical firmware since local variables can cause stack overflows. They couldn't read any of the source code comments which were in Japanese either, only get poor machine translations of them.
Most damning of all though is that actually the skid marks the article claims are evidence of the bug are easily explainable, and indeed Toyota did offer an explanation. If the mat/carpet causes the brake and accelerator pedals to become linked pressing one with of course press the other was well. The driver could not explain why she didn't push the brake pedal hard enough to stop the car (even with max acceleration the brakes will always win over the engine), but Toyota could. The mat that was also pushing the accelerator was preventing her from fully engaging the brakes. The pedal could not be pushed down fully.
The fix was to change the firmware to stop accelerating when both the accelerator and brake are heavily engaged. So, in actual fact, the supposedly lethally flawed firmware is now saving people from their own stupidity.
Simple (Score:4, Funny)
Use java
A single processor always fails. (Score:3)
Mission critical systems usually have a set of voting computers. Today electronics is cheap and the technology is not new. Maybe inertia prevents them of building something better. Very common in the automobile industry.
Re: (Score:2)
> You could be mauled by a bear when you press the accelerator.
But only if you've purchased the "mauled by a bear" feature and have forgotten to put the "bear" switch in the "off" position before putting the car in gear.
I thought everyone knew that.
Re:Never use any software. (Score:4, Informative)
Not using recursion in constrained embedded systems is a good start. It's been best practice since I started working with them 15 years ago.
Re:Never use any software. (Score:5, Informative)
Because its very easy to overflow the stack? Embedded systems tend to have a limited stack space too.
Re:Outlaw Recursion (Score:5, Funny)
Re: (Score:3)
MISRA C, which is a mandatory coding standard across pretty much all the automotive industry, does outlaw recursion. So I find this speculation on the cause to be very unlikely, and is just lawyers bringing in "software experts" to speculate. If you want to speculate, there are many other potential software bug causes as well. Some of them would even pass MISRA C coding checks, and not be easily detectable by static analysis.
Re:Outlaw Recursion (Score:5, Insightful)
Uses can protect themselves against this sort of thing by not buying a Toyota until they are compliant with the relevant standards. Only hurting them in the marketplace will get them to fix this problem.
The testimony from both expert witnesses Barr and Koopman are now a matter of the public record and actually make fascinating reading - they'll be especially interesting to computer guys because it goes into a lot of detail about the code design, though frequently translated for the benefit of the court into layman's language. It's going to go down in history I reckon as a classic case of how not to design a safety-critical system.
A great summary and links to the public court documents can be found here: http://www.safetyresearch.net/2013/11/07/toyota-unintended-acceleration-and-the-big-bowl-of-spaghetti-code/ [safetyresearch.net]
Re: (Score:3)
The death penalty for programmers that write such code will bring an end to software !
FTFY
Re: And we're going to trust self driving cars now (Score:5, Insightful)
I've known a lot of high-quality developers over my 15 years of professionally developing software. The reason I don't want an automated car is because of these people. People make mistakes, intentionally or otherwise. There are unforeseen circumstances that the software may not understand. A foreseen circumstance: I've yet to see a demo of an automated car navigating anything resembling an icy surface (safely or otherwise), let alone in stop & go traffic in a city such as Chicago, where such things are quite common.
Yeah, but we know today who pays: the insurance company of the at-fault driver (provided they have the legally required insurance - I think collision is required in all US states). Failing that, the at-fault driver. Failing that, the dead at-fault driver's estate.
The question at hand is, in the case of an automated car, who is at fault (when the automated car is deemed to have caused the accident)? The manufacturer, because, it must have been a design/implementation flaw? The owner? The driver (because owner/driver aren't necessarily the same)? It becomes more difficult when you divest yourself from current paradigms of car transport. Oh, I sent my 6 year old daughter to school as a passenger, and along the way it ran over someone. There was no driver (the daughter was a passenger), but the car still killed someone. Am I at fault because I ordered the car to make the trip? Is the manufacturer at fault because the car didn't detect and prevent the collision that killed the other party? Is my 6 year old daughter at fault, because she was the only human occupant? This is the question that is being posed.
Re: And we're going to trust self driving cars now (Score:5, Insightful)
Yes, people do make mistakes. Often while driving. The test shouldn't be whether automated cars make mistakes, but rather whether they do better than an average driver. Can they deal with icy roads as well as an average driver? That bar's pretty low, even here in Edmonton.
Once they've reached that average competence and start being deployed, they'll also improve rapidly over time; computers have the potential to be much safer drivers than humans. They'd know where other cars are and where they're going, they'd be able to apply brakes to wheels independently with lightning reactions, and would not be subject to health conditions, intoxication, aging, or inexperience.
I'm not sure how far off we are, but it's definitely coming.
Re: And we're going to trust self driving cars now (Score:4, Insightful)
I've known a lot of high-quality developers over my 15 years of professionally developing software. The reason I don't want an automated car is because of these people. People make mistakes, intentionally or otherwise.
When it comes to true high-rel software, like that written to DO-178B Level A (an avionics software standard used for things like fly-by-wire) it's almost never the software per se that's at fault. The stuff is amazingly good. It's also amazingly expensive to write and test. You might also find it frustrating because it brings new meaning to the idea of conservative design. For example, I don't think it allows recursion. I know it doesn't allow dynamic allocation.
There are unforeseen circumstances that the software may not understand.
That's more the point. When things like fly-by-wire systems have problems, it's almost never the software itself, but something that was unforeseen by the system designers.
A foreseen circumstance: I've yet to see a demo of an automated car navigating anything resembling an icy surface (safely or otherwise), let alone in stop & go traffic in a city such as Chicago, where such things are quite common.
Agreed. This technology is interesting, but it's way over-hyped. It's impressive to be able to drive on a nice clear day, but a far cry from the real world, even in Silicon Valley, let alone Chicago.
Ever hear of the flying cars that in the 1960's they said we'd all have soon? I haven't seen any lately. I suspect driverless cars robust enough not to require human intervention when the going gets tough may be developed on the same schedule.
Re: (Score:3)
Well, one of the interesting features, IIRC, was that no mapping was done by the AI, it drove strictly "by the seat of its pants", and had no trouble performing precisely controlled power-slides that would make most drivers shit their pants. If we're discussing the challenges of driving on ice I think that's very relevant. I don't remember if it had any specific moving-obstacle-avoidance code, but plenty of other projects are tackling that. Conceptually at least it shouldn't be difficult to combine one A
Re: (Score:3)
Even easier is to just TURN IT OFF using the key and get on the breaks. Trust me, the car WILL stop running and come to a full stop fairly quick.
Not if it's a diesel engine.
Re: (Score:3)
Are you sure? Diesels have electronic controls these days. Even before that, how did you shut them off normally?
Re:I know what users could do! (Score:5, Interesting)
There is a lot more to this than simple driver error. Read the court testimony, it's a real eye-opener and in fact a really great read: http://www.safetyresearch.net/2013/11/07/toyota-unintended-acceleration-and-the-big-bowl-of-spaghetti-code/ [safetyresearch.net]
Re:Live in a cave (Score:5, Interesting)
Re:Live in a cave (Score:5, Insightful)
During the trial, embedded systems experts who reviewed Toyota's electronic throttle source code testified that they found Toyota's source code defective, and that it contains bugs -- including bugs that can cause unintended acceleration.
I'm currently weighing your hunch against the industry experts who reviewed the source code, identified specific flaws in detail in an 800+ page report, and testified to their findings, presumably under oath. I'm on the fence here, it's a tough call. I'll let you know what I decide.
Although I'll admit I get the feeling that there is some kind of conspiracy going on here, probably related to an Obama Muslim invasion. I mean, what are the odds that the "several hundred [people] contending that Toyota's vehicles inadvertently accelerated" all happened to be fans of Toyota's with the same electronic throttle control system?
Re: (Score:3, Interesting)
Did they also discover a flaw in the brakes such that they could not overcome the engine power? This was the point of the parent post, I think. Modern cars have sufficient braking force to completely stop the engine even at full throttle. So if the driver is "stepping on the brake really hard," the car should stop in spite of a stuck throttle, unless a simultaneous brake system failure can be demonstrated.
This same thing happened to Audi in the 1980s and as far as anyone can tell, objectively, it was at
Re: (Score:3)
but the brake pedal is also just a switch. if the computer is malfunctioning, that can explain the lack of any brake application too.
Re: (Score:3)
Re: Live in a cave (with switches) (Score:3)
In what car?! All modern mainstream vehicles still use a master cylinder in tandem with a booster and ABS. Even if you lose all engine power, should still be able to apply the brakes. Although it will require more force to push the pedal down, it should still be doable to bring the car to a complete safe stop.
If only the throttle was also required to have a physical linkage!
Unfortunately, once we abandoned the carburetor, there isn't much to link the throttle pedal to any more, and we are stuck with using it pretty like as a mouse. A physical pedal up fuel valve to reduce fuel availability to a level sufficient only to idle the engine might return some measure of physical control. Currently most manufacturers have the computer cut throttle when the brake is pressed, even if the gas is also pressed. (Toyota
Re: Live in a cave (Score:4, Interesting)
It depends on if the ABS and/or brake servo is influenced by the bug and makes it harder to apply the brakes.
Be aware that most brake servos are using the manifold vacuum to increase the brake force. If the engine gets full throttle the vacuum is soon depleted and a much larger force on the pedal is needed which will be experienced as failing brakes.
Re: Live in a cave (Score:4, Insightful)
I even found some information for you:
http://www.caranddriver.com/fe... [caranddriver.com]
A 268 horsepower Toyota Camery can stop from 70mpg in 190 feet with the engine at full throttle. Only 16 feet longer than not throttle. At 100mph the stopping different was 88 feet.
When they tested a full throttle stop from 120mph, the car slowed to 10mph before the brakes overheated.
Perhaps you should put your foot harder on your brake pedal.
You are american though, so your Impala is probably an automatic. In first gear with the torque converter multiplying the torque it may be able to slowly move the brakes when they're fully applied.
Re: (Score:3)
Weird. The stock brakes on both the '95 Caprice and '96 Impala SS sitting in my driveway can hold the car in place. That was true when the engine was stock, and is still true after adding a shift kit, PCM tune, cat-back, intake, and valve train upgrades. It's been true on both the factory tires and the substantially wider aftermarket tires. It might be time for you to replace your brake material; you're seemingly endangering the other cars on the road.
When you're trying to power brake, BTW, you'll want
Re: (Score:3)
Well, if it was in article comments on the Internet, that's a whole new story... ;)
No one sells a car in the US with exclusive brake-by-wire, because nearly every state mandates the existence of a second braking system independent of the primary braking system. That's often the thing people call the "emergency brake," as compared to the "service brake." For IL, look at Article III at http://www.ilga.gov/legislatio... [ilga.gov]. They must be separated such that a failure in any one part does not leave the vehicle w
Re: (Score:3)
Did they also discover a flaw in the brakes such that they could not overcome the engine power? This was the point of the parent post, I think. Modern cars have sufficient braking force to completely stop the engine even at full throttle
Is this definitely always the case? Under all conditions? There's a huge difference, for example, between holding down the brake while stopped and gunning the engine and slamming on the brake while already travelling 70 miles an hour with the engine similarly gunning. Static vs Dynamic friction for one thing. Not to mention brake exhaustion due to overheating. The pads and rotors heat up and the physical properties of both change, the rotors can warp, moisture can flash into steam and create a nearly fricti
Re: Live in a cave (Score:3)
A Prius does not. More than half the brake power is supplied by regenerative breaking, and if the register for throttle position is at 100, regenerative braking never kicks in, leaving you with the relatively whimpy disc brakes. A prius crash near me last year had the brake pads worn away and smoking when the police arrived.
Re:Live in a cave (Score:4, Interesting)
Many of these uncontrolled acceleration cases involved hybrid Toyota vehicles. In addition to the electronic throttle, Toyota's Hybrid Synergy Drive uses brake by wire, so the computer can dynamically use any desired combination of regenerative and friction braking, based on the hybrid battery charge state and the severity of the driver's control input on the pedal. These cars also eschew mechanical control for the gear-shift and the push-button ignition switch, relying on interface through the ECU.
It thus seems entirely plausible that a stack overflow, race condition or other crash/freeze/whatever could result in a wide-open throttle with no brakes and no gear-shift or ignition off control. if this is the case, it represents a epic lack of fail-safe design. It certainly doesn't help prevent operator error when Toyota uses a non-PRNDL shift pattern on their hybrids, to say nothing of the lack of industry standardization of the behavior of push-button ignition.
Re: Live in a cave (Score:4, Interesting)
Ok. it's very real. happened to my father-in-law several times before any news broke of this. so claiming it's the driver is pointless waste of time. now you can actually suggest something helpful.
Re: (Score:3)
I had a Citroen 2CV that did that. Luckily, it was so slow that I could jump out and run ahead and warn people to get out of the way.
Re:In Canada Engineers Are Required to Write the C (Score:5, Insightful)
Almost all safety systems in Ontario have not been designed or written by a PEO licensed engineer. The PEO is the same organization that tried to get Microsoft to stop using the term "Microsoft Certified Systems Engineer (MCSE)" and largely lost. If you start analyzing real and deployed systems, you will be shocked at what you find.
Yes, there are a few very well designed machines out there that do hardware and software interlocks properly, and in an obviously safe fashion. These are the exception, and I am delighted to find the few exceptions that exist.
However, if you want excellent examples of obviously unsafe things, consider:
- The gas pumps at Shell, Esso, and Petro-Canada. How many brands have an Emergency Stop button? One?
- Toyota cars have a push to start button that is also a push and hold to stop button. So how do you stop the car quickly? Shouldn't a car that has push-button start, also have a push-button stop, that is a different button and works quickly? Why would Toyota follow the Microsoft standard of using a start button to stop, instead of following the very well thought out emergency stop button standards?
- Hospitals have implemented a number of computer systems that are networked, and make the job of nurses quicker, easier and more productive. This reduced nursing costs considerably, and fewer nurses are looking after more patients. However, these systems are not reliable, and the official backup plan is that a nurse will step in and do the job manually if the system fails. Unfortunately, many of these core systems are also running on Microsoft Windows (often Windows XP.) One virus, or one bad update, written by a non-engineer, to wipe out many core systems. A major hospital had its Internet linked systems disrupted because too many people watched Olympic hockey (over the critical internal network.) Has any engineer approved any of this? Does any hospital have enough nurses to cover off in the event of a computer failure?
- Most servo-motor drives are sold with a "not recommended that power be cut by an emergency stop/safety system" warning buried somewhere in the documentation. Ignoring this, and assume braking resistors are used, and power is really cut. Most motors will follow an exponential stopping curve, and appear to coast to a stop. A mechanical engineer doing a PSHSR (Pre-Start Health and Safety Review) will expect the machine to stop quickly, and not coast. The cheapest way to do that is to dynamically brake into braking resistor under full software and transistor control. The second cheapest way is to use a parking brake, but those are not rated for safety and only a fraction of the servo-motor market uses parking brakes anyway. How many PEO licensed mechanical engineers doing PSHSR reviews have passed systems with incorrect software E-STOP circuits and Safety Circuits, and failed E-STOP circuits and Safety Circuits that cut power to the motors in hardware?
Do not depend on the PEO and statutes to keep you safe ...
Re: (Score:3)
I know that sometimes its difficult to admit that someone you know well may have made such a stupid mistake, but the probability of your father-in-law being confused/mistaken is a lot more than entire teams of engineers at the world's largest car manufacturer fucking up in such a dangerous manner.