Forgot your password?
typodupeerror
Programming IT Technology

In-Flight Reboot? 594

Posted by michael
from the no-problems-until-you-have-to-fsck dept.
steelem writes "The Washington Post is running a story about how the F-22 Raptor's software requires in-flight reboots. Apparently the 2 million line software project is 93% done. Knowing most projects I've been on, it'll stay that way for another few years."
This discussion has been archived. No new comments can be posted.

In-Flight Reboot?

Comments Filter:
  • by DeathPenguin (449875) * on Friday August 01, 2003 @07:34PM (#6592799)
    This is an ideal application for LinuxBIOS [linuxbios.org]. The article says an average of 14 minutes per flight were spent rebooting computers. Even 36 seconds per reboot is too much, and would be totally unacceptable if it were say, a navigation computer on a 737 with a hundred civilians on-board.

    Nasa has an interesting project called FlightLinux [nasa.gov] specifically geared for this sort of application. Unfortunately, they have yet to release code (export restrictions), but they supposedly use LinuxBIOS for their system.

    Of course, having software that never crashes (no pun intended) would be best, but it never hurts to have a system that can boot up in just a couple seconds anyway.
  • What do you expect (Score:3, Interesting)

    by gokubi (413425) * on Friday August 01, 2003 @07:36PM (#6592821) Homepage
    when the contracting agency can't acocunt for $1 trillion [azcentral.com]? That's more than the rest of the world spent on their military last year. With that kind of accountability, I'm amazed any project gets over 80% done.
  • by BWJones (18351) on Friday August 01, 2003 @07:37PM (#6592829) Homepage Journal
    Jeez, one would think that there would be built in redundancy so that if one system went down, it could be rebooted while the other system automatically takes over. Perhaps this is the way things are working, but the thought of rebooting during ACM makes me really nervous.

  • by Illserve (56215) on Friday August 01, 2003 @07:39PM (#6592842)
    Software like this should be able to reboot midflight without a hitch.

    Flight control software has been rebootable on the fly since the earliest days of the space program.

  • Ejection Seat (Score:3, Interesting)

    by rchatterjee (211000) on Friday August 01, 2003 @07:41PM (#6592854) Homepage
    If you're the test pilot you really got to hope they finished the code on the ejection seat at least, at 1,200 mph even a few seconds of reboot time is enough to turn you into part of the scenery at the test range.
  • Disturbed (Score:1, Interesting)

    by Anonymous Coward on Friday August 01, 2003 @07:49PM (#6592906)
    Am I the only programmer here that has a problem with writing software that powers "the most awesome killing machine"? I apologise to all the yeehaw types but I personally find that distasteful, to say the least.

    Question to physicists/biologists/chemists: Would you have a problem creating and refining nuclear/biological/chemical weapons?

    (Posted anon. to avoid the right wing moderators killing my account.)
  • by Anonymous Coward on Friday August 01, 2003 @07:50PM (#6592911)
    It disturbs me in that it's the sharp end of the system. A military aircraft would be pointedly useless if during its whole developmental process everyone skirted around the objectives of the thing; that is, to blow stuff up over there, while you're sitting here, and come back. that does involve killing people quite often.

    What disturbs me too is slashdot reporting. The article wasn't "about" the system needing reboots in flight, that was just one thing mentioned. The article was "about" a piece of military hardware nearing completion. The slashdot front page description and the real article may as well have been about two entirely different subjects for all they share.
  • by realmolo (574068) on Friday August 01, 2003 @07:59PM (#6592965)
    The software required to run the Raptor is insanely complicated. The plane itself was ambitious, but the contorl systems are the real innovation. Give these guys a break. The fact that the thing flies at all is amazing. The fact that it does everything it was designed to do is unbelievable. So there are a few bugs to work out. That's how it goes. We're not talking about "normal" programming problems here- this is Real Life stuff.
  • by Black Parrot (19622) on Friday August 01, 2003 @07:59PM (#6592967)


    > Please consider having Slashdot do a quick search, esp in the last 2-3 weeks. Even if this is done at the submittor level, then they could avoid this. I have no doubt that most submittors would prefer to avoid this.

    Au contraire, I would guess that every time a story hits Slashdot about 9000 clowns immediately submit it again in hopes of duping the editors into a dupe.

  • Apollo 11 (Score:5, Interesting)

    by s20451 (410424) on Friday August 01, 2003 @08:09PM (#6593025) Journal
    Haven't read the article (typically of slashdot), but I do remember that the Apollo 11 computer nearly caused the first lunar landing to fail because it kept rebooting in-flight. Due to a configuration error that occurred shortly before flight, the computer repeatedly ran out of memory, but the software was designed so that the computer could reboot without catastrophe.

    You can read more here [nasa.gov].
  • written in ADA ! (Score:2, Interesting)

    by Anonymous Coward on Friday August 01, 2003 @08:41PM (#6593197)
    F-22 software is written in ADA, by people with experience in designing these types of systems. It is a different breed of software engineering. There are a ton of issues coordinating all the software and hardware subsystems.

  • by Anonymous Coward on Friday August 01, 2003 @09:21PM (#6593388)
    I work on fly-by-wire military aircraft (rotary wing, not fixed wing, but I presume the computer architectures are similar).

    There are typically 2 (sets of) computers on board these aircraft.

    The "flight control computers" actually fly the airplane. They are very reliable and are triply or quad redundant. They constantly monitor themselves for problems (such as bits changing in the onboard ROM chips). They reboot themselves if needed (which seldom happens). The "operating system" is just another piece of custom code. They are often compartmentalized so that a problem in one area of the computer (hardware or software) will not affect (or will have limited impact) on other more critical components.

    The "mission computers" are not designed to the same standard and may have none of the aforementioned features. They try to do complex things like target identification etc. When they fail, they can take out other connected systems, like the radios or displays - but you can still fly the airplane. In one of the machines I worked on, they had to install a button in the cockpit so the test pilots could reboot the mission computer!

    I don't know why we as an industry tolerate this situation (OK, I do - to save money). Test pilots are (understandably) very unhappy with the lack of reliability in these systems. As I'm sure most people reading this will realize, its a lot harder to fix a complex bit of code than it is to design in reliability in the first place.

    And BTW, it was mentioned above but not everyone read it: it doesn't take 36 seconds to reboot the computer. The article meant that over the course of a 1 - 2 hour flight, 36 seconds were spent rebooting the computer
  • by DesScorp (410532) <DesScorp.Gmail@com> on Friday August 01, 2003 @09:45PM (#6593467) Homepage Journal
    I'm an advocate for a strong defense, and always have been. And advanced weapons programs always have major bugs. I'm a veteran, and I follow defense issure pretty closely. With that said, now I say kill the F-22 program.

    Why? It's a problem program. It's been plagued with an abundance of serious unforseen engineering problems from the very beginning. This is just the latest one made public. Past problems have included repeated instances of various parts of the fuesalage (especially some wing and tailparts) cracking. Cost overruns have become endemic. When the ATF program (Advanced Tactical Fighter) was first launched in the mid-80's to find a successor to the legendary F-15 Eagle, the Air Force set a goal of a flyaway cost of no more than 35 million per copy. The cost is now up 200 million a copy, and before it goes into production, the F-22 might cost a quarter of a billion dollars FOR A SINGLE FIGHTER. No matter how rich a nation is, no Air Force in the world can afford to buy such fighters in effective quantities. Not even other Stealth projects have spiraled this far out of control. The F-117 NightHawk stealth fighter (really more of a small bomber), with a small inefficient production run of 64 aircraft, topped out at 61 million per copy.

    Granted, not all of the cost overrun problems are the fault of the Air Force or of Lockheed Martin. Congress keeps screwing around with the production schedule, and reducing the total buy, which drives up the cost per aircraft. But Congress has done so in large part for three main reasons:

    1- They ask "Do we really need this, or can upgraded F-15's do the job?" This is a valid question as no other nation, friend or foe, has an aircraft that equals the Eagle, save for Russia's SU-27 series of fighters. These have been produced in such small quantities that Congress still debates the need for an Eagle replacement.

    2- The number and seriousness of technical problems has made Congress reluctant to commit to the project fully. This crosses party lines, as in the past few years, several powerful Republicans have tried to kill the program on the grounds that the Raptor is a lemon. Democrats seeking money for non-defense programs have joined them.

    3- There are serious doubts emerging that the Raptor's massive complexity can ever truly be managed in an efficient manner. There are concerns that, even if the aircraft becomes operational and initial bugs are worked out, the aircraft will be unreliable, becoming what the Air Force calls a "Hangar Queen"; it looks pretty on the floor, but if it can't go up in the air regularly, how good is it? The Air Force has had aircraft before that they REALLY wanted, but turned out to be so expensive and maintenance intensive that they had to be retired early. And excellent example is the B-58 Hustler supersonic bomber, which had impressive performance...when it wasn't broken down. It was retired after only 10 years of frontline service.
  • by ksheff (2406) * on Friday August 01, 2003 @10:25PM (#6593604) Homepage

    Given the cost of one of these things, they are certainly not going to trade safety and reliability. Military systems are designed to have redundant systems because they will be deployed in harsher conditions than civilian aircraft.

  • by Anonymous Coward on Friday August 01, 2003 @10:51PM (#6593703)
    While yes, ideally, we'd like software that you don't have to reboot, it's more important for software overall to be reliable than for it to be perfect (which is an impossible goal to achieve anyway, in the past, personal recollections to the contrary, and even more so now).

    One of the interesting ideas I've heard has focused on making recovery from errors an integral part of the software design at every level. To an extent, safety-critical systems already use a number of techniques to recover from errors, rather on relying on perfection on the part of the human programmers (which is a pretty silly bet to make).

    Just think about how you go through your own life. The human "operating system" isn't 100% perfectly reliable, but it's very robust at recovering from errors. Instead of striving for an impossible goal like perfection, systems are being designed to be less brittle. This approach is both more pragmatic and more robust, oddly enough.
  • by theolein (316044) on Friday August 01, 2003 @10:51PM (#6593706) Journal
    By the time this thing ever gets into the air the only probable foes that it will ever face will be either SU-27 derivates or Mig-29 derivates, both of which cost far less than the F-22.

    In pure features the Su-27 is an amazing plane. Anyone who has ever seen the Su-27 do the cobra [lucia.it] manouver or the thrust vectored Su-30MKI or Su-35 do the 360 degree Kulbit manouver can attest to what these planes can do in close air combat. These are extreme manouvers that western planes cannot do for the simple reason that the engines in western planes receive no air at such high angles of attack and therefore often flame-out or stall. Not only this but the newer radars on the Su-30s and missiles are longer ranging than just about anything the west has with the exception of the F-14's AIM-54 Phoenix. As for stealth, newer Su-30's are coated with radar absorbant paint which reduce the advantages that a dedicated stealth fighter such as the F-22 would have in BVR combat.

    In the hands of a good pilot I very much doubt that the Su-30 would automatically lose in combat. That however is the crux of the matter: Pilot training.

    This has always been something that has been much better in the west with advanced simulators, top gun style combat training and long hours of aircraft experience. It is and has been a fallacy to believe that more modern high tech will always win the battle. It is almost always the quality of the pilots that decided the battle.

    There is a good example of an air combat situation atht happened in the first gulf war. The only western plane to be shot down in air combat was an F-18 on an attack mission that was intercepted by an obviously experienced Iraqi Mig-25 pilot. The Mig-25 was already obsolete then in terms of technology but the sheer speed of the plane (Mach 2.8+) is unmatched by any other fighter. The Mig-25 went on after shooting down the F-18 to buzz an EF-111 raven that was providing ECM for the mission causing the raven to have to manouver to avoid the incoming missiles and drop back from the attack mission which was then unprotected by ECM and subsequently another F-18 was shot down by a SAM. No less than two F-15's and two F-16's all attempted to intercept the Mig-25, two of them firing missiles, but the Mig-25 used it's tremendous speed advantage to easily avoid the interceptors and reach its base.

    This shows what a good plane , not necesserally the utterly most modern, can do in the hands of a good pilot. IMO the F-22 is an overexpensive white elephant.
  • by jjohnson (62583) on Friday August 01, 2003 @11:00PM (#6593734) Homepage
    Given the money already spent, is is at all plausible to shelve the program, write off the development costs, and come back in ten years hoping to make an economical plane using what was learned? Maybe the Raptor will cost a quarter billion, but surely the engineers have learned a hell of a lot and solved a lot problems no one foresaw.

    In other words, don't buy Raptors: buy the engineers, and let them try again, the wiser for the experience.
  • by JamMasterJGorilla (629611) on Friday August 01, 2003 @11:56PM (#6593937) Homepage
    I was on an airbus flight leaving Dulles to San Francisco in the middle of the dot com days. We hit maximum thrust on the runway the front wheel lifted off then the plane shutdown. The pilot had to "Reboot" the plane (his words). First we had to sit there for 15 minutes while the brakes cooled.... Then the best part cam, they called in the mechanics to fix the computers..... Now this plane was filled with computer people and only one got off. I was sitting in the first business class seat at the isle so I had a good view of the cockpit while the mechanics worked on the computers. They pulled several avionics parts out of the plane (about the size of a ammunition box) then replaced them while taking to the technicians in San Francisco. About 2 hours later we took off. I'm still alive today.
  • by dtake (653890) on Saturday August 02, 2003 @12:35AM (#6594059)
    "It makes me wonder why the military has less stringent requirements."

    Money.

    Maybe because the Pentagon has too much money. The recently approved defense budget is $400 billion, not including the continuing cost of the wars in Afghanistan and Iraq and whoever we invade next.

    Is this money for the "war on terror"? Nope, as the first figure on this page [d-n-i.net] shows, most of the recent increases in the defense budget occured before 9/11/01.

    Further down the page, you will see how the Pentagon can't even pass a minimal annual audit, how increases in the defense budget create pressure for more increases into the future, and how pre-production cost estimates are usually much lower than the actual cost. This is particularly relevant today as there are many projects in the pre-production stage now.

    This information was put together by Chuck Spinney, who worked in the Pentagon's Office of Program Analysis and Evaluation for over 30 years.

  • Nothing new here (Score:4, Interesting)

    by dbrower (114953) on Saturday August 02, 2003 @12:44AM (#6594085) Journal
    When I started doing OS programming, there was a story going around about the then-new F-18's display computer. The symptom being reported was 'under such and such conditions, the display flickers'. It turned out it had gotten into some mode where it was rebooting nearly constantly. (AMD 2900 bit/slice processor, if I recall).

    This was 1980.

    It got fixed.

    -dB

  • Re:Faulty specs (Score:3, Interesting)

    by Mr. Feely (23410) on Saturday August 02, 2003 @01:06AM (#6594162)
    I used to work on avionics software and one of the biggest beefs of our main liason to the regulatory agencies was that there is currently no approved standard for generating system requirements. As a result there is no agreed-upon method for dealing with this single point of failure. In contrast, there is a well-defined and approved standard for software development: DO-178B.

    This individual claimed that most of the mishaps she was aware of that were attributed to software were in fact due to faulty system requirements, and I have no reason to doubt her. Unfortunately I don't remember any specific cases that she cited.
  • Re:WRONG (Score:1, Interesting)

    by Anonymous Coward on Saturday August 02, 2003 @05:27AM (#6594772)
    >Besides, I'd love to see three sets of hardware (all totally different) run the *same* software. Without any modification.

    If you insist...

    Slackware for iBook [nathanr.net]
    Slackware for Sparc [netunix.com]
    Slackware for PC [slackware.org]
  • by Suslik (59646) on Saturday August 02, 2003 @06:13AM (#6594843) Homepage
    It's not quite that simple. :-) If you are running a safety-critical system such as the arming system or stores management system then safety and procurement regulations require you to use a high-integrity operating system / programming runtime. Given the four levels of integrity defined in RTCA DO-178B (the bible for USA avionics software), this sort of software runs at Levels A or B (high integrity); even cut-down and much-tested Linux is only really suitable for Levels C or D.

    If you want software that does not crash during operation, you have to remove subroutine recursion and dynamic memory allocation so that you can guarantee an upper bound on the stack and pool usage at any time. If you're using multiple processes then you need some way of preventing deadlock and livelock. Whether you use C, C++ or Ada, you have to enforce these conventions in some way.
    See the SPARK Ada site [sparkada.com] for an example of how you do this. But fundamentally, to write software that is crash free you cannot take an off-the-shelf language and off-the-shelf OS and simply hope for the best.

  • by azorka (694470) on Saturday August 02, 2003 @07:10AM (#6594914)
    What about MiG 1.42 MFI [fighter-planes.com] Cheaper and better than F-22.

Nothing succeeds like success. -- Alexandre Dumas

Working...