Forgot your password?
typodupeerror
Programming Botnet Bug Microsoft IT

Microsoft Fuzzing Botnet Finds 1,800 Office Bugs 111

Posted by timothy
from the running-through-the-possibilities dept.
CWmike writes "Microsoft uncovered more than 1,800 bugs in Office 2010 by tapping into the unused computing horsepower of idling PCs, a company security engineer said on Wednesday. Office developers found the bugs by running millions of 'fuzzing' tests, a practice employed by both software developers and security researchers, that searches for flaws by inserting data into file format parsers to see where programs fail by crashing. 'We found and fixed about 1,800 bugs in Office 2010's code,' said Tom Gallagher, senior security test lead with Microsoft's Trustworthy Computing group, who last week co-hosted a presentation on Microsoft's fuzzing efforts at the CanSecWest security conference. 'While a large number, it's important to note that that doesn't mean we found 1,800 security issues. We also want to fix things that are not security concerns.'"
This discussion has been archived. No new comments can be posted.

Microsoft Fuzzing Botnet Finds 1,800 Office Bugs

Comments Filter:
  • by Anonymous Coward on Friday April 02, 2010 @04:13AM (#31705036)

    ghulkgiplgbvihlnk luioguilgil.bjohj110-o; Huto;bn

    • Re: (Score:3, Interesting)

      by troll8901 (1397145) *

      ghulkgiplgbvihlnk luioguilgil.bjohj110-o; Huto;bn

      I don't understand this Score:4 Insightful comment. Can someone explain?

      • Re:xkydgtufhlofhil (Score:5, Informative)

        by sucker_muts (776572) <sucker_pvn@@@hotmail...com> on Friday April 02, 2010 @05:40AM (#31705212) Homepage Journal

        don't understand this Score:4 Insightful comment. Can someone explain?

        Even though your name does look quite suspicious, I'll try to explain anyway.

        The parent is showing how fuzzing works:
        Using random 'data' to test the various functions of software, so we can find out if a certain piece of input triggers undesirable behavior.

        In this case you could say that he's not only giving an example, but is testing the slashdot user comments code as well.

        But it's perhaps more an attempt at humor. :-)

        • Re: (Score:2, Funny)

          by Anonymous Coward

          Windows: "It's not a bug, it's a feature."

          GNOME: "It's not a bug, it's a design decision."

        • Re:xkydgtufhlofhil (Score:5, Informative)

          by jonadab (583620) on Friday April 02, 2010 @08:01AM (#31705558) Homepage Journal
          Except that, in most cases, random letters in the ranges a-z and A-Z are not where you're going to find most of your problems. The major sources of bugs that can be uncovered by random data are assumptions that programmers (sometimes subconsciously) make about what the data are going to be like.

          The most obvious of these are assumptions like "a newline can't occur in a single-line field" (a mistake web developers often make, because they assume the data are coming from an HTML input element that only allows single-line data; but an attacker can in fact send anything they want in an http request), or "nobody's going to have a single-quote character in their name" (hello, SQL injection attack). This sort of thing is probably not a major factor in Office, because it's common for documents to have those kinds of characters in them. There might be a couple of weird old control characters (like the ASCII NUL, 000), but those bugs were probably found aeons ago.

          A second major category of problematic assumptions assumptions has to do with languages and code pages and character sets. When software that was written to assume a particular character set (like ASCII, or Latin-1) or even just one code page at a time (like, whichever one is the system default) has to be extended to support more (like, especially, Unicode), you run into all kinds of nasties. Again, though, Office probably had to deal with these issues a couple of versions ago. They may have found a few more, but at this point it's probably not the most fertile ground any more.

          When you're dealing with file formats, however, there are also things like "the value at offset 0x003C from the beginning of the object header contains the size of the object, which can never be more than 0xFFFF" and "an object can embed another object by referencing it, but there are never any circular references, because the software doesn't allow the user to put an object inside itself". These sorts of assumptions pop up every time you write or change code that reads a file format, so they never go away really. This sort of thing is probably *most* of what the Office team found, I suspect.
          • Re:xkydgtufhlofhil (Score:4, Informative)

            by Helen O'Boyle (324127) on Friday April 02, 2010 @10:55AM (#31707016) Journal

            "nobody's going to have a single-quote character in their name" (hello, SQL injection attack)

            Hey, I resemble that remark! And yes, it's resulted in chuckles over the years. Microsoft, DevelopMentor, random e-commerce sites... many have fallen to the Irish. When talking to security professionals, I introduce myself as "the woman whose name is a SQL injection attack", and it seems to help them remember me.

            • Re: (Score:2, Interesting)

              by jonadab (583620)
              It isn't just names, either. Apostrophes and other special characters show up all kinds of places in data where naive programmers tend to imagine they won't appear. Did you know a less-than symbol can show up in a book title? Oh, yes, and if you aren't doing entity-encoding when you build HTML from the data you will get a surprise. With experience, you eventually learn to write the code so that it will either accept those characters as part of the data and handle them as such, or in cases where that's n
        • Re: (Score:1, Offtopic)

          by jank1887 (815982)

          somewhat relevant xkcd:

          Exploits of a Mom
          http://xkcd.com/327/ [xkcd.com]

        • Even though your name does look quite suspicious, I'll try to explain anyway.

          Thank you for your explanation.
          And your benefit of the doubt.

        • Re: (Score:3, Funny)

          by mobby_6kl (668092)

          >In this case you could say that he's not only giving an example, but is testing the slashdot user comments code as well.

          It's testing not just the user comments code, but also the moderation system code and the moderators themselves. In this case, it appears that he found a bug which causes the comment to be moderated Insightful by providing a certain combination of random characters as input. I will now attempt to replicate this problem.

          ______TEST DATA FOLLOWS______
          TvaHokVAwgZGLrzPnDsIzHnKwuOOQEgaFskFJx

      • Re: (Score:3, Informative)

        by msclrhd (1211086)

        Fuzzing is a technique where you modify the data sent to a file, protocol or data parser (e.g. code that reads an xml file) by changing random bits. Thus, if you have a 'text' command, a fuzzer could change that to 'next', or if you have a quoted striing "test", the fuzzer could change the end quote to something else, e.g. ' "tests '.

        Hence, what you can end up with is something that looks like random garbage.

        • Or user a sales. (Score:2, Interesting)

          by leuk_he (194174)

          It is an alternative to the monkey test: Take a sales person from across the ahlloway and let him click on your application. If it does not crash or give absurd error messages you can do the actual testing.

          GIGO!

          • by mOdQuArK! (87332)

            It is an alternative to the monkey test: Take a sales person from across the ahlloway and let him click on your application.

            Last time I tried that, it took me forever to get the feces out of the keyboard.

    • by jackal40 (1119853)
      Well color me red, here I thought this kind of testing should have been done prior to release. Guess the new model of software development is to have the users discover the bugs (can I get a smiley on this) instead of paying a QA team to test.
      • Re: (Score:3, Insightful)

        by nmb3000 (741169)

        Well color me red, here I thought this kind of testing should have been done prior to release. Guess the new model of software development is to have the users discover the bugs (can I get a smiley on this) instead of paying a QA team to test.

        No, color you stupid. Office 2010 hasn't been released yet.

        Nice try though.

        • by jackal40 (1119853)
          Hmmm, might want to get your sarcasm detector fixed (can I get a smiley on this - refers to the beta test feedback system). Guess you're unfamiliar with it.
  • by geminidomino (614729) on Friday April 02, 2010 @04:15AM (#31705040) Journal

    "We also want to fix things that are not security concerns."

    It's 5AM EST. April Fools' day is over everywhere but a few pacific islands. Give it up already.

    • Re: (Score:3, Funny)

      by somersault (912633)

      While a large number, it's important to note that that doesn't mean we found 1,800 security issues

      Don't worry, we all know that you haven't fixed any security issues.

    • Re:Hey, Microsoft! (Score:5, Insightful)

      by PolygamousRanchKid (1290638) on Friday April 02, 2010 @04:21AM (#31705056)
      Note that he said "want" and not "will".
      • by game kid (805301)
        ...which brings us back to where we started; they've clearly fixed n security issues, where 0 <= n <= 1800. :)
    • Re: (Score:2, Funny)

      by Arancaytar (966377)

      It would actually be believable except for the "also". :P

    • by MacWiz (665750)

      There were only 1800 bugs. Not like it was anything serious.

      Elsewhere in this "issue" of slashdot, we have the story of Microsoft's OOXML failing the standards test because they've only bothered to address about 5% of the issues there.

      You just had another "emergency" security update for IE a few days ago.

      Microsoft wants the world to run on its software and most of the business world does, not to mention the government. But they are willing to sell it to you BEFORE they bother to take a look and see how well

  • New bugs (Score:5, Insightful)

    by El_Muerte_TDS (592157) <elmuerte.drunksnipers@com> on Friday April 02, 2010 @04:17AM (#31705046) Homepage

    I wonder how many "new" bugs they'll create by fixing the found bugs.

    Anyway, nice to see that they're performing fuzzing tests, not enough people/companies do that. There's also quite little tool support for it.

    • Re: (Score:2, Interesting)

      by GF678 (1453005)

      I wonder how many "new" bugs they'll create by fixing the found bugs

      Yeah, just like the numerous regressions I see in the Linux kernel, WINE, Ubuntu releases etc.

      • I wonder how many "new" bugs they'll create by fixing the found bugs

        Yeah, just like the numerous regressions I see in the Linux kernel, WINE, Ubuntu releases etc.

        Why is this modded offtopic? It's cool and popular to poke fun at Microsoft but heaven forbid you point out Linux, WINE, and Ubuntu have regressions?

        • > Why is this modded offtopic?

          Because it is.

          > It's cool and popular to poke fun at Microsoft but heaven forbid you point
          > out Linux, WINE, and Ubuntu have regressions?

          Why is the fact that other software also has regressions relevant? Do you think that is news to anyone here?

    • Re:New bugs (Score:5, Funny)

      by beakerMeep (716990) on Friday April 02, 2010 @05:38AM (#31705206)
      fuzzing tools probably wont ever gain wide spread acceptance outside of the furry community though.
    • by anexkahn (935249)
      My teachers in school used to say for every bug you fix, you have a 50% chance of creating two more :)
  • Hmmm (Score:1, Flamebait)

    by jocabergs (1688456)
    Not sure if I buy the part about them trying to fix the non-security issue bugs... I think the proposed fix for bugs in 2007 is $300 for 2010, but its by no means a comprehensive fix.
    (I'm coming from a bitter place, I've been stuck going through idiotic publisher files for the last 3 days and I'm certain it was designed by monkeys(or for them))
  • by Manip (656104) on Friday April 02, 2010 @04:32AM (#31705072)

    This is a great methodology of testing but to be honest I'm not sure it is within the scope of most software firms. While I'm sure we could all drop entirely random data into a parser and see if it fails, to REALLY conduct a test you have to do the same thing broken down by data element in the file format and then for each of those test both realistic and unrealistic test cases.

    Then you throw on top of that UI and Web-Page fuzzing and you now have to somehow hook every element on a site and throw in random data which is not realistic with a large rich application.

    • by somersault (912633) on Friday April 02, 2010 @04:55AM (#31705104) Homepage Journal

      The whole point of the data is that it's unrealistic. There are a few tools out there for doing this type of testing, or easily modified to do it. I haven't used many testing tools but you could take something like Skipfish [google.com] and add in some fuzz testing pretty easily.

    • by owlstead (636356) on Friday April 02, 2010 @05:26AM (#31705178)

      As with all testing tools, the more of them you use, the better. There are many reasons why you don't want to employ all tests, e.g. lack of knowledge, lack of manpower, lack of money or lack of time. The good thing is that if you can get them automated, then they quickly become affordable.

      For an example: I was thinking if it was wise to put findbugs (which works on compiled byte code) next to checkstyle (which works on source code level) in my Java project. Obviously I put them both in; they duplicate bugs but who cares ? I'll just look at checkstyle first and findbugs second. If I can put in a pre-build fuzzing component I probably will.

      But fuzzing tools are different than unit tests. Fuzzing can never cover every nook and cranny. They will produce reports that are much less readable, and that cannot be directly tied to particular events (e.g. during regression testing). If anything, they'll put some pressure on developers to put in more unit tests; if the fuzzing tool finds many bugs in a component, it should be a good indicator that even the basic unit tests have not been created.

      • But fuzzing tools are different than unit tests. Fuzzing can never cover every nook and cranny.

        Neither will unit tests [c2.com].

    • Re: (Score:3, Informative)

      by SharpFang (651121)

      A fuzzer isn't really hard to write.

      Pick a word-based variant of Dissociated Press [wikipedia.org] that requires similarity a random number of words back/ahead and allows split on special characters (separators) besides whitespaces. Feed it a lot of your actual files. Actually, the amount of data it can produce may be vastly bigger than the amount of data it takes in, because it can jump back and forth in the input files recombining their fragments multiple times.

      Of course then you need a test unit that feeds the fuzz to y

    • Re: (Score:1, Informative)

      by Anonymous Coward

      This is a great methodology of testing but to be honest I'm not sure it is within the scope of most software firms.

      Microsoft runs huge (and I mean huge) server farms for all kinds of internal testing - unit tests for rolling builds, automated functional tests for the same, performance regression tests, compatibility tests (what if we run it on Vista without SP1 and with Office 2003 with latest updates installed?..) - you name it, it's there.

      But, even with all the servers, it still takes hours for a complete test run.

    • What you describe is "smart" or "generational" fuzzing, where you have a detailed knowledge of the target that you are fuzzing. The thing is, dumb (mutational) fuzzing is still effective. Very effective. Check out Charlie Miller's CanSecWest presentation - An analysis of fuzzing 4 products with 5 lines of Python
      http://securityevaluators.com/files/slides/cmiller_CSW_2010.ppt [securityevaluators.com]

      In 3 weeks of (really) dumb fuzzing, 174 unique crashes in PowerPoint were discovered.

      • by amorsen (7485)

        In 3 weeks of (really) dumb fuzzing, 174 unique crashes in PowerPoint were discovered.

        The fuzzing was dumb, but the picking of files as basis for the fuzzing was smart. Unfortunately Charlie Miller doesn't present a tool for doing that.

    • Re: (Score:3, Interesting)

      by BitZtream (692029)

      Its only a great model for testing if you've exhausted the extensive list of known bugs that people hit every day under common circumstances.

      Finding bugs in the file format is great and all, but fixing the bugs that users actually see every day is far more important and you can reset assured it will be released with a bucket load of very obvious bugs that should have been fixed rather than dicking around throwing random data at it.

      I know there are potential security issues to deal with and those are import

      • Re: (Score:3, Interesting)

        by plover (150551) *

        I wouldn't knock what they're doing. As we've recently seen with Adobe, exploits in the payload format can be used to manipulate users and even launch code. And remember how we used to be all panicky about Word macro exploitations until the defaults were changed to shut them off? "Good times", indeed.

        Consider that Microsoft dominates the market, and that the ".DOC" format is widely accepted across companies. Nowadays .DOC files are readily passed by email filters, web filters, etc. Office workers open

  • What's the connection with "botnet"?
    • by nacturation (646836) * <nacturation AT gmail DOT com> on Friday April 02, 2010 @04:44AM (#31705088) Journal

      FTFA:

      Microsoft was able to find such a large number of bugs in Office 2010 by using not only machines in the company's labs, but also under-utilitized or idle PCs throughout the company. The concept isn't new: The Search for Extraterrestrial Intelligence (SETI@home) project may have been the first to popularize the practice, and remains the largest, but it's also been used to crunch numbers in medical research and to find the world's largest prime number.

      "We call it a botnet for fuzzing," said Gallagher, referring to what Microsoft has formally dubbed Distributed Fuzzing Framework (DFF). The fuzzing network originated with work by David Conger, a software design engineer on the Access team.

      Odd that they would call it that publicly, given the negative connotation of the word. I would have called it "fuzzy clouds grid computing" or something like that.

    • by Mathinker (909784) on Friday April 02, 2010 @04:48AM (#31705096) Journal

      Let me explain: Microsoft discovered that all of their desktop computers were zombied with malware, and after wresting control from the botnet C&C, decided to take advantage of this increased ability to remotely administer their computers to run QA tests, on the off chance there might be some need for it.

      </joke>

    • by benjamindees (441808) on Friday April 02, 2010 @05:11AM (#31705140) Homepage

      They had to infect the computers with Office 2010.

  • The problem is that they write such crappy code to begin with. There really is no good reason for that. I use Word 2007 at work and it is very buggy. If I had my way, I would not use it, even OOo is better.
    • Re: (Score:3, Funny)

      by swilver (617741)

      The same as I thought. Tip, meet iceberg.

    • The large Microsoft products seem to go through the same pattern. First version with a very large change kind of sucks but it works well enough to release and get bug reports on, and the second version is a polished, faster, more robust version of the same. See: Windows ME->XP, Vista->Win7, and now Office2007->Office 2010.
      • Windows ME was a descendant of Win95/Win98. Windows XP was a descendant of NT4/Win2K.
      • by jbengt (874751)
        Your ME example is wrong; ME actually bucks the trend by being a worse version of a previous product.
        ME was based on '95/'98 and XP was not based on ME.
    • by Blakey Rat (99501)

      I use Word 2007 at work and it is very buggy.

      Undoubtedly true.

      If I had my way, I would not use it, even OOo is better.

      "Better" in what sense? It's certainly not less buggy... hell, some components of OpenOffice.org (ok, I'll name it: Impress) seem to have never been actually tested.

    • by TheLink (130905)

      > I use Word 2007 at work and it is very buggy. If I had my way, I would not use it, even OOo is better.

      I agree Word 2007 is buggy. But OOo is NOT better. It's far buggier.

      FWIW, it's not so much the bugs in Word 2007 that annoy me than the way it does formatting and selection of certain stuff. Yes I know you can customize the behaviour but I still find I have to "battle" with it a bit too often.

      OpenOffice on the other hand has rather blatant bugs like these:

      Launch openoffice writer.
      Type three lines of: "

  • *pat in the back* good one Microsoft, now you test your software. What about now to change and respect standards, PLEASE.

  • Of Imprise Delphi 4 and Corel WordPerfect 9.

    Just kidding. Seems like a good initiative on Microsoft's part.

  • QuickCheck [chalmers.se]

    But for some reason random data testing is less popular for the other languages I'm familiar with.

  • Did they fix that bug where the useful menus get replaced by that horrible ribbon thing?

    I know there are downloads to revert to the menus, but can't do that at work.

  • Coming from a programmer's point of view, this is like saying a nurse used an alcohol swab before plunging a needle into your arm, to avoid you contracting anything....i think in today's world this is all pretty standard even for the smaller budget companies, most are unit test driven and have intensive test environments to stress test their apps. I have seen this even in a company as small as 3 programmers. Do they want a medal, seriously....i am thinking of saying something....maybe someone would care to

    • by natehoy (1608657)

      Programming 101: TEST YOUR GODDAMNED INPUTS!

      Programming 102: If you missed the lesson in Programming 101 where you have to test your inputs, you fail and have to repeat Programming 101. Some people never get out of Programming 102. I've worked with a few of them. "It ain't that pretty at all".

      This is clumsy after-the-fact testing at best, just throwing random garbage at the program and hoping to hit a condition. Having said that, I do want to applaud Microsoft for at least, finally, taking some steps

      • by Jer (18391)

        Computer Science 395 (Software Engineering): Remember how back in Programming 101 we told you how to perform testing on your code? Turns out we grossly oversimplified our discussion of how to go about doing that for pedagogical reasons. While it's nearly trivial to perform the test/debug cycle on code that you wrote for a class project that does one well-defined thing and will only ever be seen by you and your grader/professor, the scope of testing and debugging transforms radically when you attempt to s

        • Re: (Score:3, Insightful)

          by natehoy (1608657)

          Yes, I've taken that class.

          I'm not talking about testing, I'm talking about design. If you expect a URL in a field and someone puts executable code in there, you should not be executing the code - you should be rejecting the URL. Data of that nature should not be put in a memory area where an instruction can be sent to run it.

          Stack overflows, buffer underruns, and things of that nature are not things that should be caught in testing. They are things that should be prevented in the first place. If your c

          • Yet so many people take this class and then some, to learn UML, Unit testing,
            variants of stress testing, even how to stand their ground against unwanted elements in the environment, but one thing will never change, if the boss says to do it his way, you can either finally do it his way, or hit the highway, and usually this is were the problem lies.

            I have come across a few times in my life that the necessary steps were avoided on purpose and left many with critical bugs in their apps, only because of some hi

    • by BitZtream (692029)

      this is like saying a nurse used an alcohol swab before plunging a needle into your arm, to avoid you contracting anything....

      You clearly don't understand what fuzzy testing is.

      Fuzzy testing and unit testing are only related in that they are tests.

      Unit tests are well defined tests for known conditions.

      Fuzzy tests, if done perfectly, would appear to be random and thrown entirely random data at the applicaiton in order to increase the chances of finding out what happens when the app gets something the develop

      • Funny i read the word "buffer overflow" and to my knowledge this is
        considered part of the unit testing and not the fuzzy.
        As for many not doing fuzzy testing, well i never considered the 2 seperate
        issues in my unit testing, so you will have to pardon my lack of
        empathy for the situation

      • Re: (Score:3, Interesting)

        by Blakey Rat (99501)

        I don't know of anyone who does regular fuzzy testing. Everyone that matters does unit testing.

        Just FYI, Microsoft does fuzz testing in all areas of business, not just Office. The "news" here is really that the Office fuzz testing is done with a cluster of the developers' own computers. (Although it's definitely a good story to get out to all the shitty software houses out there that don't already do fuzz testing.)

        When I worked in Xbox game testing back when the Xbox 360 was shiny and new, we had a large pi

  • by Geminii (954348) on Friday April 02, 2010 @09:01AM (#31705928)

    it's important to note that that doesn't mean we found 1,800 security issues.

    "...we have absolutely no idea where THOSE are."

  • by Hurricane78 (562437) <deleted@slashd[ ]org ['ot.' in gap]> on Friday April 02, 2010 @10:14AM (#31706576)

    Have you even seen the “specification” that MS tried to make a standard. It’s a horribly convoluted mess, that can only be described as an upside-down pyramid of always patching new stuff onto the old framework, while never doing a needed complete re-design. Like Windows ME.

    Hey Microsoft! If there are more bugs than features in your file format, maybe you should do a re-design, hm? ;)

    • Re: (Score:1, Flamebait)

      by BitZtream (692029)

      i'm sorry, I must have missed something ...

      Where is your example of some successful software product without backwards compatibility?

      Contrary to popular belief, a 'complete rewrite' is almost universally a retarded idea, and any developer with more than a couple years experience knows this.

      When you're programming to get something done, its a little different than sitting in mommies basement rewriting your python script because you don't actually have anything else to do.

  • by Ancient_Hacker (751168) on Friday April 02, 2010 @10:26AM (#31706712)

    Remember the very obvious maxim of Dykstra: testing can only tell you there ARE errors, it can't tell you there AREN'T errors.

    Randomly poking at data only find you the very dumbest errors. It takes some real thinking and mulling to realize, hey, if a xml field crosses this buffer boundary, and the last 4-byte Unicode code was cached, it's going to get bashed by the next 3-byte escape code. Or 255 bytes of code-page Yen symbol (255) followed by a 254 will lead to sign-extension and access to an address in the kernel trampoline DLL. Those kind of combinatorial errors are not going to be discovered by random poking at the data.

    So they're going to (and have) given everybody a false sense of security, when the basic method can do nothing of the sort. it can only fin errors of the most trivial sort. It can't find errors that thousands of unemployed Russian hackers can dream up of testing for, and it can only FIND errors, not tell you there aren't an unlimited number of remaining errors.

    • by owlstead (636356)

      Yeah, well, they find bugs and fix them, regardless of all the other testing they perform. If they are not deprecating other tests that (may not be fully covered) then yes, I can see a problem here. And don't forget, even the dumbest bug can become a vulnerability.

  • It's not a botnet. (Score:3, Insightful)

    by NotBornYesterday (1093817) on Friday April 02, 2010 @10:52AM (#31706974) Journal
    It's distributed computing.

    Wait, I suppose it could be a botnet, if MS's IT department distributed the required software by exploiting security holes in the victim OS instead of just using admin rights to install the new app. Come to think of it, that might be easier ... [me scurries off to develop new easy-to-use set of malware-based admin tools].
  • Even though some of us would easily believe MS office has 1800 security issues that need fixing.(and in my opinion every crash due to malformed input is a security issue)
    I find it hard to believe they found 1800 of these by generating random data, what is far more likely is that they recorded 1800(or more) crash events
    and after fixing two or three programming errors(problematic hidden assumptions about the input) 1800 of them were not reproduced.
    This hardly counts as solving 1800 bugs.
    The technique itself i

Promising costs nothing, it's the delivering that kills you.

Working...