Microsoft Fuzzing Botnet Finds 1,800 Office Bugs 111
CWmike writes "Microsoft uncovered more than 1,800 bugs in Office 2010 by tapping into the unused computing horsepower of idling PCs, a company security engineer said on Wednesday. Office developers found the bugs by running millions of 'fuzzing' tests, a practice employed by both software developers and security researchers, that searches for flaws by inserting data into file format parsers to see where programs fail by crashing. 'We found and fixed about 1,800 bugs in Office 2010's code,' said Tom Gallagher, senior security test lead with Microsoft's Trustworthy Computing group, who last week co-hosted a presentation on Microsoft's fuzzing efforts at the CanSecWest security conference. 'While a large number, it's important to note that that doesn't mean we found 1,800 security issues. We also want to fix things that are not security concerns.'"
New bugs (Score:5, Insightful)
I wonder how many "new" bugs they'll create by fixing the found bugs.
Anyway, nice to see that they're performing fuzzing tests, not enough people/companies do that. There's also quite little tool support for it.
Re:Hey, Microsoft! (Score:5, Insightful)
Re:If only this was easier... (Score:5, Insightful)
As with all testing tools, the more of them you use, the better. There are many reasons why you don't want to employ all tests, e.g. lack of knowledge, lack of manpower, lack of money or lack of time. The good thing is that if you can get them automated, then they quickly become affordable.
For an example: I was thinking if it was wise to put findbugs (which works on compiled byte code) next to checkstyle (which works on source code level) in my Java project. Obviously I put them both in; they duplicate bugs but who cares ? I'll just look at checkstyle first and findbugs second. If I can put in a pre-build fuzzing component I probably will.
But fuzzing tools are different than unit tests. Fuzzing can never cover every nook and cranny. They will produce reports that are much less readable, and that cannot be directly tied to particular events (e.g. during regression testing). If anything, they'll put some pressure on developers to put in more unit tests; if the fuzzing tool finds many bugs in a component, it should be a good indicator that even the basic unit tests have not been created.
Re:Speaks to the complexity (Score:5, Insightful)
Your point being? In 10 years since I started using it, I still don't know all the Vi commands and Emacs is so daunting I never even attempted it.
Re:xkydgtufhlofhil (Score:4, Insightful)
that doesn't mean we found 1,800 security issues (Score:3, Insightful)
it's important to note that that doesn't mean we found 1,800 security issues.
"...we have absolutely no idea where THOSE are."
No surprise, with that "format"! (Score:4, Insightful)
Have you even seen the “specification” that MS tried to make a standard. It’s a horribly convoluted mess, that can only be described as an upside-down pyramid of always patching new stuff onto the old framework, while never doing a needed complete re-design. Like Windows ME.
Hey Microsoft! If there are more bugs than features in your file format, maybe you should do a re-design, hm? ;)
Dumbest possible way to not find errors (Score:3, Insightful)
Remember the very obvious maxim of Dykstra: testing can only tell you there ARE errors, it can't tell you there AREN'T errors.
Randomly poking at data only find you the very dumbest errors. It takes some real thinking and mulling to realize, hey, if a xml field crosses this buffer boundary, and the last 4-byte Unicode code was cached, it's going to get bashed by the next 3-byte escape code. Or 255 bytes of code-page Yen symbol (255) followed by a 254 will lead to sign-extension and access to an address in the kernel trampoline DLL. Those kind of combinatorial errors are not going to be discovered by random poking at the data.
So they're going to (and have) given everybody a false sense of security, when the basic method can do nothing of the sort. it can only fin errors of the most trivial sort. It can't find errors that thousands of unemployed Russian hackers can dream up of testing for, and it can only FIND errors, not tell you there aren't an unlimited number of remaining errors.
It's not a botnet. (Score:3, Insightful)
Wait, I suppose it could be a botnet, if MS's IT department distributed the required software by exploiting security holes in the victim OS instead of just using admin rights to install the new app. Come to think of it, that might be easier
Re:wow imagine that (Score:3, Insightful)
Yes, I've taken that class.
I'm not talking about testing, I'm talking about design. If you expect a URL in a field and someone puts executable code in there, you should not be executing the code - you should be rejecting the URL. Data of that nature should not be put in a memory area where an instruction can be sent to run it.
Stack overflows, buffer underruns, and things of that nature are not things that should be caught in testing. They are things that should be prevented in the first place. If your code can't write data from strangers in places it can execute it, you can't be caught with your pants around your ankles when someone sends you executable code in a text field.
I'm not saying this testing is a bad thing, it's great, and necessary, and wonderful, and all that! But I sincerely hope Microsoft learned the lesson and Office 2012 or whatever the next version is will at least get some protected mode lovin' so they can separate data space from execution space and stop crossing the streams.
Maybe then Patch Tuesday will stop being so darned exciting.
Re:xkydgtufhlofhil (Score:3, Insightful)
Well color me red, here I thought this kind of testing should have been done prior to release. Guess the new model of software development is to have the users discover the bugs (can I get a smiley on this) instead of paying a QA team to test.
No, color you stupid. Office 2010 hasn't been released yet.
Nice try though.