Microsoft Fuzzing Botnet Finds 1,800 Office Bugs 111
CWmike writes "Microsoft uncovered more than 1,800 bugs in Office 2010 by tapping into the unused computing horsepower of idling PCs, a company security engineer said on Wednesday. Office developers found the bugs by running millions of 'fuzzing' tests, a practice employed by both software developers and security researchers, that searches for flaws by inserting data into file format parsers to see where programs fail by crashing. 'We found and fixed about 1,800 bugs in Office 2010's code,' said Tom Gallagher, senior security test lead with Microsoft's Trustworthy Computing group, who last week co-hosted a presentation on Microsoft's fuzzing efforts at the CanSecWest security conference. 'While a large number, it's important to note that that doesn't mean we found 1,800 security issues. We also want to fix things that are not security concerns.'"
If only this was easier... (Score:3, Interesting)
This is a great methodology of testing but to be honest I'm not sure it is within the scope of most software firms. While I'm sure we could all drop entirely random data into a parser and see if it fails, to REALLY conduct a test you have to do the same thing broken down by data element in the file format and then for each of those test both realistic and unrealistic test cases.
Then you throw on top of that UI and Web-Page fuzzing and you now have to somehow hook every element on a site and throw in random data which is not realistic with a large rich application.
Re:New bugs (Score:2, Interesting)
Yeah, just like the numerous regressions I see in the Linux kernel, WINE, Ubuntu releases etc.
Re:xkydgtufhlofhil (Score:3, Interesting)
ghulkgiplgbvihlnk luioguilgil.bjohj110-o; Huto;bn
I don't understand this Score:4 Insightful comment. Can someone explain?
Or user a sales. (Score:2, Interesting)
It is an alternative to the monkey test: Take a sales person from across the ahlloway and let him click on your application. If it does not crash or give absurd error messages you can do the actual testing.
GIGO!
One would think that this is the case... (Score:3, Interesting)
What you describe is "smart" or "generational" fuzzing, where you have a detailed knowledge of the target that you are fuzzing. The thing is, dumb (mutational) fuzzing is still effective. Very effective. Check out Charlie Miller's CanSecWest presentation - An analysis of fuzzing 4 products with 5 lines of Python
http://securityevaluators.com/files/slides/cmiller_CSW_2010.ppt [securityevaluators.com]
In 3 weeks of (really) dumb fuzzing, 174 unique crashes in PowerPoint were discovered.
Re:If only this was easier... (Score:3, Interesting)
Re:If only this was easier... (Score:3, Interesting)
Its only a great model for testing if you've exhausted the extensive list of known bugs that people hit every day under common circumstances.
Finding bugs in the file format is great and all, but fixing the bugs that users actually see every day is far more important and you can reset assured it will be released with a bucket load of very obvious bugs that should have been fixed rather than dicking around throwing random data at it.
I know there are potential security issues to deal with and those are important, but they still aren't as important as the users experience with the software and actually getting their own job done. Saying 'don't open word docs from someone else until this is fixed!' is a lot more practical than hearing that person say 'I'm not using Word, this retarded table layout bug is pissing me off, can we find something else to use instead of Word?'
I'm not saying they shouldn't, I'm just saying their priorities are wrong on a scale that is hardly imaginable.
As far as being realistic with a large rich application ...
Citation Needed.
Given enough processors sitting around the size of the application or its feature set becomes of little concern. It may take longer but thats no excuse to not do it.
Re:Speaks to the complexity (Score:3, Interesting)
Re:If only this was easier... (Score:3, Interesting)
I wouldn't knock what they're doing. As we've recently seen with Adobe, exploits in the payload format can be used to manipulate users and even launch code. And remember how we used to be all panicky about Word macro exploitations until the defaults were changed to shut them off? "Good times", indeed.
Consider that Microsoft dominates the market, and that the ".DOC" format is widely accepted across companies. Nowadays .DOC files are readily passed by email filters, web filters, etc. Office workers open them in previewers and Word without giving a second thought to security.
A buffer overrun or other fault in the handling of .DOC files could offer a hacker a way to deliver a malicious payload through channels that are today trusted worldwide. For all we know, these could already be exploited by phishing attacks.
It's definitely worth Microsoft's time and effort to execute these tests.
Re:wow imagine that (Score:3, Interesting)
I don't know of anyone who does regular fuzzy testing. Everyone that matters does unit testing.
Just FYI, Microsoft does fuzz testing in all areas of business, not just Office. The "news" here is really that the Office fuzz testing is done with a cluster of the developers' own computers. (Although it's definitely a good story to get out to all the shitty software houses out there that don't already do fuzz testing.)
When I worked in Xbox game testing back when the Xbox 360 was shiny and new, we had a large pile of Xbox 360s that did nothing but fuzz-testing of new titles by feeding them random controller input.
Re:xkydgtufhlofhil (Score:2, Interesting)
I was gratified when users came to me complaining that they got an error message about time travel not being permitted. Ha! I *knew* it was a good idea to write a test for the end of an appointment being before the beginning of the same appointment. I don't even want to *think* about the bugs that would have ensued if that data had got into the database. The routine that checks whether a room is free at a certain time wouldn't have handled it correctly, that's for certain.
You're not just paranoid. The data really are out to get you. You have to be ready for them, ready for *anything* they may throw at your code. If you're not careful, they'll get you.