Code Quality: Open Source vs. Proprietary 139
just_another_sean sends this followup to yesterday's discussion about the quality of open source code compared to proprietary code. Every year, Coverity scans large quantities of code and evaluates it for defects. They've just released their latest report, and the findings were good news for open source. From the article:
"The report details the analysis of 750 million lines of open source software code through the Coverity Scan service and commercial usage of the Coverity Development Testing Platform, the largest sample size that the report has studied to date. A few key points: Open source code quality surpasses proprietary code quality in C/C++ projects. Linux continues to be a benchmark for open source quality. C/C++ developers fixed more high-impact defects. Analysis found that developers contributing to open source Java projects are not fixing as many high-impact defects as developers contributing to open source C/C++ projects."
Not a surprise (Score:4, Insightful)
Managed langauges (Score:5, Insightful)
Java project developers participating in the Scan service only fixed 13 percent of the identified resource leaks, whereas participating C/C++ developers fixed 46 percent. This could be caused in part by a false sense of security within the Java programming community, due to protections built into the language, such as garbage collection. However, garbage collection can be unpredictable and cannot address system resources so these projects are at risk.
This is especially amusing in light of all the self-righteous bashing that C was getting over OpenSSL's problems. Seems it's true that using a "safe "language just makes the programmer lazy.
Not a surprise, but no reflection of O/S vs Prop. (Score:5, Insightful)
First, we shouldn't confuse Coverity's numerical measurements with actual code quality, which is a much more nuanced property.
Second, this report can't compare open source to proprietary code, even on the narrow measure of Coverity defect counts. In the open source group, the cost of the tool is zero (skewing the sample versus the commercial world) and Coverity reserved the rights to reveal data. Would commercial customers behave differently if they were told Coverity might reveal to the world their Coverity-alleged-defect data?
Again, having good Coverity numbers can't be presumed to be causally related to quality. For example, Coverity failed to detect the "heartbleed" bug, demonstrating that the effect of bugs on quality is very nonlinear. 10 bugs is not always worse than 1 bug; it depends on what that one bug is.
Ah, Coverity (Score:5, Insightful)
Coverity: Hey you, proprietary software developer with the deep pockets. Yeah, you. We've got this great tool for finding software defects. You should buy it.
Proprietary software developer: get lost.
Coverity: Hey, open source dudes, we've got this great defect scanner. Want to use it? Free of course!
Open source dudes: Meh, why not?
Coverity: Hey proprietary software developer, did we mention those dirty hippie neck beards are beating the stuffing out of you in defect (that we detect)-free code?
PSD: Fine, how much?
Re:Not a surprise (Score:4, Insightful)
bugs, like DRM?
Re:Managed langauges (Score:3, Insightful)
Resource leak in Java = DoS, as mentioned already
Resource leak in C = Heartbleed.
Personally, I'd rather my application crash than expose my private keys and other data that was supposed to be encrypted.
Re:Not a surprise (Score:2, Insightful)
Well - at least it was found...
With closed source at the other hand..
Well - let's say the bad guys still have great day's for a loooong time...
Re:Not a surprise, but no reflection of O/S vs Pro (Score:5, Insightful)
are you sure about that?
that's valid C#, all you need to do is inject something like that into the codebase and let the JIT compile it (using all the lovely features they added to support dynamic code) and you're good to get all the memory you like.
Now I know the CLR will not let you do this so easily, but there's always a security vulnerability lying around waiting to be discovered that will, or an unpatched system that already has such a bug found in any of the .NET framework, for example this one [cisecurity.org] that exploits... a "buffer allocation vulnerability", and is present in Silverlight. [mitre.org]
The moral is ... don't think C programs are somehow insecure and managed languages are perfectly safe.