Oracle Has More Flaws Than SQL Server

jcatcw writes, "Next Generation Security Software Ltd. of Surrey, England, compared bugs in Oracle and SQL Server that were reported and fixed between December 2000 and November 2006. The tally: Oracle had 233; MS SQL had 59. The products compared were Oracle 8, 9, and 10g; SQL Server 7, 2000 and 2005. From the article: '[The head of the survey said,] "The results show that the reputation that Microsoft SQL Server had back in 2002 for relatively poor security is no longer deserved."' Oracle's response: 'Measuring security is a very complex process, and customers must take a number of factors into consideration — including use-case scenarios, default configurations, as well as vulnerability remediation and disclosure policies and practices.'"

Summary title is vague

[ArcherB]

MSSQL is a SQL Server. MySQL is a SQL Server. Oracle is a SQL Server. Please be more specific and explain which SQL Server you are talking about.

Granted, the summary does explain that the article does indeed refer to MSSQL Server, but please stop calling it just SQL Server. MSSQL Server != SQL Server

(OK, I feel better. What is the moderation for RANT?)

Oracle is more complex

[sitturat]

Anyone that has tried to read (or even tried to lift up) one of the oracle manuals knows that this is seriously feature-rich and complicated stuff. It would be more interesting to see how many bugs per line of code the two contenders have.

Oracle is right

[Josh Lindenmuth]

While the # of vulnerabilities is unacceptable, Oracle is right ... just comparing the # of bugs is not really valid. Now if Oracle has had more Severe security violations that Microsoft, it would be a different (and far more interesting) story. Oracle is still a more robust database, so one would expect there to be more bugs than another app with fewer modules and lines of code.

Re:translation

[HairyCanary]

I tend to agree. But Oracle does have a point. Trying to distill a security argument down to number of bugs is oversimplifying. The severity of the bugs, how easy they are to exploit, etc are all important to consider. Even more important in my opinion is how quick the vendor is at fixing them. If Oracle's average time to fix was 24 hours compared to six months for Microsoft, the 4:1 bug ratio is not such a big deal.

What, specifically, are those "bugs"?

[khasim]

Between December 2000 and November 2006, external researchers discovered 233 vulnerabilities in Oracle's products compared with 59 in Microsoft's SQL Server technology, according to NGSS. The study looked at vulnerabilities that were reported and fixed in SQL Server 7, 2000 and 2005 and Oracle's database Versions 8, 9 and 10g.

Let's see that again.

The study looked at vulnerabilities that were reported and fixed...

So, if it wasn't fixed, was it counted?

The results show that Microsoft's software development life-cycle processes appear to be working, he said.

Huh? Security is not about "software development life-cycle".

That's why you have almost daily updates of anti-virus software for Microsoft products.

In an e-mailed comment, an Oracle spokeswoman said the number of reported vulnerabilities in a product alone is not a measure of the overall security of that software.

Big time. One remote root vulnerability is worth 10,000 local app crash vulnerabilities.

"Measuring security is a very complex process, and customers must take a number of factors into consideration -- including use-case scenarios, default configurations as well as vulnerability remediation and disclosure policies and practices."

Yep. Because Ubuntu has, by default, no open ports. So it is, by default, 100% resistant to worms.

Remember, you can never count on a user applying a patch. Your system has to be as secure as possible in the default, unpatched, configuration.

Basing a product's security just on the number of vulnerabilities discovered and fixed may not be the best approach, said Pete Lindstrom, an analyst at Midvale, Utah-based Burton Group.

Not only is it not "the best approach", it is a fucking idiotic approach only used by morons who have no understanding of what "security" is.

It's not the number of bugs. It's what access can be gained by that bug and how easily it is to invoke that bug in the various "standard" configurations.

I dunno about that

[palladiate]

Have you ever USED MS-SQL? At least the cheese doesn't take 45 minutes to report what flavor it is under normal load conditions...

Re:translation

[SatanicPuppy]

It's typical MS fud. They LOVE to harp on how many bugs their competition has, but there is a hell of a lot more to it than quantity. Slammer [symantec.com] anyone?

Oracle is a huge robust database with lots of extremely security conscious clients. A high number of reported bugs and fixes shows that they're executing due diligence, and working to keep their system as secure as possible. MSSQL's low number of bugs suggests that Microsoft isn't digging hard into their code, but only waiting for big public flaws.

They used the same argument in claiming that IE was less buggy than Firefox (see this crappy article [informationweek.com]) and it's just as untrue in this case.
 

Re:Oracle is right

[Anonymous Coward]

Oracle is still a more robust database, so one would expect there to be more bugs than another app with fewer modules and lines of code.

Your definition of "robust" must be wildly different than mine. I tend to use Webster's definition, where "robust" means "capable of performing without failure under a wide range of conditions". Obviously, you seem to think that the more robust the software, the higher the bug count!

Re:translation

[Anonymous Coward]

I'm not an oracle person, but from my understanding oracle allows you to have finer grained security on data, stored procedures and so on than sql server. Perhaps the complexity of oracle compared to sql server is part of the reason there are more bugs.

Lets face it, a bug report can be anything from a misspelled error message to a gaping sa/root/admin (whatever oracle calls it) compromise.

Severity is important. For instance, most popular linux distros (minus gentoo) have quite a few security holes do to third party package inclusion. Often the holes are not severe, but they do make linux look artificially insecure compared to some other operating systems. If redhat pushed 90 updates a month at you and Microsoft only 35... well who looks less secure? How many were feature enhancements? How many did each vendor NOT include a fix for?

Disclaimer: My above reference to linux distros only includes bloated packages like redhat, suse, etc. Most people using these distros tend to do a "full install". I'm a mysql or sql server user whenever possible.

Often one could argue that smaller companies get less attention so a large number of vulnerabilities would indicate a very insecure product. Oracle is obviously smaller than microsoft as a whole. In this case, oracle gets a lot of attention as its used for large scale deployments as well as their *lovely* business practices.

Re:Oracle is right

[gregmac]

Comparisons of number of bugs are NEVER fair. The situation is even worse in a closed-source environment, because we may never actually see all the bugs that get fixed. Even in open source, we sometimes fix bugs in the code with filing a report. Sometimes bugs are filed for a misspelling in the user interface. Sometimes 4 or 5 bugs are reported based on behaviour alone, and upon inspection, there's really one root problem (maybe even something simple) that's causing all of those bugs, so one fix goes in and 5 bugs get closed. Does that count as 1 bug or 5? Do these studies of # of bugs take that into consideration?

Even calling something "severe" or not is a judgement call. I've seen many times a bug filed as severe only to have a developer look at it and refile it as trivial.

On top of all of this, it's not hard to "game" this system to make your company/project look better. Just raise your standards for what can be classified as a major vs minor bug (eg, file everything a bit lower than it normally would be). This standard is going to be set differently by different management teams and companies, so it's already skewed to try to compare. Someone trying to look like they have fewer bugs may also ask their team to refrain from filing bugs if they can (kind of like factories do with workplace accidents - they have incentive systems for employees/supervisors, part of that "We've gone X days without an accident" thing.. what really happens, is employees won't report accidents if they can get away with it because then they lose their incentives). At another company, they may have a policy to file bugs for EVERYTHING, so every change to the code requires a bug/feature ticket. What happens when you compare the # of "bugs" in these two companies?

Re:translation

[ZachPruckowski]

You're right. This survey is pretty messed up. I mean, we're comparing *bugs fixed*. Not bugs still open, or any measure of severity, or what got exploited, or any measure of turn-around time.

This is like saying that Fire Department A put out less fires than Fire Department B. That's nice, but what I really want to know is how long it took for the trucks to arrive, the size of the fires, and also if there are any houses that burned down before the Fire Department got there.

Re:What, specifically, are those "bugs"?

[Rich0]

While I agree with 95% of what you said, I'd take issue with this:

Ubuntu has, by default, no open ports. So it is, by default, 100% resistant to worms.

Not all worms require open ports to spread - a worm might target a low-level kernel flaw in the network stack (remember the ping-of-death?).

Re:Features?

[ergo98]

I use SQL Server 2005 at work and it's pathetic.

My spidey senses tell me that you've never actually used SQL Server at all.

Re:David Litchfied

[geoffspear]

That's nice, but argument from authority doesn't work when the methodology used is clearly bogus. If Larry Ellison announced that MSSQL is more secure than Oracle and based that assertion on the number of bugs fixed in a given time period, I wouldn't trust him either.

Re:Summary title is vague

[ferretworks]

Have to agree with the masses. Calling it SQL Server seems to only piss off the people who don't work with it. I don't call the Office suite Microsoft Office. It is just Office. Microsoft was clever in their naming schemes. If I am talking about a SQL server that is Oracle, I wouldn't refer to it as "Oracles SQL Server", nor would MySQL be "MySQL SQL Server".

That would just be silly.

So, your anger is Microsoft's gain. And every time you get angry at Microsoft, they kill a kitten.

Re:translation

[drinkypoo]

MSSQL came from Sybase 10, which was a quite excellent database with a much better reputation than Oracle at the time. It didn't scale as well, but it was quite a bit faster on mid-size data sets. If this is the one division in Microsoft that's employing people who actually fix bugs, I'd say this is an entirely credible report. Given what a PITA Oracle is in general, it's not even unlikely.

Re:My experience

[anto]

Have you tried to call MS & log a 'support' call - more than once we have had to hand over the credit card no before the call will be forwarded on. Of course with the promise that if there was an issue they wouldn't charge it.

Oracle on the other hand request your support contract no (which they will actually look up for you) once you get past that really minor issue you never hear anything about money again. If you are unlucky enough to have a real bug that gets escalated you have the fun experience of hearing from someone from oracle every few hours - the calls seem to come from all over the world (based on accents etc)

More than once I have had a custom patch created for what to oracle must have seemed like a really minor bug.

Re:MS Labs Has No Equal

[Nocturnal Deviant]

XP quite good now? apparently "Patch Tuesday" isn't in your monthly things to do list.... or checking windows update every day.... and as to the google comment... if Microsoft wasn't worried about google(shocking realization i know) then why is microsoft finally changing their browsers, and msn search since google and firefox came around..? google: Latest Windows XP bugs http://www.google.com/search?hl=en&q=Latest+Window s+XP+bugs&btnG=Google+Search [google.com] ...OMGZ 51,500,000 results hey everyone just ordered my Kubuntu CD's I'm heading for the virtual hill's...in truth though I prefer Slackware. Back on topic though, I use MySQL, catching me using Oracle OR MSSQL, is a joke, with open source I don't have to scream and cry and throw chairs(reference http://www.theregister.co.uk/2005/09/05/chair_chuc king/ [theregister.co.uk]) I can code my own fix 99% of the time before an official one is released.

Re:If you offer a ton of additional features...

[RevMike]

If you offer a ton of additional features...then it stands to reason that you will have a ton of additional bugs.

This argument in no way excuses Oracle for their timely patch cycle (or lack thereof), but may explain the higher number of patches.

It is also important that Oracle supports virtually any server platform in current use, while SQL Server only supports a small number of similar platforms. Back in 2001 I was still getting support for Oracle 7.0 on VAX/VMS! One get Oracle on Linux, AIX, Solaris, HP-UX, zOS, OS400, Windows, a variety of Alpha platforms, Itanium platforms, etc. And this isn't shallow level support. Oracle can utilize their own file systems, so they are going at the bare hardware on all these systems. Care to guess what that does to the QA cycle?

Oracle is the shiznit when it comes to high performance general database work. It will scale far beyond almost everything else, with DB2 a close #2. Niche players like TeraData have their place too, but only Oracle can scale across the entire enterprise.

Different Uses - why compare?

[slightcrazed]

Why would one even want to compare SQL server and Oracle. Are the 2 really in the same league? I have installed both at many sites and there has always been very clear criteria which dictates which gets installed at what site: Amount of users and knowledge of sysadmin. If I know I'm working with a guy with 10 users who thinks that AIX is a type of sportbike, then he gets SQL server and my direct phone #. If I'm at a site with 1500 users with top notch sysadmins then they typically get a high end unix/linux machine with either an Oracle or Informix DB. I have flat out refused to install SQL server at some sites based on the above criteria. I just don't understand the comparison. As soon as SQL server can run on something other than an intel box (and hopefully something other than Windows) and can handle the kind of workload that I expect without grinding to a halt then I might think about installing it at some of the bigger sites I work with.