Why Some Devs Can't Wait For NoSQL To Die 444
theodp writes "Ted Dziuba can't wait for NoSQL to die. Developing your app for Google-sized scale, says Dziuba, is a waste of your time. Not to mention there is no way you will get it right. The sooner your company admits this, the sooner you can get down to some real work. If real businesses like Walmart can track all of their data in SQL databases that scale just fine, Dziuba argues, surely your company can, too."
Can't wait it to die? (Score:3, Insightful)
This is like saying "I can't wait for memcached to die" just because your site doesn't need it. Fact is, some do. It's your own fault if you choose to apply unnecessary techniques.
Don't change to newer fancy techniques if you don't understand what they are for and why would you need them.
Neither are going away so just shut the fuck up (Score:1, Insightful)
Hierarchical DBMS have been around longer than SQL-style RDBMS for a long time. OODBMS have existed for a long time as well. Many of these "NoSQL" DBs don't provide the same restrictions or assurances that an RDBMs provides but they often have other features.
BDB isn't going away and neither is SQL. Get over it.
Re:Can't wait it to die? (Score:3, Insightful)
memcached is most useful when the underlying app is hideously inefficient, e.g. it's pretty much essential to a MediaWiki installation that gets any appreciable number of users.
Re:Article summary (Score:5, Insightful)
My experience has made me believe PostgreSQL is better in every respect. It's more stable, has more features and is easier to use. The article wasn't specifically pro-MySQL.
The article is largely correct. The movement to ditch SQL databases is really naive. SQL scales just fine, if you know how to use it right. Look at Oracle solutions. All their fancy eBusiness software is still Oracle SQL DB backed and some of the biggest companies in the world are using it.
SQL isn't the problem, it's a tool. Bad programmers are the problem.
The Actual Quote is (Score:5, Insightful)
"MySQL or PostgreSQL," for what it's worth. PostgreSQL is a pretty powerful database, and you should have to make a pretty good argument why leaving a well understood technology that powers a lot (an some of the largest parts) of the WWWeb needs to be trashed for something newer and less tested.
Why? (Score:2, Insightful)
In short, people should just shut up about other people's choices and get on with their own.
Re:Article summary (Score:5, Insightful)
Look at Oracle solutions. All their fancy eBusiness software is still Oracle SQL DB backed and some of the biggest companies in the world are using it.
Yep, "nobody ever got fired for choosing Oracle".
But to get performance and fault tolerance for Oracle, you need to throw a lot of money at it -- high end hardware, RAC licenses etc. Whereas some of the NoSQL DBs promise lots of scalability on clusters of cheap hardware -- situations where failing hardware is the norm.
If your application suits it (i.e. your data fits the name/value system, and eventual consistency is adequate) why not use something fast and cheap?
Resources vs. Smarts (Score:3, Insightful)
FTA:
"In the meantime, DBAs should not be worried, because any company that has the resources to hire a DBA is likely has decision makers who understand business reality."
Bad English aside, I just don't agree. Money != Reality. I have worked both sides of this coin - Startups with plenty of money but don't see the value in proper maintainance of the data store (one almost was put out of business by a disk failure), and very smart startups that are running lean but do understand the risks.
That said, on the deeper level, why does business reality == SQL? Sure I can scale Oracle to support massive DB's (and have), but I could probably get more value from using Amazon's SimpleDB for things that don't require massive scaling. Use the right tool for the job - Hammers are for nails, etc. Do the design work up front, decide how its gonna work, and the right tool should present itself.
Re:Article summary (Score:5, Insightful)
Re:Right! (Score:3, Insightful)
And as far as it being 'a product of the braindead and buzzword-infested effluents of the American "education" system, where nobody understands math or logic', I don't care if it came from the bottom of a well in the middle of a jungle where they are masters of logic and math, if it could possibly meet my client's needs then I'm going to give it the time and attention it takes to make the decision for myself.
The Article Is Right... And Wrong (Score:4, Insightful)
Real business track their data with SQL databases, true. However, real businesses have small numbers of transactions relative to their value. If Walmart had the same revenue but the average sale was a tenth of a cent, their fancy SQL database would be smouldering rubble.
That's what Facebook and Twitter and other large social media sites are facing. Just try running Twitter's volume and Twitter's page hits and API hits off MySQL. It doesn't matter how many replicas you run, it's not going to work. Maybe you could run it on a cluster of IBM Z-series mainframes running DB2 - but where is the money going to come from?
Cassandra and HBase and the other distributed NoSQL database solve specific problems in specific ways. They won't work for Walmart, but they'll do the job just fine for Facebook and Twitter. If you have those specific scaling problems and can live with the restrictions (you lose ACID, indexes, and joins to varying degrees) then they'll work for you.
If all you know is that your site is running slow, then implementing NoSQL is unlikely to improve things.
Someone tell me again... (Score:2, Insightful)
Some people just want the holy grail (Score:3, Insightful)
I think some developers keep looking for the holy grail. Some magical solution that will turn development from punching in code, to Star Trek: "Computer do my job for me please".
Template languages, 4GL, NoSQL, Ruby on Rails... it is all part of an attempt to take the nasty out of development and they all... well... they all just don't really happen.
Because deep down, with all the frameworks and generators, if you want your code to do what you want it to do, you are still writing out if statements a lot.
And yes, OO and such also belong to this. Not the concept themselves, but the way most people talk about. OO means code re-use right?
If you said yes, then you are a manager, go put on your tie, you will never be any good at coding.
You can re-use all code. And it has been done for a long time.
What, did you think that people who wrote basic for the C64 went "Oh I wrote this bit of code for printing, now I need the same functionality, I am going to write it all over again!"
OO does make code re-use a bit easier BUT that is NOT the claim that people often make. Trust me, I ask this in interviews and it is always the same answer. Apparently you can't re-use functions. No way, no how. NEXT!
I see two kind of developers. Those who hate their job and those who don't. The former want to be managers, get away from writing code as fast as possible. And they will leap on anything that seems to make their jobs easier. Meanwhile the rest of us go on with actually producing stuff.
Just check, how many times do you get one of those managers wannabe introducing something they read in a magazine because it promises that you don't need to write another line of code ever!
Re:Why? (Score:3, Insightful)
No, just no.
That's about as information-free as one can get. I'd ask why, but then, I don't understand why I would have to, just saying "no" is void of context and explanation.
More RDBMS dogma (Score:4, Insightful)
Use the right tool for the job, except databases, eh?
The simple fact of the matter is that not every app is aiming for Google's scale. (Not every app is web-based or even going to be web-based, though people seem to forget that.) And even some large-scale apps don't fit the relational model very well, medical records being one of the more outstanding examples.
And yes, I have read Codd and Date and understand the relational model and its benefits very well, and it annoys me to no end when people break the relational model without realizing or understanding what it costs them. That said, sometimes those costs are acceptable, and sometimes an application requires features that the relational model does not (and in fact cannot) bring to the table.
It may be, as with every other silver bullet fad, that what's at work here is the basic human tendency to become familiar with something, begin to see everything in terms of it, and then try to persuade anyone who'll listen that they are in possession of the all-singing, all-dancing solution to all problems. Today, it's Ruby, multi-touch interfaces, and functional programming. But not very long ago it was COBOL and CICS. And while one must acknowledge that progress has been made, it is equally obvious that progress will continue to be made and that "one size fits all" is always BS, even in clothing.
Re:Some people just want the holy grail (Score:4, Insightful)
In some ways I agree with the general idea of your post. But stepping back a bit, code HAS gotten easier to write over the long term. I'd hope nobody would argue that writing a large application in a modern high level language is easier than writing it using 1970s technology in assembly. Those advancements in language came through a lot of trial and error (a lot of error). How many failed language exist that turned out to be dead ends (though spurred further advancements and refinements?). How do you know the technologies you mentioned won't turn into the next (your favorite productive language here)?
You're right that endlessly pursuing the latest trend is just foolhardy, as most "new latest greatest technology" turn out to be duds. The point being those duds sometimes DO pan out. Anyone that thinks that relational databases are the end-all-be-all of persistent data storage hasn't done enough relational database development to understand some of the limitations.
Re:Article summary (Score:5, Insightful)
sqlite is underrated and would be ideal for many such applications.
Re:Article summary (Score:0, Insightful)
No one wants to work for you or from your mother's basement anyhow. Perhaps if you weren't such a rude, close minded fool you'd do better.
Re:Article summary (Score:5, Insightful)
What product has Oracle ever dropped support for? What is your objection to MSSQL? SQL 2005/2008 are damn fine products which perform extremely well. Sounds to me like you're the one that is ignorant with blanket policies against industry standard tools.
Of course I run Oracle, MySQL, and MS SQL in my datacenter all without problems and some under nice and heavy loads. About the only sensible stance you have is with Postgresql which is far and away better than MySQL which in my opinion sucks pretty bad.
Re:Hardware is cheap. Developers aren't. (Score:4, Insightful)
It's really that simple. A standard dual socket server with the latest CPU's from Intel or AMD can handle hundreds of requests per second;
Hundreds of requests for WHAT per second?
Your idea of "just throw hardware at the problem" isn't generalizable. Throw hardware at WHAT problem? For some problems, you're right. For others, you couldn't be more wrong. There's really no point in saying anything further.
Re:Hardware is cheap. Developers aren't. (Score:5, Insightful)
Unless you attract enough attention to require scaling past 10M pages a day, you're wasting your time reinventing the wheel with NoSQL, just stick with a standard ORM, launch your site and start convincing customers and generate sales.
Most of the buzz about these things comes from and is aimed at people who actually believe they'll build the next Facebook or Twitter. The fallacy is in their belief that it's the size/traffic of those sites that supposedly mandates NoSQL and not the simple data models. Some of the biggest, less spectacular projects out there run on PostgreSQL for example (Skype, Affilias = .info and .org).
Re:Hardware is cheap. Developers aren't. (Score:3, Insightful)
For the type of loads 'front-page' slashdot (and your site, most likely) gets, SQL fits fine. But even then, NoSQL may give you a run for the money.
Now think of the loads incurred in the comment tree of slashdot.
Also think how something like GMail or even Google Search would fit in an SQL scheme. It doesn't, not at least, with table juggling that would be very inefficient.
Re:Article summary (Score:4, Insightful)
I suppose it's the same argument when having to choose a development language. You got to pick 4GL languages, VB, Pascal, c++/Java/c#, assembly, and machine languages. The art of a great analyst is to know which to pick and when.
SQL performance (Score:3, Insightful)
People complaining about SQL performance are most likely either using incorrectly scaled machines for the job, or believe they can throw a four-line SQL statement at the database and expect it to work out the optimization on its own ... query optimizers may be able to do a decent job on average, but once you go large databases (multi-million dataset tables), planing the query structure will go a long way preserving performance. ...
Yes one can write complicated queries to return exactly what you want in one query, but in many cases doing some logic around it and using smart grouping/loops will outperform the complex query
Storage as a Service (Score:3, Insightful)
The article focuses on NoSQL's claim to scalability, but isn't that just one of the features of (some of the) NoSQL options?
Google, Amazon, and Microsoft all provide NoSQL storage as a service that is easy to use and cheap, particularly for getting started. Those are two pretty important features and I would imagine that it is those features, rather then dreams of needing vast scalability, that attract the many web startups.
Re:The Article Is Right... And Wrong (Score:3, Insightful)
The point isn't (generally, there might be some pathological corner case) that the various web2.0 kiddies couldn't implement their stuff in SQL; but that they couldn't afford to do so. If you want to be able to serve large numbers of users in order to generate enough adsense pennies to keep the lights on until somebody buys you, your options are pretty much A). Software with a more or less zero per-node cost, running on commodity x86s with no exotic interconnect. or B) Bankruptcy.
Re:Article summary (Score:4, Insightful)
MSSQL's lock escalation isn't as efficient as Oracle's, but that doesn't make it a toy.
WalMart doesn't use SQL (Score:1, Insightful)
WalMart has one of the largest Teradata installs [eweek.com], it doesn't use SQL.
Re:Article summary (Score:5, Insightful)
Virtually no-one who's spent any time analyzing and working with large amounts of data has a good word to say about SQL.
I've spent 10 years developing intensively relational applications with SQL. I love it!
It was designed from the start as a language that would be integrated into others, and yet simple real world realities make that impossible, with 99% of implementations being of the "Build a large string, and pass that string to "the SQL connector" to be parsed and interpreted" form.
So... because people don't bother to to learn about things like prepared statements, the tool is bad? It's like saying that cars suck because they don't have cruise control!
Its handling of null and the empty string is incomprehensible and useless, in part because nobody involved ever had the cajones to do what needed to be done with both.
OK, so enlighten us with your brilliance! Share with us the ultimate answer of what should be done to differentiate a null (logically, "I don't know") with a blank string (logically, "We know there's nothing there") and what should be done differently?
IMHO, the concept of "null" is a very useful one which allows a developer to differentiate between a blank answer and a no answer.
There is no standardized set of data types in the real world. Simple issues with unstandardized case dependencies can make an application that works with Oracle and only uses standard "select" statements not work under, say, PostgreSQL.
Woah, hold on there boy! You mean to say that features specific to one database engine won't work with another? Well spank my uncle and grease my kittens - this is amazing! Unless, of course, you stick to ANSI 92 syntax, which is pretty much 100% compatible. Yes, there's some regression testing you'll have to do against the different databases. Just like you have to do with HTML, XML, or any other standards-based language.
(yawn)
And these are the surface level technical issues: talk to any relational database guru and they'll come up with numerous philosophical issues too.
Strange how you didn't manage to name even one?
But here's the part of this whole "NoSQL vs SQL" debate - SQL is an interface API to a DBMS, it's not the database itself! You can use any number of technologies "under the hood" including those
types of technologies commonly referred to as "NoSQL" and put an SQL interface in front! The whole idea that SQL is somehow the problem is just.... idiotic and betrays an astonishing lack of understanding by the programmer(s) involved.
It's like saying that you should have a stick-shift car because automatic transmissions don't go as fast. It's just moronic. Arguing about NoSQL is like arguing with a tea party dolt about the "socialist" health car plan that just passed! (that was first drafted by the "right wingers" 15 years ago)
It's argument from stupidity.
Re:Can't wait it to die? (Score:5, Insightful)
Re:Article summary (Score:1, Insightful)
I've actually seen people use SQL databases for _cdrom_ distribution, the idea being pack up the data into tables and distribute the whole database on the CD along with an "application" to run it. Or applications that accept a mysql table as input and (you guessed it) generate a mysql table as output... clearly a case for stdio.
People over-use databases, (and worse, they always use mysql, for EVERYTHING) which I believe, leads to a sort of anti-SQL movement. There are a lot of cases (the CDROM as an example) where plain old text files would work better, or, DBM's with simple key/value pairs.
Doesn't help that it's always "mysql" either. I'm glad for the NoSQL "movement" if, for nothing else, it's making people aware that you don't need the overhead of a full scale relational database when the filesystem itself works pretty good.
Re:Article summary (Score:5, Insightful)
Au contraire.
While there are problems with SQL, 95% of its users are happy as a clam that it exists. The unhappy users are the ones who are pushing the boundaries of what SQL allows and those are the people who know SQL best. When you are writing SQL queries that span 200 lines of code, then, and only then do you begin to scratch at the limits of what SQL allows. Until then, you've only hit the limits of competence.
I've been working with SQL for over 20 years now. I've worked with applications that didn't use RDBMS's. Some of them used flat files. Some of them used hierarchial databases. People who haven't had the same sort of experiences, haven't come to the realization of why SQL was invented - and that results in then making ill-founded statements like "SQL is absolutely the worst database query language ever invented". Utter tosh. SQL has its problems, but its one of the best. That's why it has left its competitors in the dust of time.
I look around at all the frameworks that have evolved to not do SQL (EJB-QL, Hibernate, etc) and I laugh. None of those languages come close to handling the same breath and width of problems that SQL can be used to solve. Whenever I see advocates of these frameworks all puff up with fervour, I feel like shaking them and say "Your emperor has no clothes!". The list of problems these frameworks can't solve is so huge that one wonders why anyone works with them at all. But I suppose, there are plenty of people who work for small businesses who haven't encountered the kind of problems that big enterprises have.
The parent poster that I'm responding to has apparently had an problems porting SQL code. But guess what? Even on the unix platform, applications written in C have had trouble being ported from one Unix to the next. People have worked around it. Nobody goes around arguing that "C is absolutely the worst programming language ever invented".
Use what is appropriate not FUD (Score:2, Insightful)
Easy. (Score:2, Insightful)
Because they're gonna tell your non-technical boss to make you use it, and he's gonna listen when they start telling him that Google, Twitter and Facebook do.
Re:Article summary (Score:3, Insightful)
Re:Article summary (Score:1, Insightful)
[citation needed]
hammer - all nails (Score:2, Insightful)
Re:I'm Still Fuzzy on NoSQL (Score:3, Insightful)
Unfortunately the NoSQL people should have called their movement "nonrelational". You can have a relational database and not use SQL; the two are not dependent on each other as there are nonrelational databases that allows the use of SQL. Although the movement for the use of nonrelational databases may be new, the use of nonrelationals is not. My first exposre to a business class database was Pick in the 70s. There are plenty of these types of systems in use today. Nonrelationals have been going strong for over 40 years
This article is another case of somebody that does not have the breadth of experience in the field. The same applies to the the people that started this movement. I have to ask, does college no longer teach the history of computing?
Re:Some people just want the holy grail (Score:1, Insightful)
You can reuse functions but you can't extend them and that's where OOs reuse shines.
Have you ever programmed in anything other than OO? I was reusing and extending functions in several different languages long before OO became fashionable.
It's very powerful to be able to lay out your code as a tree and control the reuse 'flow' at the nodes.
Yeah, until the customer asks for audit logging on everything that saves data and for a dozen distant nodes to start talking to each other. Then you have to redesign the tree from near enough the ground up. Which has happened on pretty much every OO enterprise application project I've ever seen.
Face it, for almost anything except games and GUIs, OO bites.
even if you are not Google... (Score:3, Insightful)
It's easy to hit intrinsic performance limits with SQL databases even on small apps. And for people who aren't database experts, it's even easier since they don't know the hoops to jump through to make their SQL databases perform well. For the average programmer, it's easier to get good performance out of no-SQL databases.
Using SQL databases programmatically is a fairly silly notion to begin with: SQL was originally intended as an easy-to-use query language for non-experts because people were having trouble with navigating data structures. But programmers are excellent at navigating data structures and designing efficient data structures. SQL is solving a problem that most programmers don't have, and you're paying a big performance penalty for that.
Sometimes an SQL database is the right thing to use, sometimes it isn't. People really need to use their head instead of blindly picking one or the other solution.
COBOL rules! (Score:1, Insightful)
What was wrong with COBOL? Didn't it solve most of businesses problems? What makes C/C#/C++/Java/Ruby/Perl/PHP so much better?
Re:Article summary (Score:5, Insightful)
It's not heresy. However, I have seen a lot of crap data models produced by developers (even worse than what I come up with as GUI designs). I have also seen developers produce SQL that looked OK at first glance but performed abysmally under certain conditions (and have even saved the odd project by finding those and fixing them when the system started dying under load). If you access a SQL database like you would a set of flat files, it is never going to give you the performance that a flat file access will give you for raw throughput because you've got all the extra communications latency. However if you re-write your search and extract queries to pull your data in a single SQL statement instead of a statement for each of your N tables involved in the result, then SQL is going to kick ass as soon as you start getting enough data and users placing enough queries that all the indexes and caching can pay off.
Flat files will work better for certain types of unstructured data, but most people who get crap performance out of SQL databases just don't understand how to use SQL databases properly. Which is why those True Believers tend to get upset about crap SQL implementations: because those tend to bog down a SQL server and slow down all the well-written apps too.
No, the real problem with most SQL DBAs is that they haven't adapted to agile methodologies. They still want the data model to be spring fully armored from Zeus' head according to classic waterfall planning. What they need to do is to get some data modelling tools that support round trip engineering so that they can make changes as the developer needs them and have upgrade scripts checked into source control along with the code on new builds. Right now there's only a few tools like ErWin and Data Architect that support that kind of development, and they tend to be ridiculously expensive. The one exception is DeZign for Databases Professional which is comparatively cheap. A lot of companies will lay out a lot of cash for developer tools but won't fork over the dough necessary for their data modelers/DBAs to properly support developer activity. So yeah, the DBAs tend to be a little reticent to do all that work by hand. While there are some developers who still use notepad or gedit by choice, nobody seems to expect them to do it, or to have the same productivity as someone with a decent tool chain.
Re:Article summary (Score:3, Insightful)
Whenever you port from one RDBMS to another you're going to have to put up with all sorts of stuff like this. So I don't see this as a showstopper.
So that's why I was curious on what the OP's problem was. If it's just "MSSQL names stuff differently and I didn't bother to do a bit of research to find that out" then I can ignore that particular complaint about MSSQL.
Whereas if it's something else, then it could be far more interesting and more useful to know.
FWIW, I'd think Oracle's behavior of treating an empty string as NULL could be more annoying, but lots of people still use Oracle...