Digg Says Yes To NoSQL Cassandra DB, Bye To MySQL 271
donadony writes "After twitter, now it's Digg who's decided to replace MySQL and most of their infrastructure components and move away from LAMP to another architecture called NoSQL that is based in Cassandra, an open source project that develops a highly scalable second-generation distributed database. Cassandra was open sourced by Facebook in 2008 and is licensed under the Apache License. The reason for this move, as explained by Digg, is the increasing difficulty of building a high-performance, write-intensive application on a data set that is growing quickly, with no end in sight. This growth has forced them into horizontal and vertical partitioning strategies that have eliminated most of the value of a relational database, while still incurring all the overhead."
Away from LAMP? (Score:3, Insightful)
Or away from MySQL? There is a difference.
The Monty crowd will blame this on Oracle (Score:2, Insightful)
This sad thing is that Monty's MySQL fan boys will blame this on Oracle when in reality the move to Cassandra (or other NoSQL databases) is what a lot of web sites should be doing regardless of who holds the MySQL reins.
Re:Database Evolution (Score:1, Insightful)
its already multi-dimensional. you have a record, it has keys in it, the values can be objects. that's three or more dimensions there depending on how complicated the objects are.
Re:Good for them (Score:4, Insightful)
Re:Which DB is better? (Score:3, Insightful)
If you need a comparison chart... you don't need to switch.
It's probably not necessary to change such a huge part of your architecture if it's not worth investing serious time investigating and benchmarking the alternatives.
Comment removed (Score:3, Insightful)
Allergic reaction to MySQL (Score:5, Insightful)
These slides [pgexperts.com] present a balanced and comprehensive overview of the current state of free databases. Whether you're in the NoSQL camp or not, they're worth reading.
That said, here's my take:
It's currently fashionable to replace MySQL with some "NoSQL" database or other. This trend is driven by two factors:
I haven't seen any consideration from potential "NoSQL" adopters of the benefits of using a good relational database like PostgreSQL. There's a world of difference between it and MySQL, and condemning all relational database systems because of bad experiences with MySQL is like condemning all sandwiches because McDonalds once made you sick. In giving up RDBMSes entirely, these developers lose quite a bit of safety, flexibility, an convenience. It's a huge over-reaction.
This field should not be about following trends, though unfortunately, that's how most people choose which technologies to use: it should be about choosing the best tool for the job. And I believe that in the vast majority of cases, the advantages conferred by a relational system --- enforced integrity, interoperability based on SQL, query flexibility, storage flexibility --- make an RDBMs the best choice for almost any job. If you need sloppier semantics for some cases (for example, "eventual consistency"), you can layer that on top of a robust RDBMs.
Re:Which DB is better? (Score:2, Insightful)
Re:Facebook, Twitter and now Digg (Score:2, Insightful)
According to various internet sources (so take with a grain of salt):
Mark Zuckerberg's net worth [wikipedia.org]: $2 billion. Made entirely from Facebook.
Twitter's net worth [venturebeat.com]: $589 million.
Digg's net worth [websiteoutlook.com]: $24.34 million.
Even if each individual datum is nearly worthless, the combined value is far from it. Do you think any of those companies would still be worth what they are if they're databases were irretrievably wiped?
Re:Wow... (Score:3, Insightful)
If you're trying to run a site on a $15/month hosting account, then no, this is probably not for you. But if you're at the stage where MySQL isn't able to handle all the data you're throwing at it, then chances are you won't care about the extra few MB of memory that the Java runtime requires.
Re:Facebook, Twitter and now Digg (Score:3, Insightful)
Worthless?
That data reflects our culture!
Nobody said it couldn't be both at the same time.
Re:Facebook, Twitter and now Digg (Score:5, Insightful)
Are you seriously arguing that unless the first derivative of one's salary is positive, there's no incentive to work?
No, I did not say that one's salary needs to be monotonically increasing. That is not the point at all. And did you really have to turn this into a calculus problem?
To state it differently, many entrepreneurs are willing to work temporarily for little or even nothing, and to make great sacrifices such as giving up health benefits, vacations, and normal family/social life... things most 9-5 workers would never consider. Being someone's bitch for $1M/yr (or to be pedantic let's say $1M/yr + 5%/yr^2) may sound like a splendid deal to you but there are others who would work much harder for sweat equity in their own venture.
These people exist even if you can't fathom it. I'm one of them.
"NoSQL"? (Score:5, Insightful)
Am I the only one who frowns at this moniker?
First, it creates a false premise where people need to pick "SQL" versus "no SQL", while many real-world systems intelligently combine relational and non-relational data storage for their needs. There is no conflict.
Second, there's nothing wrong with SQL as a language in particular, and in fact many of the "noSQL" engines are starting to support and extending basic SQL queries, instead of reinventing their own query language for the same purpose.
I suppose "lessRDBMSabuse" was less catchy...
Re:Wow... (Score:4, Insightful)
Bullshit. Languages don't scale: programs do.
Writing a program in Java makes is scalable in the same way that painting a car red makes it fast. The JVM is quite good these days, but don't make up advantages that don't exist.
Re:Wow... (Score:1, Insightful)
Java the language isn't scalable on it's own.. there's no magic scaling technology built into the jvm.. but the general Java "culture" tends to (in my opinion) achieve at least medium scalability.
When judging a language, you _have_ to look at the culture around it. These days nothing is 100% custom build.. a sizable project is going to import a wide variety of 3'rd party libraries. The general attitude of the community is going to determine how suitable these libraries are for whatever scale you will be using them at.
Same as how languages like perl on their own don't produce unmaintainable code.. it's the perl "write once, read never" culture that leads to so much unreadable code.
Re:Why? (Score:2, Insightful)
A bad policy when dealing with your data.
Once it's broke, it is way too late.
You can't un-LOSE the past 6 hours of transactions or table referential integrity that MySQL trashed, due to an unclean shutdown.
MySQL's great until it comes up to bite you in the arse.
Re:Which DB is better? (Score:3, Insightful)
Re:Which DB is better? (Score:4, Insightful)
First of all, if he's asking Slashdot for advice (which is barely a step above reading tea leaves [which itself is a step above asking 4chan]), he doesn't need Facebook-level scalability.
Second, you're confusing scalability and performance. Scalable solutions tend to actually be slower than non-scalable ones: the difference is that a scalable system increases in capacity linearly with the number of machines you throw at it ("horizontal" scalability), whereas a fast non-scalable system generally needs the same number of faster, individual machines to increase capacity ("vertical" scaling).
Third, PostgreSQL has excellent performance, and PostgreSQL does, in fact, scale horizontally [postgresql.org].
Re:Which DB is better? (Score:0, Insightful)
If you need a chart to help you pick the best database for your site God help you. Either hire a DBA or just stick with LAMP.
Re:Facebook, Twitter and now Digg (Score:1, Insightful)
seanadams, you are a blathering idiot who knows he's lost the argument but doesn't have the maturity to accept it. QtrMstr has patiently, and repeatedly, made his point in very clear terms (I for one, had no trouble understanding him). And you've responded with the equivalent of clapping your hands to your ears and singing "naa-na-na-na-na". I've already spent all the moderation points I had in modding you down. Posting this as anonymous because I can't imagine how your moronic and immature babblings have gotten whatever points they did get.
Re:Allergic reaction to MySQL (Score:4, Insightful)
As several MySQL experts already noted, Digg isn't even using the indexes that provide maximum performance in the query that they present as problematic for MySQL:
http://mysqlha.blogspot.com/2010/03/index-only.html [blogspot.com]
http://www.yafla.com/dforbes/Getting_Real_about_NoSQL_and_the_SQL_Performance_Lie/ [yafla.com]
So you are right about the NoSQL fashion trend. Looks like for some companies it's easier to throw a pile of cheap commodity hardware driven by some NoSQL BigTable-wannabie at the problem instead of carefully optimizing queries and indexes for the best performance.
Re:Allergic reaction to MySQL (Score:5, Insightful)
The adopters of NoSQL deal with huge volumes of worthless information. They don't care about transactional integrity as much as they care about performance, which is why they chose MySQL over a good relational database in the first place.
Re:Wow... (Score:4, Insightful)
Java is a whole platform that is scalable. Its not just about using identifiers and objects but using the vast API's. Some would Java is even an OS as it has its own I/O, threads, etc.
I suppose you could write your own threading and processes code but most Java developers just use whats built into the api.
Re:The Monty crowd will blame this on Oracle (Score:5, Insightful)
Re:Which DB is better? (Score:2, Insightful)
Putting a proxy between the client and the server to handle the replication does not make Postgre horizontally scalable. Nor does doing a periodic table dump and copying it to the other machines. Postgre might be a ton more efficient than MySQL, but it is in no way scalable.