"Slacker DBs" vs. Old-Guard DBs 267
snydeq writes "Non-relational upstarts — tools that tack the letters 'db' onto a 'pile of code that breaks with the traditional relational model' — have grabbed attention in large part because they willfully ignore many of the rules that codify the hard lessons learned by the old database masters. Doing away with JOINs and introducing phrases like 'eventual consistency,' these 'slacker DBs' offer greater simplicity and improved means of storing data for Web apps, yet remain toys in the eyes of old guard DB admins. 'This distinction between immediate and eventual consistency is deeply philosophical and depends on how important the data happens to be,' writes InfoWorld's Peter Wayner, who let down his old-guard leanings and tested slacker DBs — Amazon SimpleDB, Apache CouchDB, Google App Engine, and Persevere — to see how they are affecting the evolution of modern IT."
slashdot insult? :( (Score:5, Funny)
FTA: "The world won't end if some snarky, anonymous comment on Slashdot disappears." :/
What? Nothing more important than anonymous slashdot trolls to moderate
*mods article -1, Flamebait* (Score:5, Insightful)
Is it just me or did this article go out of its way to insult people who use "traditional" RDBMSs?
I mean, I'm well versed in SQL and data consistency et al, but I'm still more than willing to consider new technologies. What the hell?
Re: (Score:2)
I read it exactly the other way, that they were slagging on the newcomers in favor of us old fogies (PostgreSQL FTW!).
Re: (Score:2)
Phew--- At least I know that I was reasonably balanced. I guess it depends upon how you feel about the intensity of insult buried inside the use of words like "slacker", "codger" etc. If they're all about the same level, well, I've threaded the needle.
Re:*mods article -1, Flamebait* (Score:4, Funny)
You seem to imply there was more to the story than the summary. This confuses me.
Re: (Score:2)
Ha. RTFA where F=fantastic or fantabulous,
Re: (Score:2)
It's perfectly fine to say how it reads to you. If I could be all things to all people, oh, the things I could accomplish.
Re: (Score:2)
Old vs. New Simple DB's (Score:3, Funny)
Wayner's usually a good writer, and did some good theoretical-computer-science work back in the day, but this article was too short to answer the questions he asks at the beginning, and he mostly highlighted the new shiny things from big ASPs, which is generally what Infoworld wants.
I'm particularly disappointed that while he referred to the name and history of Berkeley DB, aka Sleepycat, aka Oracle Renamed-foo, he didn't actually talk about using it. (OTOH, Infoworld did review one version of it in 2005 [infoworld.com].)
Re: (Score:3, Insightful)
the thing that always puzzled me about berkerlydb is it's incessent format breakage requiring dumps and restores.
On a database server at least data upgrading can be handled centrally but on a file based DB where datafiles can be scattered anywhere a lack of a stable data format seems like a fatal flaw.
Re: (Score:2)
Cloud Zealots(google employees, amazon EC2 developers, Wall St. analysts that are trying to attach their name to the "Next Big Thing", etc.) tend to have an insulting tone to their rhetoric.
Re: (Score:2)
It went out of its way to be irreverent to everyone: 'The new twerps really get those codgers steamed when they talk about how all of the computers in the cluster will get around to replicating the data' is a playful slap at both sides. And just like the PS3 Fanboys/XBots, if you identify too much with one of the sides you will nod your head knowingly at one and gasp and fan yourself as you suffer an attack of the vapours of offense at the other.
Normalization doesn't exist to save disk space (Score:5, Insightful)
Now that disk space is so cheap and many of the data models don't benefit as much from normalization, ...
You don't want to store the same data in multiple places. Your query might run faster, but your data integrity is going to suck.
And, uh, I have the pleasure of working now with a huge data warehouse that hasn't normalized status codes, so instead of quickly searching for an integer, the queries run slow as hell scanning char fields. It's not good.
Re: (Score:2)
Couldn't you index a char just as easily as you could an int? Or are you saying their status codes are strings?
Re: (Score:3, Informative)
Oracle doesn't have a "string" datatype.
Re: (Score:3, Informative)
Ah, my apologies. Really, it should be an indexed enum (or whatever Oracle equivalent there is... it's been a while since I used it) if there's no additional data to go along with the status code... or another table if there is additional data.
Re: (Score:3, Interesting)
Re:Normalization doesn't exist to save disk space (Score:5, Funny)
But if one of them is wrong, you can check the others and correct it.
My boss - a lead senior senior lead developer from Android Whorehouse & Douche - several years back, when I tried explaining "why I'd missed some fields out of one of the tables".
Re:Normalization doesn't exist to save disk space (Score:5, Funny)
But if one of them is wrong, you can check the others and correct it.
My boss - a lead senior senior lead developer from Android Whorehouse & Douche - several years back, when I tried explaining "why I'd missed some fields out of one of the tables".
I was about to post something explaining to you why that's bad, and then I reread your post and the whooshing noise around me quieted down.
Re: (Score:2)
Re:Normalization doesn't exist to save disk space (Score:5, Informative)
People that haven't done it don't realize how easy it is to end up in that situation. Say, I write reports about people, and Robin writes reports about assets, whose owners are people, and puts a person's name in her table to make it faster. Someone gets married, their name changes, and now Robin's reports are wrong.
Re: (Score:3, Funny)
Total failure to understand the situation. You, I mean he, didn't understand the concept of "factoring out" common information - say, the customer details on an order - from the variable per item data - product code, quantity.
What he, er, you appear to be talking about is natural vs surrogate keys.
Re: (Score:2)
And, uh, I have the pleasure of working now with a huge data warehouse that hasn't normalized status codes, so instead of quickly searching for an integer, the queries run slow as hell scanning char fields. It's not good.
What the hell does that have to do with schema normalization? Normalization has to do with how you architect your tables and relations. The types you use for the columns, and how you standardize their values, is an entirely different, though somewhat related, discussion.
And as it happens,
Re: (Score:2)
Re: (Score:2)
If that's true, then his comments have nothing to do with the quoted section about how, "Now that disk space is so cheap and many of the data models don't benefit as much from normalization, ...". Because that comment is *specifically* about schema normalization in the formal sense.
Re: (Score:2)
Yeah, but as you yourself point out, this isn't specifically because the tables are denormalized. It's because they're denormalized *badly*.
Denormalization, itself, does not equate to poor performance. In fact, quite the opposite is true, when it's done properly and thoughtfully. Which brings us back to the original quote: "Now that disk space is so cheap and many of the data models don't benefit as much from normalization" is *absolutely* true. Does it have to be done carefully? Yes. Is denormalizing
Re: (Score:2)
You don't want to store the same data in multiple places. Your query might run faster, but your data integrity is going to suck.
I wonder if normalization is going to be less important as de-duplication [sun.com] gets integrated into more file systems.
It's going to be a part of ZFS later this year I think. It looks like it's going to be a block level implementation within ZFS rather than file level. Since ZFS uses a copy on write model it seems fairly easy for them to implement compared to other file systems.
Databases that work on top of existing filesystems could benefit from this whereas databases that use block level addressing (Oracle) may
Re: (Score:2)
Re: (Score:2)
A short char field isn't necessarily slower than an integer, though. Right? They could both be indexed with log(n) search time.
Re: (Score:3, Insightful)
and in the Orders table once per order.
I'd disagree on this one, it seems to me like it would be a good idea to record the customers name and address (probablly both billing and shipping) at the time of an order even if they later change the details on thier account.
Mod this down (Score:5, Funny)
Like the article says, "The world won't end if some snarky, anonymous comment on Slashdot disappears."
Laziness Rules (Score:5, Insightful)
Slacker DBs like CouchDB and SimpleDB, have taken off for the simple reason that most developers have absolutely mediocre database knowledge or skills, and rather than learning it's just as easy to just wave it all off as obsolete.
It's no surprise that the creator of CouchDB, for instance, hadn't a clue about databases when he began his project. All of that built up knowledge just ignored while someone invented their own, and it's as rational as rolling your own encryption from scratch without the slightest clue about encryption algorithms or theories.
Re: (Score:3, Interesting)
... and rather than learning it's just as easy to just wave it all off as obsolete.
I don't know about that. But maybe these slacker DBs are perfect for what they're doing? Glancing at the those mentioned in the FA, it just looks like their simple tools to do simple things.
Don't get me wrong. I once had the pleasure of working with an Oracle god. This dude was about to take his final Oracle exam in a series of exams and he turned my Join that took ten seconds into a Join that took less than a thousandth. I have no idea what he did to this day, but it took several lines of PL/SQL. We were
Re:Laziness Rules (Score:4, Interesting)
In the end, the problem is that people just want a "default tool". They don't want to think about their requirements for data consistency. The really scary bit is that while RDBMses are the "default tool" of yesterday and slacker DBs are the "default tool" of tomorrow, neither of them are really the "problem".
The "default tool" attitude IS the problem. Unless you carefully weigh your data consistency requirements, you shouldn't be making that call at all.
I welcome the slackers and all of their new options along the spectrum of speed versus consistency. It's just that most of the people developing applications scare the shit out of me. They're so cavalier (or should I say, "agile", or maybe "pragmatic") about requirements that it's truly disturbing.
That said, if you're really interested in all of the options, I also recommend checking out memcachedb, memcacheq, and redis.
Re: (Score:3, Insightful)
Problem is, you're re-inventing the wheel several times over in the process. Hint: "a flatfile and maybe a little more" could very well be all the storage technology invented today only a few years down the road.
At first, all you need is to store key:value pairs. That works with a flat file or with Oracle. Then you need some consistency checks, which are can be modelled fast in Oracle or reasonably fast in your software. Then you need some triggers, which could be written fast in Oracle and not-so fast in y
Re: (Score:3, Insightful)
not really. i think that you (and, unfortunately, the FA) are missing the point that the map and reduce functionality, while powerful, have one major advantage: scalability. simply put, a query can be, by definition of the map function, broken up into several discrete operation
Re:Laziness Rules (Score:5, Insightful)
If you want "a little more" than a simple flat file, perhaps SQLite [sqlite.org] is the answer? The people on the Firefox team seem to think so, for example.
SQLite has been a pleasure to use for a small personal project involving a few Perl scripts. Granted my background is with SQL Server and Oracle, so perhaps I'm not the target audience, but I found it extremely easy to use and surprisingly efficient--and I didn't need to set up a server or anything. I didn't even need to explicitly create a database!
Re: (Score:2, Insightful)
That's what I don't quite understand about all this. It's been the case for a while now that:
1. If you want a full RDBMS, use Oracle, or PostgreSQL, or a similar ACID + SQL92 compliant DB.
2. If you don't really care, use MySQL.
3. If you want ridiculous speed, and actively hate your data, use SQLite.
4. If you have one file, or maybe two, use BerkeleyDB or similar.
5. Flat files are fine for config.
I'm not sure we need yet another category here. Then again, we're now seeing things surfacing like database shard [codefutures.com]
Re: (Score:3, Insightful)
If you want ridiculous speed, and actively hate your data, use SQLite.
Care to explain why SQLite requires one to "actively hate [one's] data"?
Re: (Score:2)
For the most part the Overhead of running a Real DB is usually made up over time.
Small Apps tend to grow to big ones over time. Having Babbie Databases can become a stubmling block to your application. As well as for the organization. They may want to wharehouse your application data for making better business decisions and integration across apps. So you little 1 million record database will need to be integrated into a billion/trillion record database.
Re: (Score:2)
The fundamental problem of enforced consistency with our system is that it requires hard locking, and that is at odds with distributed scalability. When you're going over a few satellite hops and you've got users real-time deep collaborating (ie. they're all heavy writers) with >2s latency, a traditional DB isn't going to scale very well. Even the Facebook MySQL+memcached is going to break down in that environment.
Re:Laziness Rules (Score:5, Informative)
Damien Katz, CouchDB's creator, worked at MySQL prior to writing CouchDB, and worked on Lotus Notes prior to that...
Re: (Score:3, Interesting)
I'm just going on the statements he made about his own (lack of) knowledge in this video [infoq.com].
Re:Laziness Rules (Score:5, Funny)
Thanks for validating the OP comments....
Re:Laziness Rules (Score:4, Insightful)
>Damien Katz, CouchDB's creator ... worked on Lotus Notes prior to that...
That's not exactly a ringing endorsement.
Re: (Score:2)
After all, who wouldn't mind an application that says you need to restart your computer because the application crashed? A text pushing application at that....
Re: (Score:2, Informative)
He started work on CouchDB in 2005. Prior to that he was a Notes grunt of little significance.
He started at MySQL in 2007.
The point holds.
Worse is better (Score:2)
...especially when you don't know what "better" is and you're too lazy to learn: unwillingness to learn is stupidity. Like the quote says, ignorance is curable; stupidity is terminal.
People who use these things and think they're great and that they're doing amazing things don't realize that the time they're taking and the problems they're struggling with are long-solved trivialities. Nothing new. Nothing cool. It's like someone struggling with a bunch of complicated excel formulas to make their spreadsh
Re: (Score:3, Informative)
first some context. i architect data warehouses for a living. i also live in a world of building fairly specialized frameworks to deal with data warehouses architected as star and snowflake schemas. i tend spend quite a lot of time in pseudo-relational databases [wikipedia.org] that don't fully implement codd's rules [wikipedia.org].
for fun, i like to spend some time toying with couchdb, using it for loose data warehousing, extending it, and generally enjoying the application development freedom it gives me.
that said, let me respond to
Re: (Score:2)
It's no surprise that the creator of CouchDB, for instance, hadn't a clue about databases when he began his project. All of that built up knowledge just ignored while someone invented their own, and it's as rational as rolling your own encryption from scratch without the slightest clue about encryption algorithms or theories.
It's funny how some people react by attributing ignorance to others when confronted with things they themselves don't understand.
For a time... (Score:2)
Well, even something that's based off of at-the-time sound principles can end up being a mess.
Take, for instance, a product falled FileMaker. It's a product with a long software lineage - it's origins were FoxPro, way back when. I don't know how it performed back then, or how it was designed, but now it's got a massive WYSIWYG themableing 'frontend' to make a custom application, and the database is not directly accessible by the designer (just logical containers). It probably can be normalized, to some degr
Well, it's like... (Score:4, Funny)
Either is cool with me, as long they are cool and takes care of business, you know what I am saying?
It's all good.
a base of data (Score:5, Insightful)
"tools that tack the letters 'db' onto a 'pile of code that breaks with the traditional relational model"'
If "database" were intended to mean only "relational database", we wouldn't have had any need for the latter term...
a top of lap, a book of note (Score:2)
who needs transactions? (Score:4, Insightful)
the article is right that in some cases it doesn't matter if a transaction is lost. but in any case where money is involved it's a must. you can't just start a fund from your Oracle or SQL Server savings to pay for mistakes because it will kill your brand and you may lose a lot of future business. and any savings will be eaten up by the extra cost to hire people to solve all the data problems
i've seen this. no constraints on the data that is orginally put in, not enough referential integrity and you get customers opening up a lot of trouble tickets and you end up hiring people to clean up the data every time a mistake is found
Re: (Score:2)
i've seen this. no constraints on the data that is orginally put in, not enough referential integrity and you get customers opening up a lot of trouble tickets and you end up hiring people to clean up the data every time a mistake is found
Really not trying to troll here, but this isn't too far from what a lot of people are dealing with when they use MySQL, especially the MyISAM engine.
A lot of people are using MySQL so it's just another step in the same direction.
In some projects, RDBM's aren't necessary. Look at what Google's been able to do with Bigtable/MapReduce. The open source equivalent seems to be Apache's HBase [apache.org] in the Hadoop project.
Re: (Score:3, Interesting)
Oh, come on. MySQL suffers from the same thing that PHP does; that it's industry standard and easy to use.
The bigger thing that they both suffer from is having a rather poor history. The problem with people saying how bad they are is that the complaints are based on old versions. PHP5 is much better than PHP4 or PHP3, and MySQL is steadily becoming something resembling a real database (5.0 is good, in particular if you use InnoDB, 4.1 was decent, but anything below 4.1 barely qualifies as a database).
distributed databases and P2P (Score:5, Informative)
The problem of distributed consistency has kept researchers occupied for quite a while. For example, see project Scalaris [onscale.de]. They are using a distributed hash table to distribute data among many nodes. This should be relatively easy, at least once you have a good hashing function on your hands. But a lot of research has been done on P2P networks during the last decade, so there is quite a lot of stuff to read and take ideas from.
The interesting part is that it can maintain consistency and support ACID properties. From the site it appears that they accomplish that by using a modified Paxos Algorithm [wikipedia.org] which basically is a way to maintain consensus among many different peers in a non-Byzantine system (this means that there are no malevolent peers in the system -- peers can break down and cease working but not sabotage the system). Leslie Lamport [lamport.org] of Microsoft Research has done a lot of work on this, anyone interested may take a look at his papers, very advanced stuff there.
MySQL not listed as toy ? (Score:2)
Re:MySQL not listed as toy ? (Score:4, Insightful)
MySQL strives to provide RDBMS and ACID semantics, though its quality of service (QoS) may fall short. By contrast, these "slacker" databases don't even try to support RDBMS or ACID; even if they operated perfectly, they won't provide RDBMS/ACID.
I work for one of the companies in question (no, I don't speak for them). We rely heavily on a combination of these "slacker" dbs, Berkeley dbs, memcached, Oracle, flat files, and tape backups. Each fills a niche. I wish these articles would quit trying to create a false dichotomy.
List of non-relation DBs (Score:2)
You young whippersnappers don't know nothing! (Score:5, Interesting)
Relational DB? People forget Network Model Databases (http://en.wikipedia.org/wiki/Network_model) and flat databases.
Network model databases will outperform relational all the time. You just don't have the same flexibility.
Newer models are not based on the design or performance issue, but the distribution of the data. These are not invalid reasons, but the old issues still apply.
I have had arguments with people who consider PC programming different from mainframe. The same rules apply. The difference is that many PC programmers are just sloppier. When you have cheap CPU and memory, people don't analyze and optimize as much.
Re: (Score:2)
That network model looks useful. Too bad there don't seem to be any reality available implementations to try out.
I've never understood the UNIX world's fascination (Score:5, Informative)
I've never understood the UNIX world's fascination with relational databases.
Speaking as a programmer in mainframe online transaction environments for the past 20+ years, I've become very familiar with very fast and simple database systems like the "freespace" files we use on the Unisys mainframe platform.
We don't need relations for real-time processing. Most programs just need a place to keep data, and a simple key to retrieve that data. Some efficiency in disk usage is nice, but the primary design factor is performance.
A freespace file is a collection of pre-allocated fixed-length records of various sizes (e.g. 256 bytes, 512 bytes, 1024 bytes, 2048 bytes, 4096 bytes, and 8192 bytes). Each record size is a assigned a type number (e.g., 1 through 6 in the above case), and a given file is created and pre-allocated with a mix of various records depending on the usage pater for that particular file. If you know all you need is tiny records, create a file containing a few hundred or thousand type 1 and maybe 2 records.
Records not allocated are filled with a deallocated fill pattern.
A program uses a record by performing a Write New operation. That tells the database manager to find a record in that file closest and >= to the size required, stick the presented buffer in the record, save it, and return a key to that record to the calling program. Typical key format is where Record Number is a number from 1 ... n. If your file has 1000 Type 3 records, it'd be from 1...1000 or 0...999.
To read a record, use a key from a previous Write New (stored away somewhere), perhaps in another file) to read that record from a file. Length is not required.
Programs use a very simple read-and-lock mechanism when modifying existing records. If one program has a record locked, another program must wait. Not a problem with intelligent coding.
We've used this system in airline systems for 40+ years. It works well. Sometimes an environment has robust commit and rollback/recovery features to allow for an entire series of changes to be rolled back on error, sometimes not. It doesn't seem to matter that much, especially for transient data like weather, flight schedule data, etc.
I would LOVE to see a freespace database ported to Solaris, personally. We'd use it heavily. :-)
Re: (Score:2)
Oops. Forgot that brackets get eaten. Typical record format is RECORDTYPE/FILENUMBER/RECORDNUMBER. The first Type 1 record for File 100 might look like 01-0127-0001 or whatever (specific binary representation in hex or octal would obviously vary depending on implementation and preference).
In our case, it's a 36-bit word shown as 12 octal digits, probably not a popular choice with UNIX folks. :-)
Re: (Score:2)
01-0127-0001 is the first type 1 record for file 127. 01-0100-0001 would be the first for file 100. That's what I get for doing patchwork re-editing an existing message before sending it...
36-bit words, octal digits, old Unix hackers (Score:2)
Unix hackers are traditionally fine with octal, as long as you don't try to fit a whole digit in it, though I've generally found hex more useful. And as far as 36-bit words go, I know one local Unix hacker who has a PDP-10 in his garage. (Not sure if it's still there, and it might have been a -20 instead.) I don't think my wife's copy of "Meet Macro-10" survived our mid-90s move, and when I took a compiler course at that school, I decided to use the still-clumsy-at-the-time Amdahl mainframe Unix system at
Re:I've never understood the UNIX world's fascinat (Score:5, Insightful)
How does it work for searching though? If I just have my "freespace" file and my pointers to records, does a search for some piece of user requested data have to hit every record or is there a hash somewhere for the data contained in the record? You don't mention it in your description.
It seems that the biggest advantage to a relational DB is that the syntax for accessing it is well known, SQL. It has a human read-able interface and while sometimes whonky to work with for complex operations, it provides the simplest cross-platform way to access data. I don't need to know which data blocks hold the data, I just ask the database for them "SELECT slashdotid, name FROM users where slashdotid 20000"... and I get rows of data.
Could I just read it from a file? Yes. Would it be simpler? Maybe. But what if I have 200001 records, then I have to do some magic sorting in my program, and I have to manage memory for them, and disk space, etc. It is simpler to let the DB handle that mess and I just ask for the data I need.
It breaks up the process of programming into data storage and data manipulation/presentation. DB's for storage, my bad python for manipulation and presentation.
--Donald
Re: (Score:2)
In Re: to my Re:, I like sqlite for simple DB applications, I get DB functionality with a very low overhead. Otherwise I use postgresql.
I have used Oracle and some others before now, but those are my two current DB's (sql-engines?) of choice.
Re: (Score:3, Insightful)
Okay how do you find the data without a record number? I can see the value of the system but it also seems very inflexable.
I do agree that way to many programmer use MySQL for a file system, flat files, configs, and goodness knows what else.
Re: (Score:2)
I would LOVE to see a freespace database ported to Solaris, personally. We'd use it heavily. :-)
Sounds like a great open source project so why not start working on that? If you want it badly and would use it heavily and yet you cannot be bothered to do the work of porting one, writing one, or paying someone else to do it then why bother complaining about it?
Ad-Hoc Complex Queries, Native Unix Toolsets (Score:2)
My guess is that part of the reason is historical - RDBMSs were coming out around the time Unix machines were, and both could be used by small departments as opposed to mainframe production shops.
They're also an extension of the native Unix toolsets, which were flat files with tab-or-comma-separated columns of data, so anybody who learned Unix in its first couple of decades generally had the expectation that you could do ad-hoc queries and build tools to automate them, without needing to spend 6-12 months n
Re: (Score:2)
In my experience, most UNIX programmers tend to assume a relational database for almost everything if it isn't a vanilla flat file. That includes programmers in realtime applications, C people, Java people, etc. I can't begin to tell you how many applications I've seen written to use Oracle, Sybase, etc., just to store a simple static table of information. There's no POINT to that!
Most mainframe environments, on the other hand, have many established options, and relational is usually only considered if y
Re: (Score:2)
The answer is: simplicity and making it somebody else's problem. Think of a typical Slashdot web page. You are logged in to Slashdot so it prints out the data you chose. Specifically, it prints out the groups of data under the topics you chose, in the way (page layout) you chose. You could walk the individual data
Re: (Score:2)
I'm not sure if it is so much age or experience. The things you know end up boxing you in and it can be hard to overcome.
Re: (Score:2)
Why use something not really made for that?
It's simpler to use something already built and tested, with known strengths and weaknesses, multiple mostly-compatible implementations available, tool support, and plenty of books and trained personnel to choose from, than to use a much simpler solution that is less well understood, or worse, one that I have to design and implement myself.
Or, to put it another way, why do the Chinese and Indians do business with each other in English when Esperanto would suffice?
Harsh? (Score:3, Insightful)
I'm a DB admin, and I use things that aren't toys; but what I've heard here is kinda harsh.
Look, it's all about "right tool for the right job." Why do you need a nuclear-powered drill that can make a tunnel from here to China, when really all you needed was a shovel?
For most daily projects that have small amounts of data, they may be using something like Crystal Reports or Excel or SPSS that just does all the number-crunching client-side anyway. You don't always need Oracle or [favorite DB flavor] for that.
I feel old (Score:2, Informative)
When I saw the title I thought "I'm old-guard". Then I read the article and JOINs are a key concept to the old-guard.
My first few DB apps involved using a b-tree or ISAM library (or writing our own). Then the "new guys" started wanting to pay for a server that did JOINs. We did JOINs, just at the app layer and without the guaranteed consitency that a good relational design gives you. And getting a server that does it was expensive.
I wouldn't want to go back to pre-relational server days, but am also ver
Re:I feel old (Score:4, Funny)
I was listening to the radio (didn't pay attention the the station it was on) one day and generally liking the music I was listening to on it. Then the station ID came across between songs. It was the "oldies" station. I suddenly felt like I needed a cane (or perhaps a walker). Why does that happen? And is it going to happen every 10 years or so? I don't think I can take too many more of those moments.
Music from your teenage years gets extra cred (Score:3, Interesting)
It turns out that there actually _are_ neurological reasons that music from your teenage years is extra-evocative, just as language-learning works better with young kids. Go read "This is Your Brain on Music" for more details.
A certain amount of music sensitivity appears to be hardwired into our brains, and the extra hormones after puberty increase music-remembering ability and the emotional aspects of it that younger kids don't have as much of. There's also a lot of intellectual development going on in t
V7 on PDP-11 had joins (Score:2)
Not sure what platform you were using or what years (lots of things had b-trees, though ISAM tended to be on IBM machines), but Unix V7 had a join command, which worked on the canonical tab-delimited ascii flat files that most Unix tools did, and PDP-11s weren't that expensive.
I last used it in the early 90s; I'd prototyped an application in Informix, but my department was too cheap to buy enough licensed copies for production use. You had to sort your data for the join to work, but that also meant you cou
Berkeley DB is awesome (Score:5, Interesting)
I can attest that Berkeley DB does exactly that, and does it really, really well. We use Berkeley DB for all of the data storage in the Citadel [citadel.org] system, including the mailboxes themselves. Some sites have tens of gigabytes or even hundreds of gigabytes of data, and Berkeley DB just keeps chugging along, happily and reliably doing its thing. Our biggest problem? People who point at it and say "storing email in a database is unreliable" because they know it constantly explodes when Exchange does it. Well guess what, folks: Berkeley DB ain't the Exchange database (actually, maybe Exchange wouldn't be so unreliable if they switched to Berkeley DB).
Eschewing the full set of RDBMS features isn't slacking. It's choosing the right tool for the job.
Re:Berkeley DB is awesome (Score:4, Informative)
For others who are interested in Berkeley-style key-value stores, check out Tokyo Cabinet [sourceforge.net].
The problem is growth... (Score:2)
Non-normalized databases are fine, and might be faster, for small sites, but when things scale, the sloppy databases (or worse, sloppy frameworks like Ruby's Active Record) just cause problems.
A scalable, normalized database means consistent data, when you have multiple applications hitting it.
For a web forum, sure, a relational database may be the wrong tool, because all you care about is speed on new stuff, the archive can crawl, etc.
However, what happens when your web forum adds some actual data, and the
Not all databases have to be relational... (Score:2)
For the vast majority of web applications, the "key-value pair" class of databases work fine.
I think the real problem is that the "relational database weenies" look down on the key-value pair databases, and there are a lot of non-DB-weenies out there who like using true relational databases as nothing more than key-value pair. It degenerates into name calling, instead of getting the job done, pretty fast.
Re: (Score:2)
Ignore The Rules At Your Peril (Score:3, Insightful)
You may have seen in the news recently how in the last decade or so Wall Street ignored some of the hard-won regulations and guidelines developed in the wake of the Great Depression.
We all know what happened as a result.
The same is true when dealing with data. You don't ignore the rules completely, or follow them only when you feel like it, or when you have time. As the old joke goes, Quality is *not* Job 1.1.
If the data isn't important enough to store correctly, then it's not important enough to be stored at all.
Just data structures (Score:5, Insightful)
Databases at a very abstract level are just data structures. Choosing a relational database when you don't need that much functionality is just as wrong as choosing a flat file when you need a database.
Knowing the ins & outs of your data structures is still a vital skill of programming.
The problem is scaling (Score:3, Insightful)
so you start a small project, "we just need a few hundred/thousand records, a few key value links and the occasional transaction". so you start with a slacker DB. A slacker DB far too often implies a slacker hack software d00d.
Then it grows. Instead of educating themselves (Q: what's the difference between those who can't read and those who don't? A: nothing. ) and finding a better DB solution they thrash around trying to hack in DB functions into their code.
So they lose consistency etc. Soon they have a polluted DB that breaks all the time. Often they are proud of the heroics of the wasted effort they put into it. A good programmer know how to be correct form of lazy: do not reinvent the wheel.
Re: (Score:2)
YES. Good lazy is "I shouldn't have to do all this work, either use someone else's, or make the computer to it for me." Bad lazy is "whine, I don't want to figure anything out, I just want to get it done." The difference is crucial; the first is willing to spend time learning to save unnecessary labor, the latter is willing to do unnecessary labor to save learning. The former is laziness, the latter is stupidity.
All toys (Score:2, Interesting)
From TFA: "The problem is that JOINs are really, really slow when the data is spread out over several machines."
This is the result of a poor design, not a database flaw. If you are running a web application against multiple databases, either cluster them or store all the data for a user in one database. (i.e. hash the login_id and select the database based on the result). If someone is doing JOINs across multiple machines and doesn't have a very good rea
Mish mash of data storage (Score:2)
Fine, codger together some assemblance of data storage using notepad, access, abacuses, whatever. If, heaven forbid, these "startups" ever took hold and gained any significant size, this "new model" will break, and I can't even imagine the hell it would be to merge, the "new model" into classical rdbms.
Sorry kids, you've bitten off more than you can chew, should have stayed in school and actually attended a class in db modelling. Good luck with this "eventual consistency", you'll need it.
Just 'Insert, Update, and Delete'? (Score:2)
FTA:
The field was surprisingly diverse despite the fact that the offerings are so stripped down that they really don't have more than three major commands: Insert, Update, and Delete.
There's a write-only database now?
Forgotten Technology? (Score:2)
The term "old-school" in this context makes me laugh. Back in the days when air was clean and sex was dirty, "relational" databases were considered a resource hog and were shunned by competent programmers. The fastest and most efficient databases were the "network" databases, but they also required the most work and the trickiest coding. Right in the middle were the "hierarchal" databases. Many programmers avoided the database problem by using a "reverse ISAM" arrangement which still used up some extra reso
Re: (Score:2)
Actually, cartesian products and joins aren't gone. It turns out that they just end up being done client side.
It's the tragedy of "join-less" databases. Joins do something that you need. The lack of joins forces people to correctly normalize, which ironically they should have been doing anyway. It doesn't take away any genuine need to join though. :(
Re: (Score:2)
What is a Cartesian?....is that the water in Olympia Beer?
That wasn't water the Cartesians put the beer. . .
Ask anyone who ever drank Oly and they'll tell you what it was.
Signed,
A former Olympia resident.
SELECT * FROM SNARKY_COMMENTS (Score:5, Funny)
Can't quite fit the whole query into the title box, but if you were using one of those databases that Wayner's article talked about, you'd be able to query and find out if you were first...
Re: (Score:2)
I view these new "ideas" on data storage and retrieval as a dumbing down of DBs the way higher level languages have dumbed down programming.
Yeah, seriously. All you noobs who don't write in machine code should get the fuck off my lawn.
Re: (Score:2)
Better link (Score:2)
Don't judge based on this article. The author's "young guys playing fast and loose" vs. "stuffy but reliable old guys" way of explaining things misses the point. Either he's a bad writer, or he doesn't know what he's talking about. A much better treatment can be found here [allthingsdistributed.com].