Beyond Relational Databases 360
CowboyRobot writes "Relational databases were developed in the 1970s as a way of improving the efficiency of complex systems.
But modern warehousing of data results in terabytes of information that needs to be organized, and the growing prevalence of mobile devices points to the increasing need for intelligent caching on the local hardware.
According to the ACM, the future of database architecture must include more modularity and configuration.
Although no concrete solutions are included, the article is a good overview of the problems with modern data systems."
KISS (Score:5, Insightful)
1) Overly complex
2) Don't scale
3) Tied to a single platform/implementation
4) Poor performance
It's typical to see all four in a single try!
SQL, on the other hand:
1) Reasonably simple API
2) Scales to very large databsaes
3) Cross-platform/architecture
4) Performs very well.
Given the insane amount of inertia SQL has, it will extend into an object model, rather than be replaced by one. (EG: C/C++)
Re:KISS (Score:2, Interesting)
Re:KISS (Score:3, Interesting)
IMHO, there are too many GUI tools out there to do data transformations. SQL works great but most people do
Re:KISS (Score:2, Informative)
In the wrong hands though, such a tool can be made into a piece of crap. The problem is that too many "programmers" don't know SQL befo
Re:KISS (Score:4, Insightful)
As for SQL I do not agree with point 4: SQL does not perform very well. We're in the age of Ghz processors, fast disk drives and it *still* is an performance issue to add a few million records to a database? What SQL sorely lacks is a recognition that the 4th generation of software languages was a bad mistake, and get back to a third generation language: explicit indices, explicit loops over these indices and fast (compiled) execution of said loops. Just freaking program the database instead of waving chicken bones at it.
Re:KISS (Score:3, Informative)
create index t_c_idx on t (c);
creates an index "t_c_idx" on column "c" of table "t".
I've shaved 2 multi-second queries to under 1/10 of a second (it was an overall speedup of over 100X) just by adding indexes and letting them be used (e.g. you can't have where some_function(c)=value and expect the DB to use the index on c).
Re:KISS (Score:3, Interesting)
Re:KISS (Score:3, Interesting)
build multiple indices on a 20M row table, on the fly? Is it worth the effort to build it on the fly if the database can figure out the best way to make and use the indices on its own 99% of the time?
I think that what you're looking for is a database that will allow, but not require, you to tinker programmatically with creating and using temporary indices on the fly. It's still indespensible to have optimizing index functionality in the database
Re:KISS (Score:3, Interesting)
A) will a programmer be expected to do statistical analysis of the data, checking for value skew, and also noting the current load of the system before coming up with the best way to perform a query? This is what happens automatically today, each time you parse a SQL statement.
B) if the skew of data changes, do they have to rew
Re:KISS (Score:4, Insightful)
Google's web index and desktop search facility is a database. I don't know about point 1, but Google definitely blows any relational database out of the water on point 2 to 4.
Google is a very unique case. There are two things in Google's advantage that most RDBMS system have to take into account:
1. Google does not have to update in realtime. If I add a page to my website it is not immediately available on Google. Contrast this to a normal RDBMS where if I add a record it must be available immediately. Google is much more similar to a Data Warehouse than a RDBMS.
2. Websites indexes are more easily parallelized because complex joining of data is not needed. Naively, Google can store all websites starting with 'A' on a server, 'B' on a server, etc. You can store the User table and the Address table on separate database servers and expect to query on users and their addresses with any sort of performance.
3. Going along with 3, the queries expected out of most RDBMS systems are much much more complex than any queries expected out of Google right now. I'm assuming you haven't seen any complex financial reports or statistics that have been generated from a RDBMS. Databases do alot more than "select name from user where id=1234".
Brian
Re:KISS (Score:3, Insightful)
Google's performance and value are both amazing. But it's easy to drive yourself nuts (if you're an enterprise software architect, which I am) trying to get that kind of performance out of other types of application. You see, Google has two major advantages over nearly all other large data-backed applications:
1. There is no "right
Re:KISS (Score:2)
But definitely: stability and acidity of the database are extremely valuable, I just wish I could simply get rid of it sometimes. Again prog
Re:KISS (Score:2)
I think the best solution is to have a good administrative system so you can optimize the database based on system load without changing your code. You end up with the old Oracle / DB administrator but you get a highly tooned system that is still portable.
Re:KISS (Score:4, Insightful)
This is a BIG IMPORTANT POINT. But, unfortunately, the people that are architecting the databases in these cases usually don't realize that they're producing bad designs. All they see is "The database is slow. Why the *!?$ is it so slow!?" There's usually one or two answers: 1) Your schema is organized poorly. 2) You're just plain juggling too much data at once. And let's not forget the deadly combination of both 1 and 2. 100M rows, for example, is a lot of data, any way you look at it. If you've got 100 bytes per row (remarkably small in many cases), that's 10G of data. If you're shuffling through all that data every time, yeah, it's gonna be slow. Ya might want to look into reorganizing, or archiving.
Re:KISS (Score:2)
Usually a piss-poor database reflects on piss-poor developers.
This is a BIG IMPORTANT POINT. But, unfortunately, the people that are architecting the databases in these cases usually don't realize that they're producing bad designs. All they see is "The database is slow. Why the *!?$ is it so slow!?"
There's usually one or two answers:
1) Your schema is organized poorly.
2) You're just plain juggling too much data at once.
And let's not forget the dea
Re:KISS (Score:3, Informative)
I fully agree with this the parent post. I've built several systems (web or otherwise) running on SQL Server over the last 5 years.
I've learned this from experience myself: There is no reason why a database driven application should be slow, provided the database is layed out appropriately and the built-in performance facilities of the RDBMS are utilized.
Show me a slow performing database-driven application, and I will show you a set of indexes, stored procedures, vertical/horizontal table partition
Re:KISS (I can prove SQL will be around) (Score:5, Funny)
1) Reasonably simple API
2) Scales to very large databsaes
3) Cross-platform/architecture
4) Performs very well.
I am proof that SQL will be around for a while. When I first saw Unix back in the late 80s, I thought "this is too hard to use, why would anyone need this?" I have been a Unix/Linux user since about '92.
When I took my first SQL class, I thought "these queries are very cumbersome. SQL is stupid." I still use it today.
In '93 I heard about this thing called the World Wide Web, and thought "This is unnecessary. I can find whatever I need on gopher and ftp sites. Why would I want a gui thrown on top of it?"
As you can see, I am quite the visionary.
Re:KISS (Score:4, Insightful)
In other words, relational databases are very nice and elegant but their interface (SQL) is bad and should be replaced.
Also relational databases by themselves can't supply the needs of a typical enterprise and that's where technologies such as OLAP are built on top of RDBMS to make certain data manipulations efficient.
You are likely correct (Score:2)
The problem is not that you have terabytes of data in total, because you don't deal with the total data. It's no different from swimming - you're only using the surface. The problem is extracting only that surface, so that it can be used on a local device.
The reason this is a problem is
Re:KISS (Score:2, Insightful)
> 1) Reasonably simple API...
The outrageous success of SQL is in part because it complies with the "second law of axiomatic design": a good design has minimum entropy. Prior to SQL a lot of application knowledge was duplicated in the database which increased entropy. Also a lot of knowledge of the physical structure of the data was embedded in the programs which also increased entropy. This is bad because the more redundant information the system contains the harder it is to
Just because something is "old" (Score:4, Insightful)
Just because people are too stupid to take the time to read and understand the theory and learn the application doesn't mean the technology is no longer relevant.
Of course no solutions are proposed. There are none because relational theory is correct, and appropriate for real database driven applications. Little crap bulletin boards can use MySQL.
Netcraft confirms relational databases are dead!
Re:Just because something is "old" (Score:3, Insightful)
Little crap bulletin boards can use MySQL.
Like Slashdot :) Remember yesterday's fiasco where posts were migrating into other articles and the time before that where it happened and the time before that. :)
Why people don't use PostgreSQL is beyond me - unless they don't like fact it is under a BSD license instead of the GPL (please don't get me started on that).
To pronounce PostgreSQL, read the FAQ. (Score:3, Informative)
I think it's more because people look at the name, try to figure out how to pronounce it, then give up
Why, given that this FAQ entry [postgresql.org] clearly states that it's "post-gress-cue-ell"?
Re:Just because something is "old" (Score:2, Insightful)
Instead of blaming the technology and tools, they should improve skills in the Sytem Artchitects and all the way down the road the people involved in Software Development.
Wake me up when it's ready (Score:3, Insightful)
Have been crying for the need to replace relational databases since the early nineties at least.
We can all see where that got them.
. . . no concrete solutions are included. . . (Score:3, Insightful)
KFG
Author's conclusion in case of slashdotting (Score:5, Funny)
SQL Dying...film at 11...NOT! (Score:5, Interesting)
See "COBOL to be replaced...." for an example of just how unlikely that is...sure, the latest hip "Tres Kewl" software for business might be written in something else, but SQL will be around for a long, long time.
Consider just the fact that "Relational Database" technology as laid out by Cobb back in the early days specifically says "You don't *HAVE* to do it this way, but it will be more effecient if you do"...realize that SQL handles Denormalized Warehouse and Datamart tables just as well as it does the 5th normal form model of perfection...and relax...it ain't goin nowhere.
Re:SQL Dying...film at 11...NOT! (Score:3, Insightful)
In the future, databases will be more "good."
While we can't yet go into detail as to how this will all work, suffice it to say that we have a pretty solid idea what the future holds.
Re:5th normal form model of perfection. (Score:3, Insightful)
If a few joins (assuming you've got indexes, etc setup) make it too slow you are too close to the edge.
Saving on hardware but spending more in dealing with data problems is false economy.
Re:5th normal form model of perfection. (Score:3, Funny)
Databases? Bah! (Score:4, Funny)
Nothing builds character like manually searching megabytes of raw, unorganized information for a relevent entry. Except maybe sorting it by hand.
Databases are for sissies.
Re:Databases? Bah! (Score:2, Insightful)
Re:Databases? Bah! (Score:2)
Re:Databases? Bah! (Score:3, Funny)
True, but eventually, you have to give in to the need for less complexity - after all, there's only so many hours in a day.
Now I keep all my "1"'s and "0"'s in two separate containers. This makes it tremendously easy to find exactly what I looking for . . .
Re:Databases? Bah! (Score:3, Funny)
Re:Databases? Bah! (Score:3, Funny)
Wow, someone who likes slashdot's search feature :o
Relational Filesystems (Score:5, Interesting)
Enough stuffing metadata into filenames. Enough shoehorning all data into a file/folder/cabinet model, now less familiar to people than the networked infosystems that mimic them. Enough fake hierarchies inconsistent with accurate data models, forcing whole technologies like Apple Spotlight, GNU Dashboard, and Google Search just to transact basic relatioships buried in the data. Enough reinvention of the wheel with every initial RDBMS schema, just a layer on top of the DB's actual hierarchical filesystem - a shell for an inode database. Enough empty promises of "WinFS" and "OLEDB" vapor - get relational filesystems into developers' hands, and developers will move beyond them, building apps that meet users actual needs, dragging the database tech along.
Re:Relational Filesystems (Score:4, Interesting)
For what purpose? I used to work with a Unisys NX Series (and a predecessor that I don't remember) and it had all data stored in database tables. PC files were stored in new tables with a record length matching the size of the file. It was more a PITA than actually useful.
Enough stuffing metadata into filenames. Enough shoehorning all data into a file/folder/cabinet model, now less familiar to people than the networked infosystems that mimic them. Enough fake hierarchies inconsistent with accurate data models, forcing whole technologies like Apple Spotlight, GNU Dashboard, and Google Search just to transact basic relatioships buried in the data.
You don't need a relational model for that. All you need is a metadata area in the FS, and a metadata indexing scheme. You can then save psudeo-directories as metadata searches. i.e. A bit like the Label system used by GMail. This concept has been done to death in the reseach community, but no one has yet had any success in getting users to accept the idea.
Re:Relational Filesystems (Score:2)
Re:Relational Filesystems (Score:3, Insightful)
Because current RDBMS designs are unsuitable for filesystems. Relational theory still holds (just as it does for OODBs), but the physical design should be quite different if it's going to be effecient.
As I said, this has been beaten to death in the research communities. BeOS even included a DBFS design [nobius.org], but it went largely unused. NTFS also has all the necessary stuff in it, but Microsoft constantly removes it in final releases. Rei
Re:Relational Filesystems (Score:2)
What would work would be a SQL interface to current files, and vice versa. I've tried to get a Samba/MySQL hybrid OSS project going several times in the past 10 years, but the teams have never come together enough to get past architecture. Sometime someone (maybe me) will be able to make the case to a paying customer, a
Re:Relational Filesystems (Score:2)
I won't even trust MySQL to a bulletin board, let alone my files!
Re:Relational Filesystems (Score:2)
Re:Relational Filesystems (Score:4, Informative)
Re:Relational Filesystems (Score:2, Interesting)
I think that this is what Microsoft was trying to do with WinFS. The only problem would be that the RDBMS that replaced the FAT would be an embedded version of SQL Server.
Of course this would be *great* for p2p apps!
Re:Relational Filesystems (Score:2)
Re:Relational Filesystems (Score:2)
Well, that seems fine, except for the fact that you want your filesystems to PERFORM well. You think nothing of launching 10,000 "queries" at your filesystem, but when it comes to dealing with a relational database, you have to back off and think about caching strategies because the database is expensive.
Why? Because that flexib
Re:Relational Filesystems (Score:2)
Re:Relational Filesystems (Score:2)
Second who controls the networked resources? What happens when you use linux as a file server and Windows on one desktop and a Mac on the other?
It can sometimes be a pain keeping Mac metadata together when transfering files on non-mac FS. what happens when Windows has it's own setup.
You know they aren't goin
Re:Relational Filesystems (Score:2)
Re:Relational Filesystems (Score:2)
This might be pretty close to what you're looking for:
http://www.oracle.com/technology/products/ifs/ind
http://www.orafaq.com/faqifs.htm [orafaq.com]
Re:Relational Filesystems (Score:2)
Re:AS-400 (Score:2)
Re:AS-400 (Score:3, Insightful)
AS/400's are hopelessly complex even to seasoned IT professionals such as myself, and they're only around is not because people like them but because the work SO GOD DAMNED WELL :)
point being, tractors work well to but you dont drive one day to day do you? :)
Re:AS-400 (Score:2)
Connectivity (Score:2)
"Organizing" issues aren't as important as getting the data faster and handling a large amount of users. Databases are getting larger and larger. Query optimizations can only improve queries to a certain extent. Eventually the amount of users and data reaches a threshold that more and more hardware doesn't always fix.
SQL isn't a database (Score:5, Informative)
1) Reasonably simple API
2) Scales to very large databsaes
3) Cross-platform/architecture
4) Performs very well.
Given the insane amount of inertia SQL has, it will extend into an object model, rather than be replaced by one. (EG: C/C++)
SQL is a language for set operations. By itself it isn't a database or storage utility. There are some different versions similar to what you describe. Oracle's PL/SQL allows you to make temporary tables and materialized views. Neither solves the overall problem the article describes.
SQL by itself doesn't perform. It is based on the database engine, and how good the developer is. I have gotten SQL queries that took minutes to exectue in seconds by adding indexes, analyzing tables, and totally rewriting inefficient code. It is only "cross-platform" if you follow the ANSI SQL standard. Each database has it's own set of handy functions that make the code database centric.
SQL doesn't really have an API. It is a specification that is sometimes followed by database designers, and sometimes ignored. For example, in Oracle you can either use the ANSI joining sytax (LEFT OUTER JOIN) or use the (+) in the where clause.
It scales to large databases only when they are designed properly. I work with 18 terabytes of data. My sql code wouldn't work so hot if the tables weren't designed correctly. Indexing, partitioning, and table structure have more to do with performance at that level than the code. The code can make a large difference too, but if the underlying structure is wrong, even the best SQL won't help you.
Re:SQL isn't a database (Score:2)
That's kind of disingenuous, though, like saying that "C by itself doesn't perform". That's true, in as much as both are standards and not implementations. However, it's a bit misleading because C, at least, has features that make it inherently difficult to optimize (as compared to other languages like Fortran).
IANA DB theorist, but I could imagine that SQL might also contain some "traps" that might necessarily make its implementations slower than what another query lang
Re:SQL isn't a database (Score:4, Informative)
A truth I hold to be self-evident. The language of SQL provides all the tools you need to make your application perform well, as you state.
SQL doesn't really have an API.
Realistically, SQL is an API. It's a highly abstracted interface for communicating between two programs. (your app, and the DB server software)
It is only "cross-platform" if you follow the ANSI SQL standard.
Sorta. See, I can write a script using PHP with a particular SQL call, and do the same thing in Perl, Java, ASP, C, C++, Python, Ruby, and even BASH, on Linux, Windows, Mac, or just about anything else with a tcp stack and a compiler. Sure, SQL implementations are different, with various shortcuts and extensions, but I'd call that cross platform if ever there was one.
Let me ask you this: How often do you see an OSS product (EG: phpwiki) that doesn't offer support for numerous databases?
JDBC further standardizes SQL dialects (Score:3, Insightful)
Re:SQL isn't a database (Score:3, Insightful)
Quite a bit actually. In my experience most only support MySQL. Why? They don't understand the value of data integrity and the features MySQL lacks/lacked. As a result they have had to poorly implement things that the DB itself should do, and when they try to "port" this to another DB they run into all sorts of issues.
And SQL is a language, not an API.
So they give up.
Synchronization (Score:4, Informative)
Synchronization for a mobile device has another main requirement, robustness when the connection to the server is lost. A mobile device has to gracefully handle when the owner runs down into the subway.
It's already done (Score:2, Interesting)
This is absolutely correct when referring to databases for different application. However, why do people always assume that they have to choose the Oracle's, MS and IBM's out there? There are already databases that have been tailored for certain application environments. Take for example http://www.ianywhere.com/ [ianywhere.com] who has databases like SQL Anywhere and UltraLite which are tailored for smaller workgroups and mobile devices.
I don't think the solution to the problem is to buil
Why 'Beyond'? (Score:5, Insightful)
Essayist Clay Shirky has gone to far as to suggest that MySQL is at the center of a whole new software movement [shirky.com].
In my experience with Web applicaions the chief problem with the RDBMS seems to be that it does not do text indexing and search very well, so I have to keep a second store of data in something like Lucene.
The other major problem is the level of skill required to tune the database to achieve high-performance SQL queries, so hopefully the RDBMS will evolve with more self-configuration capability.
The article, which I only skimmed, actually addresses these two concerns but seems to pooh-pooh the notion of simply refining the existing RDBMS systems. Instead it says " Old-style database systems solve old-style problems; we need new-style databases to solve new-style problems. "
The paper seems awfully squishy on what this means. The clearest I found was a call to "produce a storage engine that is more configurable so that it can be tuned to the requirements of individual applications."
But this call for new highly modular/configurable storage "engines" seems to me to require at least as much fussy care and feeding as a traditional RDBMS. You're just replacing one DBA with another. And throwing out decades of refinement in the process.
The raison d'etre of the RDBMS is to allow the programmer to treat storage as a black box while gaining nifty ACID features. Extending this to text indexing seems logical.
Kind of OT SQL Question (Score:2, Insightful)
As I am designing more and more complex web apps, I am constantly having to think of new, innovative ways to design the tables and databases and am currently making it up as I go. Does anyone have a reccomendation for books/sites that talk about good design proactices, that is not "How to use SQL" and relatively agnostic on the specific brand on DB?
Sorry for the OT post, its just something that has been buggi
Re:Kind of OT SQL Question (Score:2, Informative)
I know it says access on the cover, but Steve Roman's book Access Database Design & Programming [amazon.co.uk] is pretty good as a starting point, IMO.
You could also try Michael Hernandez's Database design for mere mortals [amazon.co.uk]
I found both to be pretty readable and a good introduction to the theory.
Once you've got the basics down, there's lots of further stuff you can try, from the obvious books by Codd and Date, to the more esoteri
According who WHOM? (Score:2)
According to the ACM...
No... Acording to Sleepycat [sleepycat.com], who have a great name and logo, but an otherwise very annoying data store.
the usual database blatherings (Score:2, Insightful)
by MARGO SELTZER, SLEEPYCAT
Sleepycat? The guys who make a brain-dead key/value database with no data manipulation or integrity capabilities? Who are they to educate others on the topic of relational databases? (Sleepycat's products are useful tools, but they are not true databases).
while data management has become almost synonymous with RDBMS, however, there are an increasing number of applications for which lighter-weight alternatives are more appropriate.
Ahh, so the proper title of this paper sh
A thought on XML documents (Score:3, Interesting)
I can imagine XML documents created in such a manner that they could constitute an object from an OOP (Object Oriented Programming) perspective, containing their own schema, characteristics, relationships and data. Further, I can imagine the ability of accepting such an XML document object into a range of other things, such as a modular program (dynamic program extensibility) or a database (temporary database extension). I can imagine the ability to package up such XML documents so that databases can be built simply by linking the XML documents together.
So (for example), one could send a request to the XML index in the sky, find and link documents containing a subset of know medical facts, and then kick off a data mining process that could discover previously unknown medical relationships. All without needing to know anything other than where to find the XML files.
Now imagine a tool that could convert all of the terabytes of data the world is generating every day into small, linkable, OOP like XML files... Sounds like a great open source project to me...
Re:A thought on XML documents (Score:5, Insightful)
XML documents are an hierarchical database (Score:3, Insightful)
that's why XML databases flopped
Improving the efficiency???? (Score:5, Informative)
When relational systems finally began to appear (and I'm thinking specifically about IBM's System R) they were dog slow, and the extant hierarchical and CODASYL network databases of the day ran rings around them. Still do, unless you throw lots of hardware at the RDBMS.
RDBMS have lots of advantages over older technologies, but performance is not among them.
Bah... (Score:2, Funny)
The future exists today (Score:2, Informative)
MUMPS (Score:4, Interesting)
As for the future, I think that while OO databases are all the vogue, that O databases, where each bit of information is represented by it's own object and will have the ability to demonstrate autonomous agency is a good area for research. Then we can just let the data speak for itself! ;^)
"Every person takes the limits of their own field of vision for the limits of the world."
- Arthur Schopenhauer
No data integrity (Score:3, Interesting)
The idea of the relational model is simple: it is based on set theory, which has a strong mathematical model. There is no equivelent model for object databases, no
Re:MUMPS (Score:2)
Sleepycat? (Score:3, Interesting)
To me, this entire article reads like a plug for Sleepycat [sleepycat.com], written, not surprisingly, by a Sleepycat person.
Relational data model is just that -- a data model. It doesn't concern itself much with implementation (and therefore, performance in any particular environment) or with how applications use the data. And that's really the point -- relational databases are application-agnostic. They are designed to store the data that will be possibly accessed by applications that are not yet conceived. That's the reason they put great emphasis on internal correctness of the data. Once you have a database specialized for one application, that's not really a database in the relational sense -- that's a way to persist your application data. And that's where Berkeley DB [sleepycat.com] shines. It doesn't replace relational databases, it just serves a totally different purpose.
How about.... (Score:5, Informative)
How about... a query language that is fully set operations compliant, i.e., something other than ANSI SQL which is a strange mixture of set and bag operations, and a mixture of relational algebra and relational calculas and some other 'extensions'.
How about... realizing that a major design goal for the relational model was data integrity. Modularity and configurability are also good goals but if you are serious about your data, integrity will be at the top of the list.
The biggest problems I see with databases is very few people understand how to use them. Here's a few tips:
1) a table is *not* a class or an object. Tables + constraints + user defined types + constraints etc. when used properly can define domains which are close to classes and objects.
2) Learn how to normalize. A badly (or flat out not) normalized database threatens data integrity by violating the once-and-only once rule. As a rule of thumb if the table has more than 20 fields in it you should review your data model and make sure it is properly normalized.
3) Point 2 is often the consequence of mindlessly slurping in spread sheets or MS Access database tables. Anyone doing this has no business being within 50 feet of an IDE.
4) Ditch Raid 5. 0+1 will give better perfomance in most cases. Manager like Raid 5 because it is cheap, you get what you pay for.
5) Have multiple channels for data, transaction logs, large indices and O/S or user applications to reduce bottle necks. This is expensive but for large databases going cheap will hurt you.
6) Learn a little theory, it won't hurt you. In fact it can save a large amount of time and trouble. Do not be afraid of learning about the technology you are using. After all, technology is what you are good at, right?
7) If it is a read only database, turn off logging for speed (impossible to due under SQL Server 2000 btw). Also, if a table is on a purge and load paradigm (many reporting and/or datawarehouse tables are) turn off logging on the table level if your version of database engine allows you to do so. Likewise, turning off logging on a hand held or other single user system may be appropriate, just make sure two people do not try to use the database at the same time.
8) Avoid XML. Too much bloat.
9) Learn how to use indices on tables.
10) Learn how to read a perfomance monitor/top etc.
Postgresql is both working hard to become truly relational AND is adding support for geographic objects and objects. The MySQL crew is working hard to improve. Oracle has some nice perfomance features but I think their 'Object/Relational' implementation is broken. SQL Server is getting 'long in the tooth'. There is also a great need for temporal databases and lightwieght engines. But remember, there is no 'silver bullet', no short cuts. Just hard work to be done.
Re:How about.... (Score:2, Insightful)
1) a table is *not* a class or an object.
Good advice. A table is simply a snapshot of a relational value. True relational values have no top/bottom or left/right ordering so you can't really show them on paper or on the screen. It doesn't make sense to map classes or objects to tables.
2) Learn how to normalize. A badly (or flat out not) normalized database threatens data integrity by violating the once-and-only once rule.
Normalization has nothing to do with integrity per se. You can add constraint
sometimes all you need is simple object store (Score:2)
Hmm .. (Score:5, Informative)
She puts forth a case against SQL and relational databases in general and claims that many applications (like directory services and search engines) have read-heavy, hierarchial access patterns which favour lighter-weight, non-relational, transaction-optional databases.
And
Issue of dimensions in RDBMS (Score:3, Interesting)
This is a fairly common question in data warehousing: What is the data today, what did it look like yesterday, last week, and last year?
I have seen it worked around in silly ways (snapshot and rename a table every day/week/month) and more clever ways (use separate transaction tables to record changes), but never in a particularly elegant way.
Wiser colleagues whispered to me the dirty answer "object relational" and scurried away to their dens of Rob Zombie and J2EE. I never got my head around object relational databases before leaving that world, and so am left to ponder papers [ca.com] from IDC with statements like this one:
"putting object extensions on RDBMSs is tantamount to adding stereo radios and global navigation systems to horse-drawn carriages"
Ouch, is that a swipe at Oracle? Seems that as far back as 1997 pundits have said that the future is in ODBMS, and not RDBMS or ORDBMS. Hmm...
JSR 170 - Unified Data Repository (Score:4, Informative)
Standardizing the interfaces to various data resources (filesystem, database, cache,
The expert group reads like a who's who in data management. And it seems to be very near to the final draft.
Another one? (Score:3, Funny)
What I'd really like to be able to do... (Score:2, Interesting)
Sort of like stored procedures in implementation - they could be called stored query definitions.
Because these query definitions would alread
Re:I did not RTFA (Score:5, Insightful)
Or the summary
mySQL suits me quite well.
That's nice. It won't handle a multi-terabyte database, though. That's the domain of Terabase, Oracle, and (blech) DB2. It's also what the article is about.
The power of PHP and mySQL is all I need.
And a moped is all you need to get to work. If you want to haul 300 metric tons of rock from point A to point B, you need a dump truck. Again, that's what this article is about.
Back on topic, this entire article is mostly speculative for the moment. A lot of excellent work has been done in OODB and XMLDB designs, but no singular design has yet emerged to solve all our woes. For example, I love the Prevayler [prevayler.org] concept. It solves a lot of problems, lowers data access times, and provides for complete data security. It also isn't usable or scalable without a lot more design work.
The future will hold some very interesting things, but for now we'll have to keep inventing until we come up with a consolidated solution.
Re:I did not RTFA (Score:5, Insightful)
Re:I did not RTFA (Score:2)
Re:I did not RTFA (Score:2)
Re:I did not RTFA (Score:2)
Re:Multi-Terabyte Data Warehouse and MySQL (Score:5, Interesting)
The holy grail of information technology would be to eliminate the need for such cumbersome replications, and instead have a single, reliable data source that can be queried for any information needed at any time. Unfortunately, MySQL isn't it.
Re:I did not RTFA (Score:2)
Can you back that up with some real-world examples where DB2 was worse than Oracle?
I could, but it would only start a useless argument over what database everyone prefers. Let's just say that my experience with DB2 has left me with less than stellar feelings toward that database and leave it at that.
FWIW, my experience is with UDB and not the Mainframe DB2. At the end of the day, the two are very different beasts.
Re:I did not RTFA (Score:3, Interesting)
Last time I did a comparison, it was DB2 7.1 vs. Oracle 8. Most of what bit me in the ass with DB2 was its flakey management tools and multitude of minor details that needed tweaking from the days when it was a mainframe database. (e.g. Why does it need a buffer large enough to hold the entire blob chunk that's going to be transferred? That's just stupid. It should pull across as
Re:I did not RTFA (Score:5, Funny)
Re:Object-relation databases (Score:2)
Re:Object-relation databases (Score:2)
Object-relation databases has been the future of databases for at least 10 years now.
And it remains in the future.
Maybe it'll happen some day but so far I haven't seen anything happen in the industry.
support for XML in databases is far hotter.
ORDB is too complicated (Score:2)
They should be simple, but they're not. Plus, the implementation tends to tie you to a given vendor. It's not a real problem since most people never move off of their database, but people don't like thinking about it too much.
The relational model is nice because it's easy to understand, and you can always flatten an object graph into tables if you need to. You can also poke around and visually see the structure, which is nice.
Lastly, there's
Re:PostgreSQL is object-relational (Score:2)
There are many large companies (finance, insurance, shipping, utilities, etc.) who are using GemStone (www.gemstone.com) very successfully.
We use GemStone as our persistence layer and application server on my project (at a major bank).
The only limitation at the moment is it only provides 32 bit support, so we are limited to about one billion objects, but I believe GemStone are working on a 64 bit version.
It can be kind o