Become a fan of Slashdot on Facebook

 


Forgot your password?
Close
typodupeerror
×
Databases Programming Software IT

Beyond Relational Databases 360

Posted by Zonk from the data-access-for-fun-and-profit dept.
CowboyRobot writes "Relational databases were developed in the 1970s as a way of improving the efficiency of complex systems. But modern warehousing of data results in terabytes of information that needs to be organized, and the growing prevalence of mobile devices points to the increasing need for intelligent caching on the local hardware. According to the ACM, the future of database architecture must include more modularity and configuration. Although no concrete solutions are included, the article is a good overview of the problems with modern data systems."
This discussion has been archived. No new comments can be posted.

Beyond Relational Databases More

Beyond Relational Databases

Comments Filter:

  • KISS (Score:5, Insightful)

    by mcrbids ( 148650 ) on Tuesday May 24, 2005 @03:14PM (#12626311) Journal
    Some of the biggest problems that "new" database designs have:

    1) Overly complex

    2) Don't scale

    3) Tied to a single platform/implementation

    4) Poor performance

    It's typical to see all four in a single try!

    SQL, on the other hand:

    1) Reasonably simple API

    2) Scales to very large databsaes

    3) Cross-platform/architecture

    4) Performs very well.

    Given the insane amount of inertia SQL has, it will extend into an object model, rather than be replaced by one. (EG: C/C++)

    • Re:KISS (Score:2, Interesting)

      by mmkkbb ( 816035 )
      There is a language called OOSQL, but looking at it makes me squirm.

    • Re:KISS (Score:3, Interesting)

      by eggoeater ( 704775 )
      Very true. My group had to rebuild an overly-complicated MIS system for a large call center about 5 years ago. The old system was built with RedBrick DB engine, informatica data transformations (i.e. pretty GUI..oooooohhhhh), and Brio reporting. It was a mess and NEVER worked. We replaced it with SQL Server and some relatively simple SQL stored procs and dumped the POS informatica and RedBrick.

      IMHO, there are too many GUI tools out there to do data transformations. SQL works great but most people do

      • Re:KISS (Score:2, Informative)

        by 3nuff ( 824173 )
        I'm currently working on one of those GUI applications, not Informatica. I am pleased with it's capabillities and I think it's a very powerful and reliable tool. Big projects can take a couple days once a good template is built and the metadata management is simple and useful. Reusable code and SMP processing are available and greatly enhance the speed of data transfer.

        In the wrong hands though, such a tool can be made into a piece of crap. The problem is that too many "programmers" don't know SQL befo

    • Re:KISS (Score:4, Insightful)

      by NoOneInParticular ( 221808 ) on Tuesday May 24, 2005 @03:51PM (#12626675)
      Google's web index and desktop search facility is a database. I don't know about point 1, but Google definitely blows any relational database out of the water on point 2 to 4.

      As for SQL I do not agree with point 4: SQL does not perform very well. We're in the age of Ghz processors, fast disk drives and it *still* is an performance issue to add a few million records to a database? What SQL sorely lacks is a recognition that the 4th generation of software languages was a bad mistake, and get back to a third generation language: explicit indices, explicit loops over these indices and fast (compiled) execution of said loops. Just freaking program the database instead of waving chicken bones at it.

      • Re:KISS (Score:3, Informative)

        by Frank T. Lofaro Jr. ( 142215 )
        Umm SQL does have explicit indexes and they make a big difference:

        create index t_c_idx on t (c);

        creates an index "t_c_idx" on column "c" of table "t".

        I've shaved 2 multi-second queries to under 1/10 of a second (it was an overall speedup of over 100X) just by adding indexes and letting them be used (e.g. you can't have where some_function(c)=value and expect the DB to use the index on c).

        • Re:KISS (Score:3, Interesting)

          by NoOneInParticular ( 221808 )
          Yes, you can expect the DB to use the index, which is subtly different from instructing the database to use the index. The whole idea behind SQL is great: a declarative language for specifying the job, and a procedural, smart, engine for performing the job. Unfortunately, databases have failed to deliver on the latter part, leading to implicit techniques to get the performance out. What I claim here is that we should take a step back, not forward, with SQL and get back to explicit programming of the databas

          • Re:KISS (Score:3, Interesting)

            by WaterBreath ( 812358 )
            instruct the query to first make and then use the index

            build multiple indices on a 20M row table, on the fly? Is it worth the effort to build it on the fly if the database can figure out the best way to make and use the indices on its own 99% of the time?

            I think that what you're looking for is a database that will allow, but not require, you to tinker programmatically with creating and using temporary indices on the fly. It's still indespensible to have optimizing index functionality in the database

          • Re:KISS (Score:3, Interesting)

            by Stu Charlton ( 1311 )
            I disagree that a programmer can manually create a better query plan than (for example -- others apply) Oracle 10g. In certain cases -- perhaps. A couple of questions though....

            A) will a programmer be expected to do statistical analysis of the data, checking for value skew, and also noting the current load of the system before coming up with the best way to perform a query? This is what happens automatically today, each time you parse a SQL statement.

            B) if the skew of data changes, do they have to rew

      • Re:KISS (Score:4, Insightful)

        by Brian_Ellenberger ( 308720 ) on Tuesday May 24, 2005 @06:20PM (#12628344)

        Google's web index and desktop search facility is a database. I don't know about point 1, but Google definitely blows any relational database out of the water on point 2 to 4.



        Google is a very unique case. There are two things in Google's advantage that most RDBMS system have to take into account:

        1. Google does not have to update in realtime. If I add a page to my website it is not immediately available on Google. Contrast this to a normal RDBMS where if I add a record it must be available immediately. Google is much more similar to a Data Warehouse than a RDBMS.

        2. Websites indexes are more easily parallelized because complex joining of data is not needed. Naively, Google can store all websites starting with 'A' on a server, 'B' on a server, etc. You can store the User table and the Address table on separate database servers and expect to query on users and their addresses with any sort of performance.

        3. Going along with 3, the queries expected out of most RDBMS systems are much much more complex than any queries expected out of Google right now. I'm assuming you haven't seen any complex financial reports or statistics that have been generated from a RDBMS. Databases do alot more than "select name from user where id=1234".



        Brian

      • Re:KISS (Score:3, Insightful)

        by isomeme ( 177414 )
        Google's web index and desktop search facility is a database. I don't know about point 1, but Google definitely blows any relational database out of the water on point 2 to 4.

        Google's performance and value are both amazing. But it's easy to drive yourself nuts (if you're an enterprise software architect, which I am) trying to get that kind of performance out of other types of application. You see, Google has two major advantages over nearly all other large data-backed applications:

        1. There is no "right

    • Re:KISS (I can prove SQL will be around) (Score:5, Funny)

      by gosand ( 234100 ) on Tuesday May 24, 2005 @03:58PM (#12626780)
      SQL, on the other hand:
      1) Reasonably simple API
      2) Scales to very large databsaes
      3) Cross-platform/architecture
      4) Performs very well.

      I am proof that SQL will be around for a while. When I first saw Unix back in the late 80s, I thought "this is too hard to use, why would anyone need this?" I have been a Unix/Linux user since about '92.

      When I took my first SQL class, I thought "these queries are very cumbersome. SQL is stupid." I still use it today.

      In '93 I heard about this thing called the World Wide Web, and thought "This is unnecessary. I can find whatever I need on gopher and ftp sites. Why would I want a gui thrown on top of it?"

      As you can see, I am quite the visionary.

    • Re:KISS (Score:4, Insightful)

      by MSBob ( 307239 ) on Tuesday May 24, 2005 @04:00PM (#12626803)
      Relational algebra is a very nice and tidy concept but SQL is a piss poor (and limited) implementation thereof.

      In other words, relational databases are very nice and elegant but their interface (SQL) is bad and should be replaced.

      Also relational databases by themselves can't supply the needs of a typical enterprise and that's where technologies such as OLAP are built on top of RDBMS to make certain data manipulations efficient.

    • Indeed, the most "obvious" solution would be to have views (which are normally virtual tables) become physical tables on the remote device. Then, only the data that is actually important would be stored on local devices.

      The problem is not that you have terabytes of data in total, because you don't deal with the total data. It's no different from swimming - you're only using the surface. The problem is extracting only that surface, so that it can be used on a local device.

      The reason this is a problem is

    • Re:KISS (Score:2, Insightful)

      by waveman ( 66141 )
      > SQL, on the other hand:
      > 1) Reasonably simple API...

      The outrageous success of SQL is in part because it complies with the "second law of axiomatic design": a good design has minimum entropy. Prior to SQL a lot of application knowledge was duplicated in the database which increased entropy. Also a lot of knowledge of the physical structure of the data was embedded in the programs which also increased entropy. This is bad because the more redundant information the system contains the harder it is to

  • Just because something is "old" (Score:4, Insightful)

    by Anonymous Coward on Tuesday May 24, 2005 @03:17PM (#12626348)
    Doesn't make it obsolete. "Databases are old and kludgey. Teh suXX0rs for R0xxng H4XX0rs liek me.

    Just because people are too stupid to take the time to read and understand the theory and learn the application doesn't mean the technology is no longer relevant.

    Of course no solutions are proposed. There are none because relational theory is correct, and appropriate for real database driven applications. Little crap bulletin boards can use MySQL.

    Netcraft confirms relational databases are dead!

    • Little crap bulletin boards can use MySQL.

      Like Slashdot :) Remember yesterday's fiasco where posts were migrating into other articles and the time before that where it happened and the time before that. :)

      Why people don't use PostgreSQL is beyond me - unless they don't like fact it is under a BSD license instead of the GPL (please don't get me started on that).

    • I concur with this opinion, I've seen a lot of supposedly 'Top notch' designs and I've seen flaws that made me blush myself....mostly the problem is with stupid design mistakes and poor concepts.

      Instead of blaming the technology and tools, they should improve skills in the Sytem Artchitects and all the way down the road the people involved in Software Development.

  • Wake me up when it's ready (Score:3, Insightful)

    by Vile Slime ( 638816 ) on Tuesday May 24, 2005 @03:17PM (#12626349)
    People,

    Have been crying for the need to replace relational databases since the early nineties at least.

    We can all see where that got them.

  • . . . no concrete solutions are included. . . (Score:3, Insightful)

    by kfg ( 145172 ) on Tuesday May 24, 2005 @03:18PM (#12626352)
    Funny how they never are, eh?

    KFG

  • Author's conclusion in case of slashdotting (Score:5, Funny)

    by Anonymous Coward on Tuesday May 24, 2005 @03:19PM (#12626365)
    The future will not be found in the relational model, object model, or hybrid, but in the comma-delimited list.

  • SQL Dying...film at 11...NOT! (Score:5, Interesting)

    by Gorm the DBA ( 581373 ) on Tuesday May 24, 2005 @03:19PM (#12626379) Journal
    Wow..."SQL and Relational Databases to be replaced by new technology"...film at 11.

    See "COBOL to be replaced...." for an example of just how unlikely that is...sure, the latest hip "Tres Kewl" software for business might be written in something else, but SQL will be around for a long, long time.

    Consider just the fact that "Relational Database" technology as laid out by Cobb back in the early days specifically says "You don't *HAVE* to do it this way, but it will be more effecient if you do"...realize that SQL handles Denormalized Warehouse and Datamart tables just as well as it does the 5th normal form model of perfection...and relax...it ain't goin nowhere.

    • After extensive research, our Databases Of The Future department has come up with the following prediction:

      In the future, databases will be more "good."

      While we can't yet go into detail as to how this will all work, suffice it to say that we have a pretty solid idea what the future holds.

  • Databases? Bah! (Score:4, Funny)

    by GillBates0 ( 664202 ) on Tuesday May 24, 2005 @03:20PM (#12626388) Homepage Journal
    Real men work with raw data.

    Nothing builds character like manually searching megabytes of raw, unorganized information for a relevent entry. Except maybe sorting it by hand.

    Databases are for sissies.

    • Re:Databases? Bah! (Score:2, Insightful)

      by techwolf ( 26278 )
      That's what Perl is for...
    • Nothing builds character like manually searching megabytes of raw, unorganized information for a relevent entry. Except maybe sorting it by hand.

      True, but eventually, you have to give in to the need for less complexity - after all, there's only so many hours in a day.

      Now I keep all my "1"'s and "0"'s in two separate containers. This makes it tremendously easy to find exactly what I looking for . . .

    • As are filesystems. "Real men" grep their block devices for magic numbers like "ReIsEr34" when looking for the start of their data.
    • Nothing builds character like manually searching megabytes of raw, unorganized information for a relevent entry

      Wow, someone who likes slashdot's search feature :o

  • Relational Filesystems (Score:5, Interesting)

    by Doc Ruby ( 173196 ) on Tuesday May 24, 2005 @03:21PM (#12626390) Homepage Journal
    How about just getting filesystems to be relational? Replace the ancient 1960s-era hierarchical inode database that underlies filesystems with a modern relational one. Then distributed databases can provide a more consistent platform for all our distributed apps.

    Enough stuffing metadata into filenames. Enough shoehorning all data into a file/folder/cabinet model, now less familiar to people than the networked infosystems that mimic them. Enough fake hierarchies inconsistent with accurate data models, forcing whole technologies like Apple Spotlight, GNU Dashboard, and Google Search just to transact basic relatioships buried in the data. Enough reinvention of the wheel with every initial RDBMS schema, just a layer on top of the DB's actual hierarchical filesystem - a shell for an inode database. Enough empty promises of "WinFS" and "OLEDB" vapor - get relational filesystems into developers' hands, and developers will move beyond them, building apps that meet users actual needs, dragging the database tech along.
    • How about just getting filesystems to be relational?

      For what purpose? I used to work with a Unisys NX Series (and a predecessor that I don't remember) and it had all data stored in database tables. PC files were stored in new tables with a record length matching the size of the file. It was more a PITA than actually useful.

      Enough stuffing metadata into filenames. Enough shoehorning all data into a file/folder/cabinet model, now less familiar to people than the networked infosystems that mimic them. Enough fake hierarchies inconsistent with accurate data models, forcing whole technologies like Apple Spotlight, GNU Dashboard, and Google Search just to transact basic relatioships buried in the data.

      You don't need a relational model for that. All you need is a metadata area in the FS, and a metadata indexing scheme. You can then save psudeo-directories as metadata searches. i.e. A bit like the Label system used by GMail. This concept has been done to death in the reseach community, but no one has yet had any success in getting users to accept the idea.
      • You just described the relational "views" feature. Others have other relational features at the top of their priorities. A RDBMS, instead of the hierarchical DBMS we use for our filesystems, offers your feature, their feature, my feature, all in a package. Which is well understood, with lots of applications that use it. We've already got RDBMS tech - why reinvent an inadequate version of it?
        • We've already got RDBMS tech - why reinvent an inadequate version of it?

          Because current RDBMS designs are unsuitable for filesystems. Relational theory still holds (just as it does for OODBs), but the physical design should be quite different if it's going to be effecient.

          As I said, this has been beaten to death in the research communities. BeOS even included a DBFS design [nobius.org], but it went largely unused. NTFS also has all the necessary stuff in it, but Microsoft constantly removes it in final releases. Rei
          • BeOS itself was largely unused. NTFS never exposed the features for apps. And ReiserFS doesn't have a simple API, or critical mass of RDBMS features. It takes that critical mass to catch on.

            What would work would be a SQL interface to current files, and vice versa. I've tried to get a Samba/MySQL hybrid OSS project going several times in the past 10 years, but the teams have never come together enough to get past architecture. Sometime someone (maybe me) will be able to make the case to a paying customer, a
            • Try Samba + PostgreSQL.

              I won't even trust MySQL to a bulletin board, let alone my files!
              • The key to getting a project like that to succeed is to get something, anything, to work - even a little bit. Then upgrade the parts that don't work. MySQL is much simpler, so much less overhead in hacking it into Samba. And it's got a much larger developer community. But even so, I couldn't get a committed team. Maybe next time.

    • Re:Relational Filesystems (Score:4, Informative)

      by turgid ( 580780 ) on Tuesday May 24, 2005 @03:54PM (#12626720) Journal
      The Pick [wikipedia.org] OS.
    • How about just getting filesystems to be relational? Replace the ancient 1960s-era hierarchical inode database that underlies filesystems with a modern relational one. Then distributed databases can provide a more consistent platform for all our distributed apps.

      I think that this is what Microsoft was trying to do with WinFS. The only problem would be that the RDBMS that replaced the FAT would be an embedded version of SQL Server.

      Of course this would be *great* for p2p apps!
      • That's what I meant by 'Enough empty promises of "WinFS" and "OLEDB" vapor'. What MS was (now so obviously) trying to do with WinFS was lie about innovation, while selling the same old crap in new bags. There's probably an "integration" angle that a working version would offer to MS competitors, that MS couldn't monopolize (secret APIs are harder in DB schemas than in DLLs). But we'll never know, because MS has announced this tech as "immanent" for decades. All we know for sure is that it's popular enough i
    • "Enough stuffing metadata into filenames. Enough shoehorning all data into a file/folder/cabinet model, now less familiar to people than the networked infosystems that mimic them."

      Well, that seems fine, except for the fact that you want your filesystems to PERFORM well. You think nothing of launching 10,000 "queries" at your filesystem, but when it comes to dealing with a relational database, you have to back off and think about caching strategies because the database is expensive.

      Why? Because that flexib
      • IBM's extremely popular AS/400, in its "i5 server" version, uses DB2 as its fileystem. Those apps are very demanding - of performance, reliability, and flexibility. Reiser adds RDBMS features with every release, and might evolve into a true DB/FS hybrid. At the very least it proves both the concept, and its appropriateness to actual apps.
    • Two big points. Relational databases are slow at accessing data quickly. IE it's fast enough for web browsing but do you want your game to have a few milisecond lag, so it can load up a character?

      Second who controls the networked resources? What happens when you use linux as a file server and Windows on one desktop and a Mac on the other?

      It can sometimes be a pain keeping Mac metadata together when transfering files on non-mac FS. what happens when Windows has it's own setup.

      You know they aren't goin
      • One reason RDBMS'es are slower than inode DBs (filesystems) is that the RDBMS is just a layer on the inode DB. Another is the inclusion of many features not necessary for a filesystem. Strip down the RDBMS to its essentials, and let it write directly to the disk geometry (with bus interrupts), maybe even integrated directly into the disk HW, and performance will compete directly with inode filesystems. Maybe even exceed them, especially in complex storage systems, even RAIDs. And certainly when an app calls
    • Have a look at Oracle iFS (recently renamed to "Oracle Content Management SDK", apparently)
      This might be pretty close to what you're looking for:
      http://www.oracle.com/technology/products/ifs/inde x.html [oracle.com]
      http://www.orafaq.com/faqifs.htm [orafaq.com]

      • Yep, they rolled out a halfassed version with Oracle 8i, and seem to be getting there with 10g. IBM is also trailblazing with i5 Server, an AS/400 version with DB2 for its filesystem. Today's minicomputer architectures are tomorrow's distributed PC platforms. Once companies like IBM and Oracle stabilize the approach, Linux filesystem developers will be there to make it cheap, flexible, and available to everyone.
  • Local caching data gets ugly. Eventually the connectivity issues will be fixed. Mobile devices will seamlessly connect to databases preventing the need for caching.

    "Organizing" issues aren't as important as getting the data faster and handling a large amount of users. Databases are getting larger and larger. Query optimizations can only improve queries to a certain extent. Eventually the amount of users and data reaches a threshold that more and more hardware doesn't always fix.

  • SQL isn't a database (Score:5, Informative)

    by Nytewynd ( 829901 ) on Tuesday May 24, 2005 @03:30PM (#12626477)
    SQL, on the other hand:
    1) Reasonably simple API
    2) Scales to very large databsaes
    3) Cross-platform/architecture
    4) Performs very well.
    Given the insane amount of inertia SQL has, it will extend into an object model, rather than be replaced by one. (EG: C/C++)

    SQL is a language for set operations. By itself it isn't a database or storage utility. There are some different versions similar to what you describe. Oracle's PL/SQL allows you to make temporary tables and materialized views. Neither solves the overall problem the article describes.

    SQL by itself doesn't perform. It is based on the database engine, and how good the developer is. I have gotten SQL queries that took minutes to exectue in seconds by adding indexes, analyzing tables, and totally rewriting inefficient code. It is only "cross-platform" if you follow the ANSI SQL standard. Each database has it's own set of handy functions that make the code database centric.

    SQL doesn't really have an API. It is a specification that is sometimes followed by database designers, and sometimes ignored. For example, in Oracle you can either use the ANSI joining sytax (LEFT OUTER JOIN) or use the (+) in the where clause.

    It scales to large databases only when they are designed properly. I work with 18 terabytes of data. My sql code wouldn't work so hot if the tables weren't designed correctly. Indexing, partitioning, and table structure have more to do with performance at that level than the code. The code can make a large difference too, but if the underlying structure is wrong, even the best SQL won't help you.
    • SQL by itself doesn't perform.

      That's kind of disingenuous, though, like saying that "C by itself doesn't perform". That's true, in as much as both are standards and not implementations. However, it's a bit misleading because C, at least, has features that make it inherently difficult to optimize (as compared to other languages like Fortran).

      IANA DB theorist, but I could imagine that SQL might also contain some "traps" that might necessarily make its implementations slower than what another query lang

    • Re:SQL isn't a database (Score:4, Informative)

      by mcrbids ( 148650 ) on Tuesday May 24, 2005 @04:22PM (#12627079) Journal
      SQL by itself doesn't perform. It is based on the database engine, and how good the developer is.

      A truth I hold to be self-evident. The language of SQL provides all the tools you need to make your application perform well, as you state.

      SQL doesn't really have an API.

      Realistically, SQL is an API. It's a highly abstracted interface for communicating between two programs. (your app, and the DB server software)

      It is only "cross-platform" if you follow the ANSI SQL standard.

      Sorta. See, I can write a script using PHP with a particular SQL call, and do the same thing in Perl, Java, ASP, C, C++, Python, Ruby, and even BASH, on Linux, Windows, Mac, or just about anything else with a tcp stack and a compiler. Sure, SQL implementations are different, with various shortcuts and extensions, but I'd call that cross platform if ever there was one.

      Let me ask you this: How often do you see an OSS product (EG: phpwiki) that doesn't offer support for numerous databases?
      • JDBC (probably ODBC too, tho haven't used it in eight years) helps to standardize key generation (in JDBC 3.0 FINALLY), and Date processing (christ, date functions are so annoying). Most other operations can usually be done by the platform language that is processing the data, so you can avoid the tie-in that results from various SQL dialects' built-in functions. XML databases were a total flop, and so were object databases. I agree, SQL engines are so mature now that I don't see any database tech replaci
      • Let me ask you this: How often do you see an OSS product (EG: phpwiki) that doesn't offer support for numerous databases?

        Quite a bit actually. In my experience most only support MySQL. Why? They don't understand the value of data integrity and the features MySQL lacks/lacked. As a result they have had to poorly implement things that the DB itself should do, and when they try to "port" this to another DB they run into all sorts of issues.

        And SQL is a language, not an API.
        So they give up.

  • Synchronization (Score:4, Informative)

    by 3770 ( 560838 ) on Tuesday May 24, 2005 @03:30PM (#12626479) Homepage
    Most mainstream databases support replication. They are designed to be as fast as possible under heavy load.

    Synchronization for a mobile device has another main requirement, robustness when the connection to the server is lost. A mobile device has to gracefully handle when the owner runs down into the subway.

  • It's already done (Score:2, Interesting)

    by duffer_01 ( 184844 )
    "one size no longer fits all"

    This is absolutely correct when referring to databases for different application. However, why do people always assume that they have to choose the Oracle's, MS and IBM's out there? There are already databases that have been tailored for certain application environments. Take for example http://www.ianywhere.com/ [ianywhere.com] who has databases like SQL Anywhere and UltraLite which are tailored for smaller workgroups and mobile devices.
    I don't think the solution to the problem is to buil

  • Why 'Beyond'? (Score:5, Insightful)

    by Anonymous Coward on Tuesday May 24, 2005 @03:34PM (#12626510)
    Designed in the 1970s, the RDBMS has nevertheless proven to be the cornerstone of Web development three decades later. Thanks to systems like MySQL deployments are surely at record levels.

    Essayist Clay Shirky has gone to far as to suggest that MySQL is at the center of a whole new software movement [shirky.com].

    In my experience with Web applicaions the chief problem with the RDBMS seems to be that it does not do text indexing and search very well, so I have to keep a second store of data in something like Lucene.

    The other major problem is the level of skill required to tune the database to achieve high-performance SQL queries, so hopefully the RDBMS will evolve with more self-configuration capability.

    The article, which I only skimmed, actually addresses these two concerns but seems to pooh-pooh the notion of simply refining the existing RDBMS systems. Instead it says " Old-style database systems solve old-style problems; we need new-style databases to solve new-style problems. "

    The paper seems awfully squishy on what this means. The clearest I found was a call to "produce a storage engine that is more configurable so that it can be tuned to the requirements of individual applications."

    But this call for new highly modular/configurable storage "engines" seems to me to require at least as much fussy care and feeding as a traditional RDBMS. You're just replacing one DBA with another. And throwing out decades of refinement in the process.

    The raison d'etre of the RDBMS is to allow the programmer to treat storage as a black box while gaining nifty ACID features. Extending this to text indexing seems logical.
  • I agree with the sentiments of the posters that SQL is not going anywhere, but I had a question.
    As I am designing more and more complex web apps, I am constantly having to think of new, innovative ways to design the tables and databases and am currently making it up as I go. Does anyone have a reccomendation for books/sites that talk about good design proactices, that is not "How to use SQL" and relatively agnostic on the specific brand on DB?
    Sorry for the OT post, its just something that has been buggi
    • Depends what level you're after really, but here are some pointers that might help:

      I know it says access on the cover, but Steve Roman's book Access Database Design & Programming [amazon.co.uk] is pretty good as a starting point, IMO.

      You could also try Michael Hernandez's Database design for mere mortals [amazon.co.uk]

      I found both to be pretty readable and a good introduction to the theory.

      Once you've got the basics down, there's lots of further stuff you can try, from the obvious books by Codd and Date, to the more esoteri

  • According to the ACM...

    No... Acording to Sleepycat [sleepycat.com], who have a great name and logo, but an otherwise very annoying data store.

  • the usual database blatherings (Score:2, Insightful)

    by Anonymous Coward

    by MARGO SELTZER, SLEEPYCAT

    Sleepycat? The guys who make a brain-dead key/value database with no data manipulation or integrity capabilities? Who are they to educate others on the topic of relational databases? (Sleepycat's products are useful tools, but they are not true databases).

    while data management has become almost synonymous with RDBMS, however, there are an increasing number of applications for which lighter-weight alternatives are more appropriate.

    Ahh, so the proper title of this paper sh

  • A thought on XML documents (Score:3, Interesting)

    by It doesn't come easy ( 695416 ) * on Tuesday May 24, 2005 @03:54PM (#12626721) Journal
    I woke up and had an interesting thought...

    I can imagine XML documents created in such a manner that they could constitute an object from an OOP (Object Oriented Programming) perspective, containing their own schema, characteristics, relationships and data. Further, I can imagine the ability of accepting such an XML document object into a range of other things, such as a modular program (dynamic program extensibility) or a database (temporary database extension). I can imagine the ability to package up such XML documents so that databases can be built simply by linking the XML documents together.

    So (for example), one could send a request to the XML index in the sky, find and link documents containing a subset of know medical facts, and then kick off a data mining process that could discover previously unknown medical relationships. All without needing to know anything other than where to find the XML files.

    Now imagine a tool that could convert all of the terabytes of data the world is generating every day into small, linkable, OOP like XML files... Sounds like a great open source project to me...

    • Re:A thought on XML documents (Score:5, Insightful)

      by The Slashdolt ( 518657 ) on Tuesday May 24, 2005 @04:29PM (#12627140) Homepage
      Here is the problem with your idea. Unlike the relational model, XML does not link facts. XML documents can be joined in any way, either valid or invalid, without you knowing one way or another. The relationships between documents are weak. There is no referential integrity. Within a proper relational model you are stating facts and factual relationships. Joins of those facts generate derived facts that are as true and accurate as your original model. Why add the overhead and complexity of xml? Why not just use a proper relational model?

  • Improving the efficiency???? (Score:5, Informative)

    by Dammital ( 220641 ) on Tuesday May 24, 2005 @03:56PM (#12626746)
    "Relational databases were developed in the 1970s as a way of improving the efficiency of complex systems.
    Huh? Go back and reread some of Codd's papers (in the late 60's, BTW) and you'll see that efficiency was never a motivator. Simplicity was his aim, filesystem details were made irrrelevant, explicit navigation was obsoleted, and a built-in security model was included.

    When relational systems finally began to appear (and I'm thinking specifically about IBM's System R) they were dog slow, and the extant hierarchical and CODASYL network databases of the day ran rings around them. Still do, unless you throw lots of hardware at the RDBMS.

    RDBMS have lots of advantages over older technologies, but performance is not among them.

  • Bah... (Score:2, Funny)

    by Danathar ( 267989 )
    ASCII Flat files are the way to go. If I can't fgrep it...then to hell with it. SQL my ass..
  • Many of the capabilities that, according to the article, should exist in the next generation of databases exist today in IBM Informix Dynamic Server. Examples: The hability to use other than B-Tree+ indexing technology, the hability do describe data beyond traditional types, etc. Furthermore the article relies heavily on comments by Stonebreaker, the father of Informix's Object Relational capabilities.

  • MUMPS (Score:4, Interesting)

    by stuffduff ( 681819 ) on Tuesday May 24, 2005 @04:15PM (#12626985) Journal
    MUMPS had one data type, a string and one data structure, an array. Commands abbreviable to a single letter. MUMPS ran whole Hospital Information Systems on DEC PDP-11's where MUMPS was both the Operating System and the Language and was called Digital Standard MUMPS or DSM for short. It was way ahead of it's time, it predated SQL, and was one of it's progenitors. Anyone who offers a survey of databases and fails to include it is seriously myopic. Google 'MUMPS Language' for the tip of the iceberg.

    As for the future, I think that while OO databases are all the vogue, that O databases, where each bit of information is represented by it's own object and will have the ability to demonstrate autonomous agency is a good area for research. Then we can just let the data speak for itself! ;^)

    "Every person takes the limits of their own field of vision for the limits of the world."
    - Arthur Schopenhauer

    • No data integrity (Score:3, Interesting)

      by Tony ( 765 )
      The problem with MUMPS is the data integrity issue. As essentially a collective of hashed indexes where nodes can contain both data and more branches, it sucks for *any* kind of serious data retrieval. You basically have to create and maintain your own indexes, unless you go with Fileman, which is a massively moby hack, but a hack nonetheless.

      The idea of the relational model is simple: it is based on set theory, which has a strong mathematical model. There is no equivelent model for object databases, no

  • Sleepycat? (Score:3, Interesting)

    by halosfan ( 691623 ) on Tuesday May 24, 2005 @04:19PM (#12627032) Homepage

    To me, this entire article reads like a plug for Sleepycat [sleepycat.com], written, not surprisingly, by a Sleepycat person.

    Relational data model is just that -- a data model. It doesn't concern itself much with implementation (and therefore, performance in any particular environment) or with how applications use the data. And that's really the point -- relational databases are application-agnostic. They are designed to store the data that will be possibly accessed by applications that are not yet conceived. That's the reason they put great emphasis on internal correctness of the data. Once you have a database specialized for one application, that's not really a database in the relational sense -- that's a way to persist your application data. And that's where Berkeley DB [sleepycat.com] shines. It doesn't replace relational databases, it just serves a totally different purpose.

  • How about.... (Score:5, Informative)

    by plopez ( 54068 ) on Tuesday May 24, 2005 @04:21PM (#12627055) Journal
    really implementing a relational model to begin with? Then we can decide if the relational model is broken or just the vendor implementation.

    How about... a query language that is fully set operations compliant, i.e., something other than ANSI SQL which is a strange mixture of set and bag operations, and a mixture of relational algebra and relational calculas and some other 'extensions'.

    How about... realizing that a major design goal for the relational model was data integrity. Modularity and configurability are also good goals but if you are serious about your data, integrity will be at the top of the list.

    The biggest problems I see with databases is very few people understand how to use them. Here's a few tips:
    1) a table is *not* a class or an object. Tables + constraints + user defined types + constraints etc. when used properly can define domains which are close to classes and objects.

    2) Learn how to normalize. A badly (or flat out not) normalized database threatens data integrity by violating the once-and-only once rule. As a rule of thumb if the table has more than 20 fields in it you should review your data model and make sure it is properly normalized.

    3) Point 2 is often the consequence of mindlessly slurping in spread sheets or MS Access database tables. Anyone doing this has no business being within 50 feet of an IDE.

    4) Ditch Raid 5. 0+1 will give better perfomance in most cases. Manager like Raid 5 because it is cheap, you get what you pay for.

    5) Have multiple channels for data, transaction logs, large indices and O/S or user applications to reduce bottle necks. This is expensive but for large databases going cheap will hurt you.

    6) Learn a little theory, it won't hurt you. In fact it can save a large amount of time and trouble. Do not be afraid of learning about the technology you are using. After all, technology is what you are good at, right?

    7) If it is a read only database, turn off logging for speed (impossible to due under SQL Server 2000 btw). Also, if a table is on a purge and load paradigm (many reporting and/or datawarehouse tables are) turn off logging on the table level if your version of database engine allows you to do so. Likewise, turning off logging on a hand held or other single user system may be appropriate, just make sure two people do not try to use the database at the same time.

    8) Avoid XML. Too much bloat.

    9) Learn how to use indices on tables.

    10) Learn how to read a perfomance monitor/top etc.

    Postgresql is both working hard to become truly relational AND is adding support for geographic objects and objects. The MySQL crew is working hard to improve. Oracle has some nice perfomance features but I think their 'Object/Relational' implementation is broken. SQL Server is getting 'long in the tooth'. There is also a great need for temporal databases and lightwieght engines. But remember, there is no 'silver bullet', no short cuts. Just hard work to be done.

    • Re:How about.... (Score:2, Insightful)

      by Anonymous Coward

      1) a table is *not* a class or an object.

      Good advice. A table is simply a snapshot of a relational value. True relational values have no top/bottom or left/right ordering so you can't really show them on paper or on the screen. It doesn't make sense to map classes or objects to tables.

      2) Learn how to normalize. A badly (or flat out not) normalized database threatens data integrity by violating the once-and-only once rule.

      Normalization has nothing to do with integrity per se. You can add constraint

  • Hmm .. (Score:5, Informative)

    by ghakko ( 261165 ) on Tuesday May 24, 2005 @04:38PM (#12627233)
    Has anyone noticed that the author of the article is from Sleepycat (which sells commercial licenses for Berkeley DB to embedded systems developers)?

    She puts forth a case against SQL and relational databases in general and claims that many applications (like directory services and search engines) have read-heavy, hierarchial access patterns which favour lighter-weight, non-relational, transaction-optional databases.

    And .. it just so happens that Sleepycat's flagship products are Berkeley DB (a flat-file database) and DBXML (an XQuery engine built on top of that).

  • Issue of dimensions in RDBMS (Score:3, Interesting)

    by jonbrewer ( 11894 ) * on Tuesday May 24, 2005 @04:42PM (#12627276) Homepage
    One of the problems of current databases when is that a typical relational database doesn't have enough dimensions. Designing a table to store data is trivial - but what happens when you need to know the intersection of X and Y at time Z?

    This is a fairly common question in data warehousing: What is the data today, what did it look like yesterday, last week, and last year?

    I have seen it worked around in silly ways (snapshot and rename a table every day/week/month) and more clever ways (use separate transaction tables to record changes), but never in a particularly elegant way.

    Wiser colleagues whispered to me the dirty answer "object relational" and scurried away to their dens of Rob Zombie and J2EE. I never got my head around object relational databases before leaving that world, and so am left to ponder papers [ca.com] from IDC with statements like this one:

    "putting object extensions on RDBMSs is tantamount to adding stereo radios and global navigation systems to horse-drawn carriages"

    Ouch, is that a swipe at Oracle? Seems that as far back as 1997 pundits have said that the future is in ODBMS, and not RDBMS or ORDBMS. Hmm...

  • JSR 170 - Unified Data Repository (Score:4, Informative)

    by uss_valiant ( 760602 ) on Tuesday May 24, 2005 @04:59PM (#12627467) Homepage
    Another approach to the problem: JSR 170: Content Repository for JavaTM technology API [jcp.org]
    Standardizing the interfaces to various data resources (filesystem, database, cache, ...).

    The expert group reads like a who's who in data management. And it seems to be very near to the final draft.

  • Another one? (Score:3, Funny)

    by ColdWetDog ( 752185 ) on Tuesday May 24, 2005 @05:03PM (#12627509) Homepage
    I've already got Windows and a girlfriend, I really don't need another irrational database.
  • If any smart DBMS developers are listening, is to define a set of queries within the database (like for a _simple_ example "male" and "over 60" and "salary x") and then be able to refer to these criteria by name only, having the database build the query based on these rules as I choose to combine them (select xxx from yyy where 'male' or where 'male' and 'over 60').

    Sort of like stored procedures in implementation - they could be called stored query definitions.

    Because these query definitions would alread

Slashdot Top Deals

This file will self-destruct in five minutes.

Close