Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!


Forgot your password?
Databases Programming Software IT

F/OSS Flat-File Database? 702

Leemeng writes "I'm looking for a simple, free, and F/OSS flat-file database program. I'm storing info about Wi-Fi access points that I come across, maybe 8-9 fields per entry. I've outgrown Notepad. This info is for my own reference only; it is not going on a Web server. Googling was unhelpful, with results skewed towards SQL, Access (MS), and Oracle, all of which would be overkill for my purposes. My criteria are: it must be simple, F/OSS, must work in Windows Vista, preferably use a portable format, must not be an online app, and must not require Java. Does such a beast exist?"
This discussion has been archived. No new comments can be posted.

F/OSS Flat-File Database?

Comments Filter:
  • Python? (Score:5, Informative)

    by fyngyrz ( 762201 ) * on Tuesday May 20, 2008 @06:59PM (#23484234) Homepage Journal

    Can't be Java... well, how about Python?

    Here is [ideaspike.com] a completely free (PD, not GPL-style "you're free to do as we tell you") database engine that will do what you have described thus far.

    The database engine is about 19k bytes (not a typo), has no dependencies (other than Python itself), supports a useful subset of SQL so you can actually create flexible queries that produce well-sorted results from your database, and it works everywhere Python does, which is to say, it works pretty much everywhere. It's just as happy operating on a command line as it is on a web server. The results (the actual databases) are 100% portable from OS to OS. I use it on various linuxes, OS X, and Windows for tasks very similar to yours.

    Comes with tutorial examples, sample databases and extensive docs. In a 13k (not a typo) archive.


    • Re:Python? (Score:5, Insightful)

      by maxume ( 22995 ) on Tuesday May 20, 2008 @07:24PM (#23484648)
      Why not sqlite? Comes with Python 2.5 on Windows. It adds a relatively weighty 800k or so to my python installation on Windows, but the installation is dozens of megabytes overall, so it doesn't really matter.
      • Re:Python? (Score:5, Informative)

        by fyngyrz ( 762201 ) * on Tuesday May 20, 2008 @07:39PM (#23484894) Homepage Journal

        Why not sqlite?

        • SQLite isn't present or compiled in, in all Python installations, 2.5 or otherwise
        • At 800k, it's about 50x the size of class dbtxt (executable)
        • Source is huge compared to class dbtxt, so maintainance is not easy
        • SQLite is considerably more difficult to use (it's also more capable, though)

        That's all I have for ya, offhand. ;-)

        • Re:Python? (Score:5, Informative)

          by dixonpete ( 1267776 ) on Tuesday May 20, 2008 @08:25PM (#23485524)
          There's an add-on for Firefox that makes SQlite use fairly painless: SQLite Manager. Brilliant work.
        • Re:Python? (Score:4, Informative)

          by jhol13 ( 1087781 ) on Tuesday May 20, 2008 @11:38PM (#23487418)
          dbtxt can only handle ASCII (<128 so even ISO-8859-1, etc. are no-no). This can be killer (it is for me).

          Other than that the dbtxt looks very nice indeed.
        • Re:Python? (Score:4, Insightful)

          by XNormal ( 8617 ) on Wednesday May 21, 2008 @03:03AM (#23489002) Homepage
          How is this moderated as "informative" when it contains blatantly incorrect information? SQLite is most definitely present in all Python 2.5 installations. It's part of the Python source tree. It's as much a part of the Python distribution as the regular expressions library you use in your code. And if you insist on using an older version of Python SQLite is just one apt-get away.

          In what specific ways does the supposed "bloat" of SQLite affect the user, exactly?

          What kind of "maintenance" does the user need to do on the SQLite source code? It's maintained well enough for some some pretty big users [sqlite.org]. Who maintains your code and why should I trust you with my data?

          How can SQLite be "difficult to use" when it's just standard SQL with a standard Python DBAPI interface?

          It's really nice that you wrote a cute little datastore. But there's no need to badmouth what you perceive as the competition.
    • by spookymonster ( 238226 ) on Tuesday May 20, 2008 @07:25PM (#23484662)
      Just get Python, and use the version of SQLite that comes with it:

      import sqlite3
      mydb = sqlite3.connect('sample.db')
      mydb.execute("create table contacts (fname text, lname text, email text)")

      mydb.execute("insert into contacts values('Spooky','Monster','spook@spammity.spam')")


      You can then use the free and open SQLite database browser [sourceforge.net] to browse, edit, and print your table.

      You may think you're keeping it simple by using a flat file, but you're really not. It may be somewhat easier to manually edit, but it's also easier to screw up, and I've never heard of one with the ability to undo changes.
      • by fyngyrz ( 762201 ) * on Tuesday May 20, 2008 @07:50PM (#23485032) Homepage Journal

        You may think you're keeping it simple by using a flat file, but you're really not. It may be somewhat easier to manually edit, but it's also easier to screw up, and I've never heard of one with the ability to undo changes.

        Class dbtxt includes the ability to undo changes, and features human-readbility for the flat database files. It's trivial and not error-prone at all to mod a database file to delete, or undelete a record, and of course you can do it through the database engine as well.

      • by dkf ( 304284 ) <donal.k.fellows@manchester.ac.uk> on Tuesday May 20, 2008 @08:09PM (#23485324) Homepage

        mydb.execute("insert into contacts values('Spooky','Monster','spook@spammity.spam')")
        Couldn't you have at least done an example that doesn't promote bad practices with quoting? My experience with sqlite (admittedly using the Tcl binding) is that it gets this stuff far more correct and easy to use in practice than any other database API that I've seen. I don't know if the python bindings are of that same quality; if they are, they're top notch. The code I would have written? Like this:

        set first "Spooky"
        set last "Monster"
        set email "spook@spammity.spam"
        # In practice it's really easy to put values into variables
        mydb eval {insert into contacts values (:first,:last,:email)}

        The advantage? That code is now totally armour-plated against SQL injection attacks as well as being fast. Which is nice, really really nice.
        • Re: (Score:3, Informative)

          by Just Some Guy ( 3352 )

          I don't know if the python bindings are of that same quality; if they are, they're top notch.

          The sqlite3 module as shipped with Leopard says that it supports question mark substitution, so his example would be better written like:

          mydb.execute("insert into contacts values(?, ?, ?)", ('Spooky', 'Monster', 'spook@spammity.spam'))

          BTW, the same could be written for PostgreSQL like:

          mydb.execute("insert into contacts values(%(first)s, %(last)s, %(email)s)", {'first': 'Spooky', 'last': 'Monster', 'email': 'spook@spammity.spam'})

          Hopefully sqlite3 will get the same standard substitution in the fu

          • Re: (Score:3, Informative)

            by ubernostrum ( 219442 )

            Actually, the Python SQLite adapter supports the standard Python DB API (PEP 249 [python.org]), and so you can just follow its recommendations (which includes using placeholders).

        • by spookymonster ( 238226 ) on Tuesday May 20, 2008 @10:43PM (#23486886)
          1) My example was done to show how quick and easy it would be to set up and populate a table in Python.

          2) He's explicitly mentioned several times that this is for his own private non-web use.

          3) Your fly is open.
      • by tknd ( 979052 ) on Tuesday May 20, 2008 @08:25PM (#23485528)

        He doesn't need python. He just needs a database. He can download a precompiled binary for windows [sqlite.org] that allows one to work with the database at the command line [sqlite.org]. Python is not necessary.

        And if the command line is too much, as others have noted there is already a convenient firefox extension [mozilla.org] for graphically interacting with a sqlite database.

    • Re:Python? (Score:5, Interesting)

      by goombah99 ( 560566 ) on Tuesday May 20, 2008 @07:26PM (#23484682)
      You need YAML [wikipedia.org]

      I think is exactly what you are requesting.
      • Re:Python? (Score:5, Funny)

        by el americano ( 799629 ) on Tuesday May 20, 2008 @07:35PM (#23484816) Homepage
        YAML is a recursive acronym for ...


      • Re:Python? (Score:5, Interesting)

        by goombah99 ( 560566 ) on Tuesday May 20, 2008 @07:42PM (#23484924)
        YAML meets the flat file requirement and can be made to be the persistent DB behind Python, Perl, Ruby.

        If it were not for the Flat File requirement then a simple Python shelve or Perl Tie would be the most logical solution since they are both part of the standard library so don't require installing libs on random computers you might use or port to.

        of those two Perl Tie is probably the most suited because it's backed by a real DB operating off the disk not fully in memory.

        But why not do both: use YAML as the DB backing the Perl Tie.
          then you get a nice human readable flat file.

        • Re:Python? (Score:4, Interesting)

          by sticks_us ( 150624 ) on Tuesday May 20, 2008 @09:01PM (#23485894) Homepage
          I'm late to the party, and there are certainly a ton of other good suggestions, but I just had to pipe in and cast another vote for YAML [yaml.org].

          Even if you don't end up using it for this particular app, it definitely deserves a look. Although it'll never displace XML, it definitely answers at least some of the questions XML attempts to answer, reduces "tag bloat," and is easy on the eyes.
    • Been searching flat files with this open source VB5 program since 1999. Searches flat files at 20,000,000 cps displaying hi-lited hits in matching lines only or full context mode. Results can be exported to a results file. Append txt files together. Search and replace option. Run multiple versions with various default settings. I have been yapping and yapping about flat files, only to get great amounts of resistance from the Geeks. The search uses lots of defaults and quick keys (for typists). From startup
  • No Java? (Score:5, Insightful)

    by jawtheshark ( 198669 ) * <slashdot@ja w t h e s h a r k . com> on Tuesday May 20, 2008 @07:01PM (#23484254) Homepage Journal

    I would have recommended HSQL, but you don't want Java. Frankly, usually, when we're talking databases I won't say "use a spreadsheet", but with 10 fields, you might as well use a spreadsheet. Of course OpenOffice.org Base is out, because it uses HSQL.

    Something like CSQL [sourceforge.net] might fit, but I have no experience with it.

    • Re: (Score:3, Insightful)

      In fact, the requirements are pretty extreme.

      Must not be "online" -- does that rule out web apps? Even if they'll run on your local machine?

      And why no Java? Can't be for efficiency, otherwise you wouldn't be running Vista. Maybe something's broken with your Java?

      Also, what are the required features? You may have outgrown Notepad, but have you outgrown vim, grep, and friends? That's probably why there's not much F/OSS for this kind of thing. Just a guess.
    • Re:No Java? (Score:5, Insightful)

      by geekboy642 ( 799087 ) on Tuesday May 20, 2008 @10:10PM (#23486560) Journal
      Mod parent up.
      This guy wants a spreadsheet, he just doesn't know it. Excel--or the free alternative from OpenOffice.org--will do everything he could possibly want, and although it saves as its own infernal file format, it exports competently into a tab or comma-delimited format.

      All the extra "requirements" are just pseudo-intellectual mumbo-jumbo that have no bearing on reality.
  • Err ... (Score:4, Informative)

    by Anonymous Coward on Tuesday May 20, 2008 @07:01PM (#23484264)
    Comma Separated Variable Text Files, as exported and imported by Excel. You can get libraries to read and write these, and search these in most languages.

    Otherwise what's wrong with a simple database like MySQL or PostgreSQL on your computer?
    • Re: (Score:3, Informative)

      by djmurdoch ( 306849 )

      Comma Separated Variable Text Files, as exported and imported by Excel.
      Don't use CSV with Excel, it will by default change values on import.

      For example, if you ask it to read and write this:

      May 20,5/20,1.000

      you get this:


      That might be okay if those were really two dates and a number, but I was never asked if they were. (OOO Calc is just as bad: it asks, and by default modifies the data).
  • OOO? (Score:5, Informative)

    by iamhigh ( 1252742 ) * on Tuesday May 20, 2008 @07:02PM (#23484272)

    it must be simple, F/OSS, must work in Windows Vista, preferably use a portable format, must not be an online app, and must not require Java. Does such a beast exist
    Maybe this will do? [openoffice.org] I think it meets all your needs. You can even use it with a web app if desired. Some functionality may need Java, but most doesn't. I don't know what parts of OOO are Java-driven, but I am sure somebody here does!
    • Re:OOO? (Score:5, Insightful)

      by ADRA ( 37398 ) on Tuesday May 20, 2008 @07:27PM (#23484694)
      OpenOffice uses Java for its database tool. I'm fine with it, and its probably the best choice for anyone actually wanting something simple / usefull / free, but obviously the OP is a trolling wanker, so it has to be san-Java.
  • sqlite (Score:5, Informative)

    by nguy ( 1207026 ) on Tuesday May 20, 2008 @07:02PM (#23484276)
    Sqlite is used in many apps (including Firefox), it's small, and it's efficient. It also has bindings to just about every imaginable language.

    I find it amazing that you didn't come across it in Googling...
  • SQLite (Score:5, Informative)

    by kcbanner ( 929309 ) * on Tuesday May 20, 2008 @07:03PM (#23484280) Homepage Journal
    Get it http://www.sqlite.org/ [sqlite.org] here.
    There are GUI clients that work fine for this sort of thing, SQL is simple for doing basic things. One file, one database.
  • how about... (Score:4, Insightful)

    by ramirez ( 51663 ) on Tuesday May 20, 2008 @07:07PM (#23484366)
    a comma delimited file?

    Umm... just write a few one-liner perl scripts to get info out.
  • heh (Score:5, Funny)

    by B3ryllium ( 571199 ) on Tuesday May 20, 2008 @07:08PM (#23484390) Homepage
    Since the majority of the comments so far have pointed at SQLite, I'm kind of surprised that the post didn't come "from the sqlite dept." :)
  • Perl DBFile (Score:3, Interesting)

    by Anonymous Coward on Tuesday May 20, 2008 @07:09PM (#23484398)
    Use perl with a tied hash like BerkeleyDB http://search.cpan.org/~pmqs/BerkeleyDB-0.34/ [cpan.org]

    Easy, fast, simple.
  • by bluefoxlucid ( 723572 ) on Tuesday May 20, 2008 @07:09PM (#23484400) Homepage Journal
    Trying to solve such an issue as a flat file is a poor design characteristic. This says "I don't want to learn SQL" or "I want the output human readable." If you just want to store information for programmatic use, use SQLite and quit worrying about data storage format. If you want it human readable, grab libxml2 (works on Windows too) and store it as XML. Decide which problem you are solving because if you don't need (i.e. it's not helpful) it human-readable and/or want the ability to search it quickly without loading the whole thing into memory, then SQLite is probably a better solution.
  • Spreadsheet (Score:4, Informative)

    by benwb ( 96829 ) on Tuesday May 20, 2008 @07:11PM (#23484418)
    Open office should do the trick.
  • SQLIte or BDB (Score:4, Informative)

    by willyhill ( 965620 ) <pr8wak@noSPAM.gmail.com> on Tuesday May 20, 2008 @07:11PM (#23484424) Homepage Journal
    I'd recommend SQLite or Berkley DB. I've used BDB on a couple of projects where I needed to basically store an enormous hashtable that could be read quickly by key, and I don't think anything else comes close to the speed of that thing.

    BDB is *not* a relational database though, it's just a storage/indexing engine, which is used most notably by MySQL as a backend. SQLite on the other hand is a full file-based RDBMS with a small runtime, so it might be a bit of overkill for you in this particular scenario.

    But both of them run on Windows, Linux and the BSDs, so you won't have portability problems. And most languages have bindings for them.

  • Uhh (Score:3, Interesting)

    by Auckerman ( 223266 ) on Tuesday May 20, 2008 @07:11PM (#23484428)
    The only thing that comes to mind is Mac only. Bento [filemaker.com] (which from the write up is exactly what you want). I really think the only thing on Windows that really comes close is Microsoft Access and even that isn't what you're looking for.

    If you are just doing tabulated data in a piece meal form, Excel with an Access back-end will do the trick, I'd be willing to be you're going to be able to find templates that help you start. You could also use File-maker (which is overkill, but easy to use) and go to the user community for starting templates.
  • A CSV File? (Score:3, Interesting)

    by shades66 ( 571498 ) on Tuesday May 20, 2008 @07:12PM (#23484436)
    Can be used by all spreadsheet programs (Excel,Gnumeric,OOCalc probably even Google's online offerings) for complete portability

    Depends what you want to be able to do with your data? If it's just quick searches to find local wifi points then it seems overkill to use a multi-platform/FOSS database

  • GDBM (Score:5, Informative)

    by jschmerge ( 228731 ) on Tuesday May 20, 2008 @07:13PM (#23484450)
    If you're doing simple Key,Record storing, try GDBM (or one of its analogs: DBM, NDBM). IIRC, it's included as part of glibc. The interface for it is analogous to that of a hash table... In fact there's even native Perl support for tying hash tables to GDBM.

    If that doesn't satisfy your need, take a look at Berkley DB. It offers a more sophisticated interface than DBM.

  • What about XML? (Score:4, Interesting)

    by SpinyNorman ( 33776 ) on Tuesday May 20, 2008 @07:16PM (#23484496)
    A database seems overkill.

    Read the XML records into an STL container for easy access.
  • by diamondsw ( 685967 ) on Tuesday May 20, 2008 @07:17PM (#23484520)
    For a small number of fields like that, why re-invent the wheel? Grab OpenOffice and just create a spreadsheet. Easily searchable, sortable, extendable, and all with zero maintenance on your part.

    I recently moved my collection of serial numbers out of a defunct proprietary program and into a spreadsheet - couldn't be happier.
  • Try Metakit (Score:4, Informative)

    by kawabago ( 551139 ) on Tuesday May 20, 2008 @07:26PM (#23484666)
    Metakit is a small footprint database that might fit your needs. Metakit [equi4.com]
    • Re: (Score:3, Informative)

      by slackergod ( 37906 )
      Much as I like sqllite, I agree with you completely... metakit is definitely what he wants.

      For those not familiar with metakit,
      it straddles a very interesting (to me at least)
      line between "some csv files i threw together" and "full on sql db" in terms of it's storage
      and query semantics, while at the same time being
      a flat file like sqllite.

      And it really deserves to be better known from a
      theoretical standpoint, in how it structures the
      data... especially some of the more interesting
      applications it's been put t
  • by holophrastic ( 221104 ) on Tuesday May 20, 2008 @07:29PM (#23484720)
    If it's just for your reference, you don't need a database at all. Databases don't become technically worth-while until you get to indexing. And until you have fifty-thousand records, or complex queries, you don't need indexing.

    So why not simply write out a tab-delimited file? Retrieval is as simple as reading every line, splitting by the delimeter, and regexp'ing whichever field you're examining. 15'000 records takes about two seconds on a modern machine. Of course, for anything more complicated, that same file gets easily imported into your favourite spreadsheet application.

    You don't need to worry about locking because you're the only one using it. And otherwise, your application simply locks the file handle, or creates & destroys a traditional lock-file.

    It's a thirty-line perl script (of legible perl). You can do it in JScript as a local HTA if you want the benefit of html etc. interfaces.
  • by Doofus ( 43075 ) on Tuesday May 20, 2008 @07:37PM (#23484850)
    There are a number of XML databases, several free and open source, that will rely only on "flat-files". You could probably get by with Microsoft's xml libraries, though there are a number of ways to manipulate and query a set of xml documents. Several of these XML databases implement XQuery which may help if your dataset grows beyond effective queries by visual inspection in Notepad.

    eXist [sourceforge.net] is one alternative; while I haven't personally used it the home page indicates it's a fairly capable project.

    Sedna [ispras.ru] also appears to be feature-rich.

    There was a similar discussion on Slashdot specifically with reference to XML databases, here [slashdot.org].

    Happy hunting -
  • by myspace-cn ( 1094627 ) on Tuesday May 20, 2008 @07:52PM (#23485066)
    Why mess about with filty batchfiles, bashscripts and clunky sqlite, mysql. How about a full Oracle Database 11g Enterprise? o;)
  • PCFile (Score:3, Informative)

    by Linker3000 ( 626634 ) on Tuesday May 20, 2008 @07:59PM (#23485180) Journal
    1993 called to recommend PCFILE!

    http://www.umich.edu/~archive/msdos/database/pcfile/ [umich.edu]

    Mind you, you'll have to toss Jim Button $10 as it's shareware!
  • by wernst ( 536414 ) on Tuesday May 20, 2008 @08:03PM (#23485240) Homepage
    I'm guessing he wants a FOSS version of FileMaker Pro; something that is an application with a GUI. All you guys are suggesting various frameworks he can use to program his own. No, I don't have an answer either, other than to suggest the spreadsheet in OpenOffice.
  • by SixDimensionalArray ( 604334 ) on Tuesday May 20, 2008 @08:13PM (#23485378)
    Nobody has yet mentioned that MySQL has a CSV storage engine [mysql.com] - just create a table with type CSV and away you go. It does require the MySQL engine though.

    Otherwise, I agree with most posters who say just use a simple text and/or xml file if the data volume is relatively small. That should be more than enough!

  • Kexi (Score:3, Interesting)

    by erikdalen ( 99500 ) <erik.dalen@mensa.se> on Tuesday May 20, 2008 @08:19PM (#23485460) Homepage
    Kexi should be able to hande that pretty easily, and can even create a nice form for the data entry:
    http://www.kexi-project.org/ [kexi-project.org]
  • by ChrisA90278 ( 905188 ) on Tuesday May 20, 2008 @08:30PM (#23485588)
    Your problem is that you are looking for a "database". This is a very complex kind of program that is total overkill for your purpose. Will you really be storing hundreds of thousands of records and wanting to do some complex querries? Ifnot why not just use a speadsheet? If you want a free spreadsheet look at OpenOffice.
  • use a spreadsheet (Score:3, Insightful)

    by crevistontj ( 1032976 ) on Tuesday May 20, 2008 @08:39PM (#23485682)
    Why one earth would you use a database to store one table on information? Save youself some trouble and use a spreadsheet.
  • Berkeley? (Score:3, Informative)

    by mindstrm ( 20013 ) on Tuesday May 20, 2008 @08:59PM (#23485882)
    Berkely DB?

An elephant is a mouse with an operating system.