Slashdot Log In
F/OSS Flat-File Database?
Posted by
kdawson
on Tue May 20, 2008 05:59 PM
from the no-relation dept.
from the no-relation dept.
Leemeng writes "I'm looking for a simple, free, and F/OSS flat-file database program. I'm storing info about Wi-Fi access points that I come across, maybe 8-9 fields per entry. I've outgrown Notepad. This info is for my own reference only; it is not going on a Web server. Googling was unhelpful, with results skewed towards SQL, Access (MS), and Oracle, all of which would be overkill for my purposes. My criteria are: it must be simple, F/OSS, must work in Windows Vista, preferably use a portable format, must not be an online app, and must not require Java. Does such a beast exist?"
Related Stories
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Python? (Score:5, Informative)
Can't be Java... well, how about Python?
Here is [ideaspike.com] a completely free (PD, not GPL-style "you're free to do as we tell you") database engine that will do what you have described thus far.
The database engine is about 19k bytes (not a typo), has no dependencies (other than Python itself), supports a useful subset of SQL so you can actually create flexible queries that produce well-sorted results from your database, and it works everywhere Python does, which is to say, it works pretty much everywhere. It's just as happy operating on a command line as it is on a web server. The results (the actual databases) are 100% portable from OS to OS. I use it on various linuxes, OS X, and Windows for tasks very similar to yours.
Comes with tutorial examples, sample databases and extensive docs. In a 13k (not a typo) archive.
Re:Python? (Score:5, Insightful)
Parent
Re:Python? (Score:5, Informative)
That's all I have for ya, offhand. ;-)
Parent
Re:Python? (Score:5, Informative)
Parent
Re:Python? (Score:5, Insightful)
IOW, SQLite, as "lite" as it is compared to Oracle, MySQL, PostgreSQL, etc. is still overkill.
I'm with the GP -- dbtxt looks really cool. Although, I do gotta say that OpenOffice.org would fit the bill just fine for what his requirements are: Calc can be used as a flat-file database, and I think there is support for simple CSV databases in Base.
Parent
Re:Python? (Score:5, Interesting)
Second, I'm not sure that you know any better than I do what the poster really needs. Open office calc might be an option, but certainly not if 800k is too much space; open office is what, like 200 megs? And dbtxt might also be an option, if doing everything in Python is what the person wants; I love Python, but there are plenty of people who don't. I'm sure that all three proposals are equally valid.
Now, to defend SQLite: in most cases, dealing with a relational database is "overkill" not because of SQL per se, but because of the fact that software needs to be installed and configured*, and because complex sets of processes need to be managed and maintained. SQLite is useful, in part, because it provides the functionality of a relational database without the complexity of the engine. Now, if you don't want or need SQL, then you might not consider that particular feature to be of particular benefit. Nonetheless, there are many situations where SQL might not be required, but where it might be useful, where in the past SQL might not have been used, but where now, because of SQLite, it will be used. There are very few cases where real-world information is not most accurately represented as relational data (in comparison to flat data). SQLite allows you to avoid making compromises regarding the way that that data is stored. And SQLite is fast.
*(Although some interpreted languages might require that sqlite's libraries be installed on a system for it to be used by a program, that is not the case for compiled languages like C, C++, Obj-C, etc., where the sqlite libraries can be compiled directly into the code distributed to end users.)
Parent
Re:Python? (Score:5, Interesting)
Sure, it can be. For instance, adding 800k of stuff to your application means that you are giving up control. You might have the source (which is going to be somewhat larger in this case)... but do you have the time to dig into it if there's a problem? Does reporting the problem necessarily mean it's even going to get fixed? For instance, I've reported many problems to various authors, some of which got fixed, some didn't. Likewise, adding a feature can be non-trivial with a large (source code) project. Compare that to a very lightweight thing that, let's say, almost does what you want but you just need feature X. Might be a whole lot easier to add that to a 20k "almost-there" set of source than megabytes of source, is that not possible?
Distribution of a class module with a python app is an entirely trivial thing to do. For that matter, you can take the class and put it in the same file with your application if that appeals, giving you the advantages of atomic distribution -- harder for an end-user to separate things and make whole app, as you handed it to them, fail to work.
And I suppose the elephant in the room is that the end user may have no need for the features. I sell a commercial program with features not available in other programs in the same general application space, functionality-wise; that doesn't mean it will sell to anyone on that basis unless they actually need those features, right? In this specific case, based on the article, the needs described seemed to be met by what I'd written, and hence my post.
PostgreSQL whips the living hoo-ha out of SQLite; it is larger, but offers more features (sound familiar?) Yet SQLite has a place, as your advocacy clearly indicates. I would venture that the difference between SQLite and class dbtext is similar, just occupies another place on the size/features scale. In the end, the user, based on their requirements, may go "what a POS" and step up the ladder. From either one.
dbtxt also seems to be very stable. It's never failed for me, nor have I ever received any indication that it wasn't doing what it was supposed to do when used as indicated. Also -- as a user myself -- I'm quite interested in fixing it, should it turn out to have problems.
Also, just as an aside, If someone can explain cross-platform file locking in python to me in terms I can understand, and if python's ability to lock can be made to implement a reasonable form of "wait for lock to release", I'd like to add that as a feature, too. I found python's docs on locking to be more than a little opaque. I'm sure it's just me.
Parent
Re:Python? (Score:5, Informative)
Parent
Re:Python? (Score:5, Funny)
That'd make the first one a shameful link, yes?
Parent
Re:Python comes with SQLite (Score:5, Informative)
import sqlite3
mydb = sqlite3.connect('sample.db')
mydb.execute("create table contacts (fname text, lname text, email text)")
mydb.execute("insert into contacts values('Spooky','Monster','spook@spammity.spam')")
mydb.commit()
mydb.close()
You can then use the free and open SQLite database browser [sourceforge.net] to browse, edit, and print your table.
You may think you're keeping it simple by using a flat file, but you're really not. It may be somewhat easier to manually edit, but it's also easier to screw up, and I've never heard of one with the ability to undo changes.
Parent
Re:Python comes with SQLite (Score:5, Interesting)
Class dbtxt includes the ability to undo changes, and features human-readbility for the flat database files. It's trivial and not error-prone at all to mod a database file to delete, or undelete a record, and of course you can do it through the database engine as well.
Parent
Why are we trying to promote python? (Score:5, Informative)
He doesn't need python. He just needs a database. He can download a precompiled binary for windows [sqlite.org] that allows one to work with the database at the command line [sqlite.org]. Python is not necessary.
And if the command line is too much, as others have noted there is already a convenient firefox extension [mozilla.org] for graphically interacting with a sqlite database.
Parent
Re:Python? (Score:5, Interesting)
I think is exactly what you are requesting.
Parent
Re:Python? (Score:5, Funny)
Next!
Parent
Re:Python? (Score:5, Interesting)
If it were not for the Flat File requirement then a simple Python shelve or Perl Tie would be the most logical solution since they are both part of the standard library so don't require installing libs on random computers you might use or port to.
of those two Perl Tie is probably the most suited because it's backed by a real DB operating off the disk not fully in memory.
But why not do both: use YAML as the DB backing the Perl Tie.
then you get a nice human readable flat file.
Parent
Re:Python? (Score:5, Funny)
Parent
Re:Python? (Score:5, Funny)
Parent
Re:Python? (Score:5, Insightful)
I can't legally utilize GPL'd source code within a commercial application without doing some very specific (and not always possible) things that the GPL license instructs me I must do. Any other ways I can help with your basic F/OSS education? I'll be here all day.
Parent
Re:Python? (Score:5, Interesting)
Parent
Re:Python? (Score:5, Insightful)
Why should the GPL hackers share with you if you aren't willing to share with them?
You don't like their license. That's fine, they don't like your license either.
Parent
Re:Python? (Score:5, Informative)
Parent
Re:Python? (Score:5, Insightful)
IMO, the people who still, after all these years, haven't figured out how to make money with GPL software are the failures, not the GPL.
Its just a software license; it doesn't preclude you from charing money for your work, and it doesn't mandate you make your work avaiable to the general public. I can't fathom why some seemingly intelligent people have such a hard time grasping this concept.
Parent
Re:Python? (Score:5, Insightful)
To be fair, a proprietary licenser is also free to fix the bugs ASAP and well, provide such fixes for free, also to provide new features for free if they like. I do this all the time with a commercial application. A PD author also retains such freedom. These are not benefits that exclusively arrive via the GPL. And it is well to consider that with a GPL, PD or commercial application, one is free to demand money to fix something, to add features to it, to explain or teach its use, or even to simply use it. Doesn't mean you'll get it, but it doesn't mean you won't, either.
I don't think you've really analyzed this far enough. Giving someone something, and then telling them what they can and cannot do with it, carries ethical problems in the form of imposing your will on someone else (and it also devalues the gift, in my personal opinion.)
"Here's a gift of a book; but you can only read it if you use LED lights." "Here's a gift of a cat; but you can only have it if you'll eat it." "Here's a gift of a some money, but you must spend it upon me."
You see? This is why the term "freedom" has always seemed to go so poorly with the coercive requirements that the GPL applies to the various recipients in the chain of "gifts."
Parent
Re:Python? (Score:5, Funny)
Parent
No Java? (Score:5, Insightful)
I would have recommended HSQL, but you don't want Java. Frankly, usually, when we're talking databases I won't say "use a spreadsheet", but with 10 fields, you might as well use a spreadsheet. Of course OpenOffice.org Base is out, because it uses HSQL.
Something like CSQL [sourceforge.net] might fit, but I have no experience with it.
Re:No Java? (Score:5, Insightful)
This guy wants a spreadsheet, he just doesn't know it. Excel--or the free alternative from OpenOffice.org--will do everything he could possibly want, and although it saves as its own infernal file format, it exports competently into a tab or comma-delimited format.
All the extra "requirements" are just pseudo-intellectual mumbo-jumbo that have no bearing on reality.
Parent
Err ... (Score:4, Informative)
Otherwise what's wrong with a simple database like MySQL or PostgreSQL on your computer?
Re:Err ... (Score:4, Informative)
Smith,John,703-555-5555
into
Smith,John,-5407
The latter being 703 minus 555 minus 5555.
when importing a CSV file. No joke. There are ways around it, but the default is pretty braindead.
Parent
Re:Err ... (Score:5, Funny)
Parent
Re:Err ... (Score:5, Insightful)
Parent
OOO? (Score:5, Informative)
Re:OOO? (Score:5, Insightful)
Parent
sqlite (Score:5, Informative)
I find it amazing that you didn't come across it in Googling...
Re:sqlite (Score:4, Insightful)
Parent
Re:sqlite (Score:5, Informative)
https://addons.mozilla.org/en-US/firefox/addon/5817 [mozilla.org]
Parent
SQLite (Score:5, Informative)
There are GUI clients that work fine for this sort of thing, SQL is simple for doing basic things. One file, one database.
how about... (Score:4, Insightful)
Umm... just write a few one-liner perl scripts to get info out.
sed, awk, and grep... (Score:4, Informative)
Parent
heh (Score:5, Funny)
Flat file is useless (Score:5, Insightful)
Re:Flat file is useless (Score:4, Insightful)
Well... XML is human readable in the same way as a sand is edible.
If the OP wants to use a tool better than notepad for editing, he probably should try Open Office.
Parent
Spreadsheet (Score:4, Informative)
SQLIte or BDB (Score:4, Informative)
BDB is *not* a relational database though, it's just a storage/indexing engine, which is used most notably by MySQL as a backend. SQLite on the other hand is a full file-based RDBMS with a small runtime, so it might be a bit of overkill for you in this particular scenario.
But both of them run on Windows, Linux and the BSDs, so you won't have portability problems. And most languages have bindings for them.
GDBM (Score:5, Informative)
If that doesn't satisfy your need, take a look at Berkley DB. It offers a more sophisticated interface than DBM.
What about XML? (Score:4, Interesting)
Read the XML records into an STL container for easy access.
Try Metakit (Score:4, Informative)
For your reference only (Score:5, Insightful)
So why not simply write out a tab-delimited file? Retrieval is as simple as reading every line, splitting by the delimeter, and regexp'ing whichever field you're examining. 15'000 records takes about two seconds on a modern machine. Of course, for anything more complicated, that same file gets easily imported into your favourite spreadsheet application.
You don't need to worry about locking because you're the only one using it. And otherwise, your application simply locks the file handle, or creates & destroys a traditional lock-file.
It's a thirty-line perl script (of legible perl). You can do it in JScript as a local HTA if you want the benefit of html etc. interfaces.
How about a full Oracle Database 11g Enterprise (Score:4, Funny)
Sounds like he wants an APPLICATION not a framewrk (Score:4, Insightful)
You really do NOT want a DBMS, use a spread sheet. (Score:5, Interesting)