RDF For Desktop Metadata?

claes writes "There is an article "Metadata for the desktop" that suggests that RDF should be used to describe data in desktop environments. This is an interesting idea. RDF is already used by Creative Commons to attach license metadata to its works. Mozilla also supports it. RDF was designed for the web, but can it also find its way to the desktop? And what metadata is most important to describe?"

The killer app for metadata on the desktop

is porn!
Suppose today I want to see shaved asian hardcore action. Now provided that metadata searches are integrated into the OS(like they will be in Tiger), all I need to do is a quick metadata search on my hard drive and boom, there is what I am looking for.
I mean provided there was a decent standard(a porn standards body would rule!) and good regex capabilities built into the OS, I would be willing to pay for porn. I know that there are comments built into the jpeg standard, but there are all sorts of porn file formats, it would be helpful to have a universal standard across them. It saves time, beats trying to search on google and going through a lot of crap just to get to something good. I am a man on the run, I have places to go, I can't be bogged down by my porn. Plus, think of the people that get to catagorize this stuff(well, the fun stuff anyway, not goatse), what an awesome job that would be!
I should probably post AC, but I figure this post is bound to earn me at least one fan and/or freak.

Re:The killer app for metadata on the desktop

Suppose today I want to see shaved asian hardcore action.

Just check your email. If it's not there now, it will be soon enough.

Definition:...?

Why don't slashdoters define what meta-data is in the first place? Google's define: metadata lists not less than 20 definitions. Are we talking about "data about data"?

Re:Definition:...?

Yep, it's called like that.

I don't see with the thread started wanted a definition by Slashdotters in the first place, since it's already pretty well described [wikipedia.org] and AFAIK the word doesn't have several meanings.

Implicit feedback for filesystem information

I am a big fan of implicit filesystem feedback [peertech.org]. This can support all kinds of services from file sharing to most recently accessed search requests. Even fine tuning access controls in an RSBAC security policy.

The big concern is keeping this data protected and private. You dont want to share all of your metadata with everyone, so security of these systems should be something to look at carefully.

What happened to forked files?

Are there any filesystems left that use forked files? Resource, Data and Metadata forks? Any at all?

While MacOS was at a disadvantage being one of the only ones to use it, wouldn't it have been an excellent advantage for ALL filesystems to be forked?

(I don't know the answer to this - anyone who knows more about filesystems, give your thoughts)

Re:What happened to forked files?

Forks? Would that be the NTFS streams [alcpress.com]?

I think the new filesystem WinFS in Longhorn is basically just an evolution of NTFS streams to make them more accessible for the users. They've always been there, just not very accessible besides a limited set of text fields in the file properties dialog box in Windows. (i.e. they've always been able to hold custom data and have custom key names)

Integration

Why does the document complain about the lack of integration, then mention that Microsoft, Apple, the ReiserFS people, etc. are coming up with solutions, and then adds a completely new one? Shouldn't they just be supporting one Apple's or ReiserFS's efforts?

What is wrong with you people?

Sure. I have no objection to a more extensive use of metadata. In fact I crave it - must have it.

But why oh why do people think that XML-based solutions is the way to go? An RDF solution would be bloat beyond belief. Ok, so it's not that bad for a few files, but when we get down to it - we don't have just a few files. We have plenty of them.

So why not use something smaler? A simpler protocol?
We can still have RDF-frontends for those that crave their daily XML-fix. Get real.

This is largely irrelevant if you have experience

Since most of us are advanced computer users or even computer experts, I think we largely know how to search for content.

For one thing, I always give my filenames relevant titles, not things like document06.doc.

Also, I already know how to search through files for content using basic grep or advanced Windows searching.

I mean, sure, meta data like ID3 tags for MP3s that I steal offline are important because my Nomad mp3 player indexes based on that info, but in general I'd say meta data is not quite as important as some may suspect.

FS support for metadata

I've heard the NTFS file system is designed to allow the system to add any number of properties (besides the obvious filename, last access time and permissions) to any stored file. This is likely to be exploited by Longhorn, which is planned to be capable of appending metadata to newly created files (for example, if you download a file from the Internet, the system would likely append a Originated-From-URL property to it).

What I wonder is, is there any filesystem in the FOSS world that supports something like this, or are there plans to make it supported before 20??, when Longhorn hits the stores? I see this as a critical feature that must be made available by non-Windows OSes.

Re:FS support for metadata

Actually, you can. To add a metadata item called "hidden.txt" to a file called picture.jpeg, just type on the command line:

notepad picture.jpeg:hidden.txt

Notepad should say that it "created the file." You should notice that no new files have been created: just look for them with explorer. But you can later open this "file" and read and edit it.

You can do this with any file with any metadata name.

let's keep the Meta data simple...

Who
What
Where
When
Why
and possibly How...

Spotlight

I'm mostly wondering if the new Spotlight [apple.com] feature of MacOS X 10.4 is going to be based on this, or a proprietary technology. I've been itching for cross-platform metadata file support for years now...

Can't wait..

for when I can just throw out the whole desktop in favor of a "cloud" of data... using google-like interfaces to find my stuff. I think it would be interesting to figure out how to tell a compiler where to find stuff...

discussions about winfs and rdf

Danny Ayers [dannyayers.com] has some interesting discussion on his blog about winfs and rdf [dannyayers.com]. There's also discussion of Jon Udell's Questions about Longhorn [infoworld.com].

Haystack and Metadata efforts

A group at MIT is using RDF for an integrated data management system. It's sorta like Outlook (or Kontact, if you prefer ;-) on steroids. It's called Haystack [mit.edu].

I have to say, their ideas are intriguing, but after using it... I think the big shortcoming is that it's tough to come up with a generalized user interface for manipulating any data thrown at it. Haystack tries at this, and I think, fails at providing any kind of cues or context that tells you what your are dealing with. In Haystack, every task and piece of information you deal with looks very much like every other piece of data, because, as a design choice, Haystack every piece of data has the same rank as every other piece of data.

Having different applications for different types of data usually make sense, if only to limit the amount of options presented to the user so they can make an intelligent decision about what action they want to perform. See this article on Slashdot about how users need limited [slashdot.org] since it makes decision-making too difficult psychologically.

Inevitably, discussions around RDF and metadata always devolve into hand-wavy discussions on how the computer will be able to "magically" do smart things based on the metadata. But it really isn't magic and it isn't automatic at all. Equivalencies and mappings have to be created by humans along with the rules about what to do.

RDF uses many concepts from AI research. Anybody who has read about this branch of computer science knows that the discipline has pretty much given up on creating AI in the 'sci-fi' sense as an impractical dream. That's what makes the Loebner prize [loebner.net] so controversial. I don't expect that computers will be intelligent enough able to relieve users of too much of the burden in assigning metadata.

RDF is a promising approach, but if you read the article, it makes a lot of assumptions about what needs to happen to make the benefits real. Among them are establishing standards for what metadata fields apply to different types of objects: photos, people, music, etc. That kind of standardization won't happen overnight, if at all.

The computer also needs to know what to do when it encounters that kind of data. The article mentions MIME and browsers and, in effect, says the browser can make a rational decision even if it hasn't seen a particular MIME type before. That isn't really true.. you have to install a plugin that tells the browser what to do, or have a registry that someone has put together where the browser can install the right plugin at the right time.

That said, KDE's unification of contact information and passwords does show some of the promise of metadata efforts. And Apple's Spotlight looks like a good solution as far as it goes. I guess I'm just trying to make the point that the magic of metadata needs to be taken with a fairly large hunk of salt.

Many community websites don't permit RDF

I have a couple of articles that have Creative Commons licenses, and I tried at first to include RDF in them.

But when I tried to publish one article at Kuro5hin, the RDF code, which took the form of HTML comments, was displayed literally in the visible body of my article. That is, all the tags had been turned into entities so the tags appeared literally in the rendered text.

I think Kuro5hin's Scoop content management system doesn't permit HTML comments. Maybe it's not trying to suppress comments, but it didn't occur to scoop's developers to allow them.

RDF on the web would likely be much more popular if one could count on publication sites allowing it in the submitted markup.

Another problem I had is that Creative Commons' recommended way to apply a license to a web page is not permitted by any of the community sites I frequent. CC-licensed web pages usually have a small banner that links to the license text. But for obvious reasons, sites like Slashdot and Kuro5hin don't permit images in article or comment submissions.

The result is that, even for the copies of my articles on my own website [goingware.com], I use neither RDF nor the CC banner, because I want to make it easy for others to copy my CC-licensed articles to site that don't permit RDF or graphics.

The way I apply the license is the much-less-cool method recommended for plain text files. I have the following text appear in the body of my articles:

This work is licensed under the Creative Commons Attribution-NoDerivs License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nd/1.0/ or send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, California 94305, USA.

libferris && RDF

Hi,
A bit of a shameless plug, but none the less: I think that folks who
liked the ideas in Edd's article might also be interested in my
project, libferris [sourceforge.net].

Ferris allows metadata to be extracted from files and presented through
a uniform interface. It supports inference on metadata and has the
ability to index that metadata in many ways (eg. Berkeley db, odbc
LDAP). Note that the metadata index can be used to index anything
libferris can mount (XML, ODBC, RDF, LDAP, http, ftp...)

A cool thing related to Edd's piece is that you can read an inferred
attribute "as-rdf" to obtain all the metadata that libferris knows
about for a file as a single RDF/XML file.

Isn't it the same problem?

After reading this article, I'm wondering if metadata is really going to be as effective as the author thinks it is. The author points out that, "the computer makes us do the work of a filing clerk". In other words, when you place a files on your computer, you normally place them into a folders to organize them, which is, "not fun". The author implicitly claims that metadata will solve this situation.

But that's the problem! If it's not fun to organize items into folders, how is it anymore fun to add metadata to a file? I'm not talking about text files. Text files are easy, because you can pull the metadata out of them automatically (in fact, you can do this now with search tools). I'm talking about files that have to be explicitly tagged with metadata, like pictures. How is adding metadata to each picture file to categorize your vacation pictures any less laborious than placing the vaction pictures into their own directory?

That's the problem as I see it. You still end up being a filing clerk! If people don't even organize their folders now, are people going to use metadata when it's available? Will improved search capabilities make users want to be clerks?

In a nutshell, isn't it the same problem?

Creative Commons & Desktop Metadata

CC is interested in desktop metadata developments. See this CC weblog post [creativecommons.org] from a few days ago.

RDF is not practical

I've yet to see a real world example of how to use RDF that wasn't for research(ie to prove RDF works) purposes. Most of the projects listed for semantic web are purely research, toy projects, or completely unproven. I know of several companies that have tried, but they usually end up extending the hell out of RDF to make it practical and useful. That makes me think RDF is flawed.

what metadata is most important to describe?

1. Ditto to the post which said, "Separate the apps, not the data." The current proliferation of app-specific formats is absurd and counter-productive.

2. I file hundreds of docs &/or URLs per day. I need something which offers some degree of assistance in immediate auto-categorization (e.g. Bayes) with feedback, while still allowing user-defined hierarchy. "Yes, thank you for intelligently recognizing that this new info is about device interrupts; but now I need to tell you that it's about kernel-coding vs. crash-debugging vs. performance-analysis."

3. One poster calls the article, "self-maturbation on the part of bored jobless software engineers that aren't solving any problems that need solving".
Speak for your yourself. Yeah, I'm a developer, but most of my minute-to-minute usage of my desktop isn't all that different from "lusers" or PHBs, i.e. massaging info.
Get some perspective. Your statement is like saying, "Cars are really primarily made for mechanics and automotive engineers, not for soccer moms and commuters."

4. Forked-data: sure, as long as it's restricted to the app-specific stuff. Take that table the user just created: use forked-data for the meta-data which is specific to the spreadsheet or WP app, but leave the table data as ASCII data which anyone can read.

5. Someone said, "a file-name should be enough". Speak for yourself; a lot of my needs go waaayy beyond that. If the metadata goes beyond your neeeds, then your course is clear: just don't use it. It costs you nothing to architecturally allow for its use by other people.

6. re: "clouds", there are times when I'd really like to know -- what app created this file? what OS? which host? which user? what other files had been opened (e.g., stdin)? what was the original volume label? etc.

Watching the XML kiddies reinvent the wheel

It's fun watching the XML kiddies re-invent concepts from LISP. They just re-invented property lists, "is-a" links, and much of the baggage that made SGML painful.

Knowledge representation via "is-a" links has been tried, and it breaks down rather quickly. Read "Artificial Intelligence meets Natural Stupidity", by Drew McDermott, for a 20 year old critique of this concept. It's overkill for searching, and not powerful enough for reliable automated question answering.

The Cyc debacle [cyc.com] illustrates how much work you have to put into tagging to get very little out. After twenty years of that money sink, it's still useless.

RDF (and OWL) in Pike

I noticed the article made no mention of Pike (also the name of a fish - see language logo). Pike's a fine C-like scripting language ...that I know extremely poorly myself, but anyway..

From Pike's official homepage [ida.liu.se] (at the University of Linkoping, Sweden):

The release of Pike 7.6 marks the first results of a long-running project to make Pike the first scripting language for the Semantic Web. The current highlight in that respect is the support for W3C's standard formats RDF and OWL.

Worth downloading [ida.liu.se] and checking out for other reasons [ida.liu.se] than "just" RDF & OWL [w3.org]. Free software, available under LGPL, GPL, and MPL (Mozilla Public License).

Look also at XMP

When looking into metadata, people should probably be sure to check out XMP [adobe.com]

It's from Adobe, and whereas RDF just says how to format metadata, XMP addresses what to include in your RDF, and how to place it into different types of files. They have free libraries, but it's simple enough to follow even with your own code. And... given that it's how all Adobe products are doing metadata, at least in the publishing world it will probably stay something to pay attention to.

Creative Commons has addressed this [creativecommons.org], and I first hit it in researching implementing metadata support for Inkscape [inkscape.org].

The more things play nice together, the more users are likely to adopt using them.

RDF

Did anyone else read RDF and think.. Reality Distortion Field ( Steve Jobs)

Another format..

Good, yet another format to use/suffer!

No matter how good those formats are (XML/RDF/etc) they all fail at the simplicity norm, the KISS principle.

In the example of the article, by not using a simple text oriented format they innecesarily complicates the access by any program to these values, and that leads to the second point.
The computational cost involved in parsing / validating all those formats; the day that our cpu's can process hundreds or thousands of simultaneous parsings without a noticeable impact on performance, that day it could start to make sense to popularize his usage, until then, they are a luxury and as such restricted to a limited (especialized) usage.

On the RDF case, metadata is data, the 'meta' part is a human hability and can be used wherever we want, no need for a special format. By pretending to format the 'metadata' concept we are just defining a new stream format, and if we consider how wide the 'meta' concept is, it seems dificult to limit to a simple ontology. The result? the need of another international consortium to stablish a reasonable set of vocabularies, big deal!

I think there are better ways to spend our cpu cicles than to parse verbose formats, but how knows?

Desktop.ini

Maybe it can replace Windows' crappy Desktop.ini file, allowing for complex Desktop setups.. one use i see for it is to have links to all the icons, programs they start, etc...

XML?!?!?

I would say that RDF, or any XML format, is unacceptibly wasteful for metadata. Besides, many filesystems already support extended attributes. Why not use the mechanisms that were developed exactly for this purpose, instead of introducing a new and inferior one?

Forgive my zeal, I just really hate the XML for everything mentality.

Example of missing metadata

I think there is too little metadata about installed applications in a regular Linux system. There is metadata in the packages (RPM for example), and there is metadata in the .desktop files. But the package metadata is on the package level, and does not describe each individual application it contains. The .desktop files are very sparse, and describes things that fit on one line of text or less. This makes it hard to write new kinds of user interfaces. I can't find any way to make a freshmeat like user interface acting on the software that is actually installed in the system, since there are no descriptions that are longer than one sentence in a .desktop file.

Metadata with clipart & Inkscape

There's been work on adding Dublin Core metadata support to Inkscape, for its next release [inkscape.org].

The need for the metadata support is entirely practical in this case: the Open Clip Art Library [openclipart.org] requires all SVG submissions have proper metadata embedded, to ensure licensing and authorship correctness. Also, there is an SVG Clip Art Browser [openclipart.org] that uses the metadata info for its display.

One interesting observation that's come up recently and is being discussed on the lists [inkscape.org] is what happens when you embed several pieces of clipart into a larger document, how do you access the RDF of the individual bits in Inkscape?

Re:It's coming

The "core" of windows is what makes windows, well, windows.. and is therefore not cross platform. do you even know what you just said?