Tom's Hardware Looks At WinFS 809
Alizarin Erythrosin writes "Tom's Hardware Guide has an article about the new WinFS file system. The article talks first about some of the problems and advantages with FAT[16|32] and NTFS, then talks briefly about WinFS. Here is the summary: 'Microsoft is breaking new ground with Longhorn, successor to XP. The upcoming WinFS file system will be the first to be context-dependent, and promises to make long search times and wasted memory a thing of the past. Today, THG compares it to FAT and NTFS.' Personally, I still have reservations about using a relational database to keep track of files. Unless they can keep the overhead to a minimum, I can't see it being as efficient as a file system should be."
other FSs are out there (Score:3, Interesting)
Re:other FSs are out there (Score:5, Interesting)
Re:other FSs are out there (Score:5, Informative)
Writes are about 10% less efficient, which a pretty good tradeoff for that peace of mind.
Re:other FSs are out there (Score:5, Insightful)
Re:other FSs are out there (Score:5, Insightful)
And how would you propose to achieve that without performance going down the drain?
The guys writing these filesystems are not dumb, and the reasons why journals are used are well considered. Another thing is that ACID compliant databases also use something like a journal to achive the atomicity.
Oh, and forget softupdates, they are _not_ comparable to journaling filesystems, for instance you still need to fsck, it's just faster.
Compare that with one of the funnies manpages I know [sgi.com].
Re:other FSs are out there (Score:3, Interesting)
That's true, but misleading. If soft updates are done right, the only reason to fsck is to reclaim resources (orphaned blocks etc.). It is not necessary to get your filesystem into a usable state, and can therefore be done in the background after you've come up. Journaling filesystems also still need to fsck, it's just faster and it's called a log redo, and that is nec
Alas it doesn't help (Score:5, Informative)
Re:other FSs are out there (Score:5, Informative)
Re:other FSs are out there (Score:5, Informative)
Journalling makes sure that the state of data, or meta-data is always defined. So even if you turn the computer off while in the middle of a write command the file-system will be in a correct state afterwards. (The write operation will be lost however.) It is basically a way of making HDD operations atomic. Either they succeed or they fail, the in between state is what puts your drive in a corrupted state.
When talking about 10% it is refering to space on the disk for the log file to make this possible.
Now storing the meta data in a database, which is essentially what WinFS and such are doing, is not as clear a benfit. Personally I can imagine that it would be a very practiacal FS for keeping movies and MP3's on. I don't really see the benefits of running the OS files on that FS though. A lot of unneccesary overhead. (I don't search for files in my OS partition very often.)
Re:other FSs are out there (Score:5, Interesting)
Now storing the meta data in a database, which is essentially what WinFS and such are doing, is not as clear a benfit. Personally I can imagine that it would be a very practiacal FS for keeping movies and MP3's on. I don't really see the benefits of running the OS files on that FS though. A lot of unneccesary overhead. (I don't search for files in my OS partition very often.)
Indeed. It seems like what they are claiming as an improvement, (i.e., faster searching for files), does not appear to help what people actually do most of the time. It is similar to claims of "boots much faster!" that you used to hear about new versions of windows. I would think the thing that would be important to people is data integrity and access efficiency. I know my primary concern is "how safe is my data".
I also question the need to include the overhead of a database frontend to the filesystem. Seems like a catastrophe just waiting to happen.
Also, since the DB is always active, what issues do you have with backups? I'd be concerned about backup and restoral issues with this type of filesystem. I haven't seen that addressed at all.
Re:other FSs are out there (Score:5, Insightful)
BeOS used a metadata enabled fully journaled multithreaded filesystem and it was the bomb.
Mail client? What mail client? You email inbox was a directory in the OS with all the email metadata attributes enabled in the file manager. Mail filters literally became shell scripts. Want to index your MP3s according to artist, album, etc all at the same time? Use the shell scripting and metadata, Luke!
It also does away with the stupid limitation of extensions. When all your data is stored as a datatype in the metadata listing, who needs them?
I think that Microsoft are going a little overboard embedding a database into the filesystem, proper meatdata enabled filesystems are a GOOD thing in my book.
Performance is a question of whether they care (Score:5, Interesting)
The difference isn't features - BeFS supported everything HFS+ does and arbitrary attributes, journaling, much larger file/filesystem support, and indexing and it was still faster. Be simply made performance a much higher priority than Apple has so far; fortunately they've hired the BeFS lead developer and perhaps 10.3 will have some surprises.
Another good example is ReiserFS - while some of their choices reflect overall design goals (e.g. targeting large numbers of small files instead of BFS's massive videos) they've largely passed the traditional filesystems in most areas despite having to do more work to keep all of the extra features going.
Microsoft has a number of engineers who do understand performance; the question is simply whether it'll be a significant priority for them to make WinFS fast enough that we'll realistically be able to use it.
Re:other FSs are out there (Score:3, Funny)
you can get WindowsXP to run in 2 KibiBytes of ram?!
or
Sorta like running XP in 2K mode
not that that's a bad thing....
Re:other FSs are out there (Score:5, Informative)
So when is the last time you actually used WindowsXP? What you said is pretty silly.
You can install WindowsXP on FAT32, even create FAT32 partitions during setup etc, all the same options you get with NTFS during the install.
Are you trying to make up stuff or just never used it?
Geeshâ¦
Re:other FSs are out there (Score:5, Informative)
The limit is 32GBs.
FAT32 Formatting in Setup is an option, but is known to frequently fail or have significant errors
No it doesn't.
FAT 32 Formatting once you are in the OS requires 3rd Party wares.
Again, only if creating partitions bigger than 32GBs.
Re:other FSs are out there (Score:4, Informative)
You cannot format a volume larger than 32 gigabytes (GB) in size using the FAT32 file system during the Windows XP installation process. Windows XP can mount and support FAT32 volumes larger than 32 GB (subject to the other limits), but you cannot create a FAT32 volume larger than 32 GB by using the Format tool during Setup. If you need to format a volume that is larger than 32 GB, use the NTFS file system to format it. Another option is to start from a Microsoft Windows 98 or Microsoft Windows Millennium Edition (Me) Startup disk and use the Format tool included on the disk.
I've done it myself, with and without SP1 slip-streamed, and it should have let you do it as well. Are you certain the partition was only 30GBs?
Re:other FSs are out there (Score:5, Informative)
FAT32 Formatting in Setup is an option, but is known to frequently fail or have significant errors.
FAT 32 Formatting once you are in the OS requires 3rd Party wares.
SCO has performed an illegal operation and will be shut down.
I hate to rain on your parade, but the information you are providing you are either mixing with NT4 information or just making it up as you go.
Nothing I can say is convincing you, so let me reference you to the information you need.
http://www.microsoft.com/windowsxp/pro/using/ho
It clearly provides information on the only limitations of doing a FAT32 installation of WindowsXP, and that is the inherent limitation of FAT32 only being able to recognize a 32GB partition size.
As for '(I belive)2Gb', you are referring to the FAT16 installation of NT4. It doesn't apply to WindowsXP.
FAT 32 Formatting once you are in the OS requires 3rd Party wares.
Here let me quote from a command prompt for your edificationâ¦
Format
Formats a disk for use with Windows XP.
FORMAT volume [/FS:file-system] [/V:label] [/Q] [/A:size] [/C] [/X]
FORMAT volume [/V:label] [/Q] [/F:size]
FORMAT volume [/V:label] [/Q] [/T:tracks
FORMAT volume [/V:label] [/Q]
FORMAT volume [/Q]
volume Specifies the drive letter (followed by a colon), mount point, or volume name.
Notice the last line?
So where do you get your information, tooth-fairy, or just make it up as you go? Third party utilities are NOT needed to format FAT32 in WindowsXP, as you said âonce you are in the OSâ(TM) â" BTW, did you know that even during Setup â" YOU ARE âIN THE OSâ(TM)? It is running a stub version of NT already at that point; you are already in the NT OS.
FAT32 Formatting in Setup is an option, but is known to frequently fail or have significant errors.
Since when? Where do you document this at? Funny in all the time our tech team has put into working with WindowsNT, they can find no reference to this, either online or in their own experience. Tooth-fairy again or just making it up as you go?
There are NO known issues or documented issues of WindowsNT EVER having a problem running on a FAT partition, either FAT16 from NT4 days or FAT32 of current days. The NT core has an installable File System, like most modern OSes, and its interaction with the file system underneath is irrelevant to how it operates in regard to failure or problems.
Besides, this is really a moot point. If you are running WindowsXP and it is your only OS, then there is NO reason not to use NTFS. It is faster with larger files, faster with large volumes, supports compression, supports encryption and can have partitions up to 16 exabytes in size.
If you are running WindowsXP on FAT32, you are missing half the benefits of WindowsXP and the security of NTFS as well as its reliability of sitting on a journalled file system, etc, etc.
Geesh.
Re:other FSs are out there (Score:5, Informative)
AFAIK, the NTFS installation of NT4 won't allow you to have a primary (system) volume > 4GB. This is because NT will install a FAT16 volume and later convert it [is-it-true.org]. This may have been fixed in a service pack, but until you install the OS, how are you going to get a SP on there? Sure, you can
grow the partition later, but you're being a bit disingenuous in your specifying that this problem is confined to FAT16 installation on NT4, since an NTFS installation *uses* the FAT16 installation.
Re:other FSs are out there (Score:4, Informative)
grow the partition later, but you're being a bit disingenuous in your specifying that this problem is confined to FAT16 installation on NT4, since an NTFS installation *uses* the FAT16 installation.
Nice information, but it has nothing to do what I was talking about.
If you read the post, you will see that I was responding to a post that said WindowsXP had a 2gb partition limit for FAT32 installations. Which is NOT true.
I never said anything about the primary boot partition of NTFS on NT4 or its size.
In reference to what you said, you are partially correct, SETUP of NT4 and previous versions will not allow for a boot partition larger than 4GB; however, you can pre-partition a Hard Drive with NTFS using another WINNT computer to whatever size you would like and then install WINNT4.0 on the blank larger NTFS boot partition. (This is a well known workaround in the industry.)
As an additional note the original Pre NT4.0 setup did not fully load the NT Kernel and related drivers to support the drives or the NTFS file system, this is why the installation created a FAT partition for the file copy portion of the install. NT4.0 did fully load the NT Kernel, but the original installation mechanism for partitioning was left in from the NT 3.x days.
Windows2k and WindowsXP do not have these restrictions and fully load the NT Kernel during the initial file copy portion of the Setup.
People wanting more information try:
http://support.microsoft.com/default.aspx?s
Take Care,
The Net Avenger
Re:other FSs are out there (Score:4, Interesting)
The term for that is 'non-sequitur', and you've just posted a lovely example of one. Let's go back to my post and see what I was replying too (hint, nothing to do with XP at all) - it's the bit you snipped:
As for '(I belive)2Gb', you are referring to the FAT16 installation of NT4. It doesn't apply to WindowsXP.
That's what I was replying to. I was attempting to clarify that the limit (4GB, not 2GB) also applied to an NT install in which you specified NTFS (your post seemed to imply FAT16 only).
I don't think we're disagreeing. I was clarifying a point you made which could imply something which wasn't the case.
Re:other FSs are out there (Score:5, Insightful)
Re:other FSs are out there (Score:5, Informative)
Re:other FSs are out there (Score:3, Informative)
ADM291 Tour of the Sysinternals Tools The Sysinternals.com web site is source of dozens of powerful freeware tools that reveal aspects about the internal behavior of the Microsoft® Windows® operating system as well as the activity of applications that run on it that are otherwise not available, making them valuable troubleshooting and administrative tools. Many are used on a daily basis with
Can you say SQL Slammer x 100? (Score:3, Insightful)
Re:Can you say SQL Slammer x 100? (Score:5, Insightful)
How many gaping security holes have there been in NTFS?
By most accounts, NTFS is one of the better filesystems ever written. Journaling, ACLs, decent performance.
There is talent in Redmond. Ignoring that is as flawed as assuming the entire Linux community is represented by Sendmail and SCO (security holes and bad publicity). Pretty bad, right?
Re:Can you say SQL Slammer x 100? (Score:5, Informative)
Re:Can you say SQL Slammer x 100? (Score:5, Interesting)
Re:Can you say SQL Slammer x 100? (Score:4, Informative)
Slammer exploits a buffer overflow in the sql server "named instance" resolution mechanism. It has nothing to do with the storage/querying/indexing/etc of relational data.
db filesystem (Score:5, Interesting)
BeOS used indexing for certain attributes, and it is GREAT. Maybe someone is just sour that linux didn't do it first?
Re:db filesystem (Score:5, Interesting)
I gathered that the quote was alluding to the fact that while the BFS did initially use a full relational database backend, it performed very poorly. Be replaced the backend with a more conventional one, but kept the SQL-like interface to it. It increased performance, but just wasn't quite as cool anymore. Maybe now that PCs have increased in power by several magnitudes since Be last tried this, Microsoft may actually be able to pull it off.
Re:db filesystem (Score:3, Informative)
Well according to their past history this is exactly what they would do. Messing with NTFS might break backward compatibility.
Why do you think MS has changed their stripes? Same management, same phi
Re:db filesystem (Score:3, Insightful)
Re:db filesystem (Score:3, Interesting)
"You dont need a database when you have our file system." they now can say.
MS is basically putting everythingn under the sink into the operating system so that no one can compete with them. A practice they started after they could not have hidden systems calls to make MS own applications faster than competitors and could not force others to ship IE.
This way all other software companies go belly up since they canÃt offer anything that not alreadu is p
Re:db filesystem (Score:5, Insightful)
a) I want to store that it's a jpg file. I am Japanese and see no reason why the file type should be indicated by a small dot shape followed by three symbols left over from the roman empire that stand for some words in a language I don't speak. File name extensions are very archaic technology.
b) I want to record when I took the photograph that this picture represents.
c) I want to record that this photograph is going in my family album
d) I want to record that this is an 'outdoor' photograph
e) I want to record that this photograph is of Mount Fuji
f) I want to record that this particular file is the high-resolution version, suitable for output onto my photo-printer.
g) get the point? I could have a lots of symlinks of this file to folders called 'my photos' and 'outdoor photos' and 'photos taken in July' but that would be a stupid ugly hack. I could call my file:
mount_fuji_july-hi_res-outdoor.jpg
but that would be even more stupid.
Now, at this point someone says "well, if you want all that then just buy a content-management application that sits ontop of the OS and lets you catalog all your stuff."
I think MS has realised that pretty much every file _should_ be in a content management system of some kind. They are simply adding a lightweight CMS to the OS, which seems perfectly sensible to me. That way, other applications can make use of the meta-data the CMS holds. The file open dialogue on all windows apps will, instead of a directory browser have the ability to query the CMS for, say 'most recent versions of my hi res outdoor photos' which seems good to me.
I never fail to be baffled at the degree of inertia in the IT world. I sometimes thing every computer person thinks technology should be frozen at whatever point they got tired of learning new stuff. "File name extensions and symbolic links were good enough for me lad!" It's a weird attitude.
Re:db filesystem ... will never be used by most (Score:5, Insightful)
How many people click on Properites in M$Word and put in the information?? How many people download files and leave them lying in whatever directory they just happen to fall in.
Don't take me wrong
It's just that I have a hard time getting excited over something that is going to simply bloat a system and the odds are no one will use.
My girlfriend is fairly smart, but she still downloads all her pictures into the default folder, and uses thumbnails to find the ones she wants. She has about 1000 of them, and it only takes here a few minutes to find the ones she wants. It would take here no extra work than what this new FS is suggesting to rename the file and/or store it in another folder
Useful feature, bloatware, Linux beater, or disaster waiting to happen. My guess is all of the above, at one time or another. Some people will use it and spend hours cross-linking files. I'm sure the initial releases will have security or data loss issues until the bugs get worked out. It will take 0.10 minutes for some Linux hacker to reverse engineer the ability to at least read it. And it will probably take up gobs more memory.
It's all a matter of perspective....
Re:db filesystem ... will never be used by most (Score:5, Interesting)
For most file data, perhaps.
I will use this, and to good effect, as well.
The point to take into consideration is that the context will also change depending on the metadata available. Your view of the aggregate file objects changes, depending on the context. Not to mention that this same metadata will be available, in the same format, to all participating applications. Your apps can have all the same view, if you like.
What this means in concrete terms is that your carefully sorted directory of MP3's can look like a file library in iTunes. There are searchable, sortable columns for Title, Album, bitrate, Cover Art, year, label, and whatever (note I did not say "filename", which is just another attribute under a modern filesystem). This is possible with only the most basic gestures on the part of the user, and is remembered for the next time you visit this same view.
Similarly, a tree of photographs appear in any participating file browser with whatever columns you want (bit depth, format, date taken, date published, ICC info). It's important to consider that you can do this with any arbitrary collection of data, even one's you define yourself (to take the BeFS example, anyway).
So you can take your collection of widgets, define attributes about these widgets, and your file browser applet works the same for the same user in all applications. It should, anyway. This is why we have APIs.
To cite your example, why visual grep through a bunch of thumbnails looking for a particular photo when you can just indicate with a few gestures the "type" of photo you are looking for? I like the iPhoto interface when I'm browsing photographs, but if I want a particular photo of the GF from a rough date taken at night, I certainly don't want to browse through 1000's of images, especially when some of them can be hard to discern at thumbnail resolutions. I certainly don't want to do this repeatedly when I'm assembling a photo album on a specific subject.
Let the computer do the grunt work of selecting a result set that matches my criteria, and then I can use my human abilities to select the object I want, or refine the search.
Most of us already keep our aggregate file types in associated groups on the filesystem already. In most cases, the tree structure of most filesystems is sufficient. All this does is extended the functionality of the filesystem so that you can choose to abstract aggregate file objects and treat them in a a myriad of different ways. In the most basic sense, you tell the OS, "look, when I have the Explorer/Finder open on this directory of MP3's, make sure you change the column view so it shows this, this and that. In icon view, make sure that mouse-over pop-ups (if enabled) display this that and that. Default sort is alphabetically by Artist's Last Name. I don't want to see the filename, as that doesn't contain any useful information."
That is, you don't have do anything special to make use of the file attributes in this way. You just tell the ultimate app that all of us use the most (the operating system's file browser) to treat certain directories in a different manner.
SCO (Score:5, Funny)
I'll reserve judgement (Score:5, Insightful)
In any event, Microsoft still has a few years to refine this "Future Storage" file system, so all judgements concerning it's effectiveness are a bit premature on some levels. Then again, it's always good to start planning as early as possible - especially when you consider that it may be introduced into Windows Server 2003 some time during the next 12-18 months. For now, all we can base judegement off of is Microsoft marketing hype and comparisons to existing file systems that operate in a similar way.
Re:I'll reserve judgement (Score:5, Insightful)
Re:I'll reserve judgement (Score:5, Informative)
What WinFS adds is more powerful indexing, not a new storage system. Whenever you add or modify a file, WinFS adds the file's attributes to its indexes. The attributes stored are customizable, and vary depending on file type (MP3s have their ID3 info indexed, etc.). For example: somerwhere on my 120 GB disk I have a file named "code.txt" but it will take 10 minutes to find it by scanning the directory structure. Instead, I do a "SELECT Path FROM Files WHERE Filename='code.txt'" and WinFS comes up with the answer right away. If I have full text indexing, I could search for a specific phrase. Even more useful, you don't have to make playlists anymore. Just put all of your MP3s in the same directory, and when you want to hear all of your Bon Jovi, just perform the appropriate query. (Obviously you don't have to know SQL to make this useful.)
Some of this is already present in a more limited form in Unix. For example, I still can't figure out why Windows doesn't have something like the LOCATE database that is set up by default on my FreeBSD box. But WinFS will blow LOCATE away (update is real time, not daily, and it has much more than just the file name).
However, from what I've heard, so far it is a bit of a dog performance-wise. I hope it gets up to speed by ship time... At least it can be turned off if you can't take the perf hit.
Re:I'll reserve judgement (Score:4, Insightful)
Simple, none of the *nix filesystems out there support the Windows security model. So, even if they based it on an open source file system, they would have to extend it to cope with the NT permissions model.
However, that would provide lots of scope for slashdot rantings about embrace and extend...
Re:I'll reserve judgement (Score:5, Informative)
Plenty of filesystems are out there that do ACLs. FreeBSD's UFS2 does, I'm pretty sure SGI's XFS and IBM's JFS do also.
Re:I'll reserve judgement (Score:3, Interesting)
Re:I'll reserve judgement (Score:5, Interesting)
First, the async, means that not all reads and writes are syncronous which is an incredibly good thing for speed. Try putting your UFS/FFS filesystem into fully sync mode and then talk about performance, I'm willing to bet that UFS/FFS isn't sync by default either. However, calling fsync in the mail server (normally sendmail) in Linux will actually make it sync before returning. So no worries about RFC 1123. It's the SMTP server's job to ensure that it tells the filesystem, make sure the bits are on the disk. If Linux didn't have the ability to ensure bits where actually on the disk nobody would use it. That's why in Moshe Bar's series comparing Linux, FreeBSD, and OS X, he always said he recompiled after removing the fsync calls, otherwise you just compared how fast the disks in each system were.
For goodness sakes, Oracle ships on Linux, if Linux couldn't get the bits on the disk Oracle would have never ported to it. Not a chance. If Linux tells you the bits are on the disk, they are on the disk in my experience.
I've heard of people losing UFS filesystems while running them under NFS, or losing them due to any number of naferious VM race conditions. So what? Welcome to the real world, people lose data, buy a tape drive, make backups. Knew a guy who got really good at rebuilding filesystems by using dd on Solaris to recover email for customers.
Oh, and as I recall, async actually affects directories more then files, if you put the sync modifier on the filesystem, it only affects directories, not the file data for ext2/3. In ext3, directory writes are always journaled as I recall, so it shouldn't make much difference.
Now, from what I've heard of Linux and FreeBSD, is that until the late 2.2.X and early 2.4.X, there we're certain jobs Linux couldn't do like run big Usenet News services, or really disk intensive applications they the filesystem buffering was really hard to get right, and might cause corruption. The guy who ran a local ISP always said FreeBSD never did that when he was running the Usenet server on it, but Linux did with some regularity.
ext2 hasn't lot any data of mine in my 7 years of using Linux, including running a 120GB Oracle Database for the past 30 months. Ext3's never lost any data since I started using it. I've lost disk drives, I've lost mirrors, I've lost files, never lost a complete ext2 filesystem unless the disk just stopped spinning. Lost a couple of ReiserFS filesystems after installing RedHat7.0. Never tried most of the other journalling filesystems.
Kirby
This article is bullshit (Score:5, Troll)
Re:This article is bullshit (Score:5, Insightful)
Be proud mate. (Score:5, Funny)
Re:This article is bullshit (Score:5, Insightful)
Tom's Hardware must have degenerated while I wasn't looking. The articles used to be full of detail, two printed pages long per "digital article page" (what you see on the screen
Re:This article is bullshit (Score:5, Interesting)
It's really funny how they try to compare it with a file system, since they're just looking at NTFS with a layer giving the user an easier time to do certain things.
Re:This article is bullshit (Score:5, Informative)
Re:This article is bullshit (Score:5, Funny)
Mmmm Hmmm Sure (Score:4, Insightful)
Windows Explorer (Score:3, Funny)
Well, yes; we must preserve those system resources for the most recent incarnation of explorer.exe.
nothing new (Score:3, Interesting)
WinFS is on top of NTFS (Score:5, Interesting)
This was actually confirmed at WinHEC:
"Microsoft has scaled back its 'Big Bang', and its Future Storage initiative will build on, rather than supersede the NTFS file system, when the next version of Windows 'Longhorn' appears in 2005."
"WinFS is not a file system
NTFS will be the only supported file system in Longhorn, from a setup and deployment standpoint, though the OS will, of course, continue to support legacy file systems like FAT and FAT32 for dual-boot and upgrade purposes. The oft-misunderstood Windows Future Storage (WinFS), which will include technology from the "Yukon" release of SQL Server, is not a file system, Mark Myers told me. Instead, WinFS is a service that runs on top of--and requires--NTFS. "WinFS sits on top of NTFS," he said. "It sits on top of the file system. NTFS will be a requirement."
Interestingly, when WinFS is enabled, file letters are hidden from the end user, though they're still lurking there under the covers for compatibility with legacy applications. This reminds of when Microsoft added long file name (LFN) support in Windows 95, but kept using short (8.3) file names under the covers so 16-bit applications would still work. Expect this to be the first step toward the wholesale elimination of drive letters in a future Windows version."
Re:WinFS is on top of NTFS (Score:4, Interesting)
The article that i got some of that information from was from The Register: http://www.theregister.co.uk/content/4/30670.html [theregister.co.uk]
Also, there is more information here: http://www.winsupersite.com/showcase/longhorn_pre
Re:WinFS is on top of NTFS (Score:3, Insightful)
Re:WinFS is on top of NTFS (Score:4, Insightful)
I suspect people confusing DOS kernels and NT kernels is your pet peeve, and it bothers you so much you see it everywhere.
Maybe you should reread your post, you said that it 'still isn't'. - Implying that Windows still isn't an operating system, and with the previous reference to the DOS underpinnings you set yourself up for that interpretation.
Besides, now that we are on the subject, Win9x and its successors also were TRUE Operating Systems, as they handled ALL Input/Output of the OS and the only DOS it relied upon was for compatibility with legacy application or drivers. DOS was basically a 'boot loader' for the Win9x Operating System.
Go look up the terminology of Operting System, and you will find that Win9x easily fits the definition. This is why the Win3.1 box said 'Operating Environment' and Win95 box said 'Operating System'.
Maybe your pet peeve is that some people notice when you make an ass out of yourself when you are so quick to find a reason to bash Windows and yet you still have to find a way to respond.
Geesh...
NTFS Streams (Score:3, Informative)
I believe that NTFS supports multiple streams of data per filename much like HFS. However, Windows doesn't make the pervasive use of the capability that Apple did in OS 9. I think one reason is that pretty much anything can go in these multiple streams meaning that particular files are tied even more closely to particular appli
Re:NTFS Streams (Score:3, Informative)
Sort of.
OS9 suppored two streams, the resource and the data fork (this went way back before OS9, BTW.) The resource fork was usually used via a defined structure that allowed storage of elements like code segments, icons, text strings, etc. in a common format. (ResEdit rocked.) The data fork was freeform.
NTFS streams are arbitrary in definition and number. MS played with using streams for Office documents, as best I can
Maybe it'll work (Score:3, Insightful)
Also, I'm wondering if they'll finally give up on that stupid drive lettering. I don't see any reason why that ever had to exist, and now that they're doing an overhaul of the whole filesystem, it seems like a good oportunity to get rid of it. You'd think, since they try to be user friendly, that they would want to give devices and partitions names instead of letters. I do still see it in that screenshot, but things could change by the time it's released.
Terrible article (Score:5, Informative)
The closest it gets to examining the (possible!) new Windows filesystem is calling it a relational database, and going on for a bit about how paths (ie: directory structures) will be irrelevant. Oh, and yeah, the closest thing he found to the implementation was called "winfs.exe" and did nothing but produce errors.
The bulk of the article is a (poor) attempt at explaining filesystems in general, and FAT and NTFS in particular. However, it gets a number of things wrong and - at best - garbles a lot of things. If you already know what he's trying to say, you *may* be able to pick out truths, but if you don't you'll walk away with misinformation.
I would suggest instead perusing arstechnica.com and aceshardware.com. I don't know if they've done any filesystem stuff, but if they have it'll be of reasonable quality.
Oh so informative! (Score:5, Interesting)
"There has been much speculation"
Uh huh.
"Win FS is modeled on the file system of the coming SQL server"
Uh huh.
"In its latest build (M4), Longhorn contains few hints of the technology's imminent implementation."
Uh huh. You're saying you don't know anything, yeah, I'm getting that part.
"One of those is more than 20 MB in size and bears the name winfs.exe."
Neat.
"In the end, Win FS will probably emerge as an optional file system beside FAT and NTFS. It's also possible that Win FS will supersede its predecessors, however."
So in the end, it'll be A... but it is also possible it'll be B. I see.
"That would most likely produce problems for multi-boot systems"
An astounding feat of logic Mr. Spock!
This is the most uninformative article I've ever had the displeasure of reading on Tom's Hardware. These people know exactly nothing more about WinFS than any of the rest of us have heard in rumors and vague press releases.
My turn (Score:5, Insightful)
Looks like I was wrong - or, actually, right all along. Musta been a slow news day?
I'd be happy if they just let me use write caching (Score:5, Informative)
See http://forums.storagereview.net/index.php?act=ST&
Re:I'd be happy if they just let me use write cach (Score:3, Interesting)
Good idea (Score:4, Interesting)
I don't understand the concerns of the poster regarding performance (at least without evidence of truly dismal performance): no one is forcing anyone to use the FS if they are not satisfied with performance.
For most users, they main bottleneck in storage is their own organizational faculties. I used to be exasperated when users didn't know where they put their files, but once you get past the 100GB mark, it becomes very understandable.
Consider what most people use their massive storage for these days: videos, music, multimedia, games. Not only is this the kind of content that SHOULD be stored in a database, it's the kind of content that is ALREADY being handled through a database because the filesystem is not enough: people are using their media players, P2P programs and other software to handle their files, up to the point they rarely ever interact with the filesystem unless they lost a file.
For most users, the performance penalty is well worth the price.
For those for whom it is not, it doesn't take a genius to realize you can use more than a single filesystem, and perhaps rediscover the joy of proper partition organization: keep the OS and applications separate from your data, and you can use your highly efficient filesystem for the first and your metadata-loaded one for the second.
For such an in-depth article... (Score:5, Informative)
1) NT4 (certainly from SP3) allows you to make partition alterations without a reboot. Even 2K requires a reboot for alterations to the boot partition, however.
2) 2K doesn't dispense with the drive letter concept, despite the implication in the article that this is the case. That you can mount partitions under folders doesn't change this.
3) You can specify the cluster size when formatting a drive under NT4.
Also, has anyone actually come across a data centre that is making use of multi-hundred-TB NTFS volumes?
And, will Longhorn finally do away with the whole drive letter concept?
Better, not best (Score:5, Interesting)
However, the best solution is that used by EROS [eros-os.org], which is for the kernel not to provide a file system at all, but instead provide Orthogonal Persistence.
This is a much simpler layer for applications, since it doesn't require them to explicitly access the memory and disk separately. It is also much simpler to recover from because the entire state of the whole disk is always known to be coherent with itself at all given points in time, without an expensive journal.
In terms of performance - it beats the hell out of explicit disk access systems (Both conventional and database systems) because it performs big continuous reads and writes (that don't move the head much) rather than small writes on metadata and file data that forcibly jump the disk head around.
In EROS then, on top of the Orthogonal Persistence, you can create any arbitrary Objects you want easily - because they're just normal processes with normal memory. Conventional File Systems become useless and objects implemented by processes become a much better and more powerful alternative to files.
A relational database of the user objects is then much more powerful than a string hierarchy, but this is all the user's choice - and not hardcoded into a kernel.
Re:Better, not best (Score:5, Insightful)
Re:Better, not best (Score:3, Insightful)
No it doesn't - it just means that from the perspective of an application, there is no difference between malloc() and open(). You ask for some storage, you get it (unless something goes wrong, of course). You read and write to your bit of storage. At some point it may be in main memory, in some cases it will be on disk. The OS take
Truth be told... (Score:5, Interesting)
A filesystem based on a relational database will have some characteristics to which today's filesystems can only aspire:
1. ACID - In every way that the underlying database supports Atomicity, Consistency, Isolation, and Durability [techtarget.com], so now will the filesystem. In so far as the database is robust, the filesystem will be robust. Please spare me the comments about the supposed unreliability of SQL Server. Itâ(TM)s certainly more reliable than NTFS; which is itself very good.
2. As an offshoot of the above - Imagine multiple file updates to a filesystem which is transactional! Imagine that transaction failing and being able to just rollback the changes without touching every file in your program! Imagine being able to make file changes programmatically without having to worry about locking because the engine will do it for you (just handle any exceptions)! Yeah, you could do all that today if you like. But it takes extensive to make it happen.
3. Operational characteristics - We can run queries against databases. We can index them. We can cluster them. We can replicate them. We can access them easily from any development platform you can imagine. Now your filesystem is a database. The possibilities make me shiver!
4. Another offshoot from #3 - Security. Databases are inherently better than filesystems (IMNSHO) at enforcing security and enabling administration of security.
I only have reservations about one issue with the database as filesystem area: recovery. Currently, all good and low-tech filesystem recovery tools really are based on the filesystem allocation table sort of scheme. Obviously, databases usurp this category of tried and true tools. However, good tools already do exist that allow recovery of relational databases. Itâ(TM)s just a matter of getting easily accessible tools of this sort into the hands of professionals that need them. It's more of a training issue I guess, but it will still need addressing.
I know many people will have a knee jerk reaction to this idea, and I understand why. But I would encourage people to keep an open mind to this. While there will probably be some issues with the idea, there's so much more that could easily be done with a filesystem on top of a database than could be done easily (or well) with a traditional filesystem.
And for you hard-core naysayers out there, you have to ask yourself this: If this is such a bad idea, then why did Oracle provide this as a feature too? [oracle.com]
We beta test an atomic filesystem next month (Score:5, Informative)
and at www.namesys.com/v4/v4.html.
We will be adding support for semi-structured data querying in the next major release, assuming that we find funding for it. The semantics for it are described at www.namesys.com/whitepaper.html, which also explains why I don't think the relational model is effective for semi-structured data stores such as a general purpose filesystem is normally used for.
Best,
Hans
Good idea, but ONLY much further out... (Score:4, Interesting)
Mid-Term: FS finally works, and allows easier retrivial by relevance, author, source, etc. in ways that we can just dream of now. It's the kind of thing we didn't realize we needed until we had it...until it inevitably blows up as all MS products must do eventually. But when it works, we will be fairly happy to have it...especially end users, most of whom can't figure out a hierchical file system in the first place.
Far-Term: FS is finally able to use it's relational roots to distribute filesystems over multiple processors in an cluster or over a network. Such a system would support atomic, distributed file updates by threads of processes on differing processors (including HyperThreaded procs). Imagine a virtual filesystem that can span your whole-house network, with a single file system image...in WINDOWS.
So I guess my view is: painful in the near-term, but may be cool to have when they get it right.
Fast != Fast (Score:4, Interesting)
I recently installed a Win2K server that is blindingly fast at finding documents and such... but horridly slow at serving up portions of files, for things like legacy database programs. Three of the customer's applications started running at 1/4 speed.
It got so bad, even after all the "fix win2k speed" patches, that we re-introduced the 200MHz NT4 server to feed the database apps, and the dual-processor 2GHz system just serves up documents!
We don't need no steenkin efficiency (Score:4, Insightful)
They may have goals other than efficiency. Security, probably. But probably also security's perverted uncle, DRM. As DRM becomes more common, and we "pirates" look for more innovative ways to get around it, locking us out of our hard drives would seem to be a logical if not downright necessary step. It's pretty obvious that a lot of entities out there would benefit greatly from a model where we don't really OWN our computers, we just lease the right to use them. I mean, look, they're already floating the notion out there, at least for software and entertainment media, that we don't really OWN ANYTHING, and it's not that much of a stretch for that to become literal truth, aside from the hardware, which will be as impervious to meddling as they can possibly make it. (You decide who "THEY" might be, but I have a long list with a lot of familiar names on it.)
How technically difficult would it be for, say Microsoft, to "rent" out portions of your hard drive to various media and software providers, using a combination of hardware and software controls to assure those companies that you and I ABSOLUTELY CANNOT meddle with "their" product while we (temporarily) posess it? A database-driven file system provides exactly the access control and accountability that would be required to successfully implement something like that.
Good idea, bad implementation (Score:5, Interesting)
A directory tree is a very useful structure, at least to the software. Similar stuff is grouped together, and easily cached. It provides a very clean and simple way of putting data somewhere and getting it back later. This should not lightly cast aside.
So, you want to use a relational database to keep track of files? Go for it, but instead of keeping track of the files themselves, keep track of their paths. Let the filesystem do the efficient storage, and the database do the efficient lookups. The database can be made faster and smaller, the filesystems can remain as fast as they are, and the files are still there even if the database gets corrupted.
Put hooks wherever necessary to update the database when the filesystem changes. For example, put a database in the root of each filesystem. Use a stacked mount to mount that disk, so when interesting things happen, the kernel tells a userspace process that updates the database. Then, make some standard libraries that use the database. Make file browsers that can query it, but pass the path to programs. Make save dialogs that can also save metadata about the file, and open dialogs that can search for it. Use LUFS or FUSE to make directories that correspond to queries.
This is just as effective as what MS is doing [theregister.co.uk], but it's more efficient, it's more compatible, and it doesn't reinvent the wheel.
Anyone remember "On Location"? (Score:4, Insightful)
On my IIci it was perfectly fast. Faster than BeOS queries on a dual 603 box.
It took a little time to build your index, but keeping it up-to-date was pretty painless. Apple's developer CDs used to ship with On Location indices on them.
So where's the article? (Score:3, Informative)
Then at the end there's a few paragraphs with no real info about the FS at all. What meta info will be stored? How will the files be laid out on disk? Is there going to be journaling? How about file system integrity and recovery?
The only thing _really_ learned was that there exists a 20MB beta executable that doesn't do anything. What the frell? It's two years before Longhorn is to be released. (As if Microsoft is going to get it right the first time anyway).
Mod the whole article down. Way down.
New FS (Score:3, Interesting)
New FS = New corruption?
Rus
Once again... (Score:3, Insightful)
RTFA, /. !!! (Score:5, Insightful)
Right on the first page of the article, the "journalist" who wrote it describes disk storage as "memory". In the "Summary" of the article posted on every page, current file systems are described as wasting "memory". This reminds me of every-day users who confuse their computer running slowly (or literally telling them they don't have enough memory) with the need to delete files from the hard drive -- two completely separate things in most situations. This is all aside from the fact that the article doesn't actually tell you much that anybody who's used computers for more than six months doesn't already know. This guy sounds like some of the kids who come to me interviewing for I.T. positions thinking they've got a leg-up on everyone else because they've got some basic experience with PC's and Windows.
The bottom line is that the guy who wrote this article doesn't have any business writing tech articles without heavy supervision from someone who KNOWS tech, and I don't just mean someone who knows enough to rattle off performance numbers for CPU comparisons (read some other articles on the site). Lastly, Slashdot has no business posting amateurish and misinforming articles like this for the rest of us to waste our time on.
Backards(w) (Score:4, Funny)
Just a joke SQLiers, just a little joke. I know they are indispensible. Really. I believe you.
WinFS was originally a desktop usability thing (Score:5, Insightful)
Usability engineers looked at what users were doing in Windows and they saw that tons of people weren't using the filesystem - at least not directly. They were just putting everything on the desktop. If it was on the desktop, they could find it. They kept folder structures to a minimum and organized things visually (or not at all).
This posed a significant problem, so indexing and searching and abstracting the filesystem was one of the solutions. Instead of having to navigate a filesystem (hard for many users), you just type in what you're looking for and *poof* it appears. Not sure what you're looking for? Start describing it... *poof* it appears.
I'm not saying this is the right solution, but technology is not always about cluster size and performance - especially if the system isn't usable. It will be interesting to see how user friendly this WinFS thing is...
Microsoft Marketing 101 (Score:5, Funny)
2 years before release: Product will do everything for the majority of users.
1 year before release: Product will do many things for many users.
Release: Well it does something.
data recovery nightmare (Score:5, Insightful)
Yipee! (Score:3, Funny)
"Memory"? (Score:5, Insightful)
Been done already: PGFS (Score:3, Informative)
Designed mainly for version control. Could easily be modified for other purposes though.
http://www.linuxjournal.com/article.php?sid=138
Reservations? (Score:3, Funny)
A hugely conservative Slashdot reader? No way!
Re:For lots of files... (Score:3, Interesting)
I ran a Mac lab where a lot of the machines had 20meg drives, and that wasn't all that long ago. They used to sell a 10Mb drive (I forgot how ungodly it cost) for Apple ][s. Apple DOS 3.3 could only recognize floppy size chunks, about 140Kb IIRC, so the thing had to be partitioned into along the lines of 80 pseudo-drives. I never saw one physically, but I can imagine what the P.I.T.A. that was.
Re:For lots of files... (Score:3, Insightful)
I have no idea what "most" is supposed to mean in this context, but systems like NTFS, ext3, ReiserFS, XFS, and JFS were definitely designed for those kinds of applications.
A relational database setup should do wonders for file search and access.
Relational databases aren't designed for millions of records with potentially huge BLOBs in them. In fact, the way Oracle and DB2 handle that kind of data is by putting i
Re:For lots of files... (Score:5, Interesting)
One word: FAT. You are making three assumptions here. The first is that the underlying implementation is capable of supporting near-infinite extension without degradation. Invalid for FAT, valid for the FS types mentioned in the grandparent, and the reason for what I said. The second is that the file system will be used as a hierarchy, which is invalid for most end users. The third is a combination of the first and second, being that the file system extends without unreasonable degradation to a vasst number of files in a single directory, and performing operations (esp. searches) on them quickly. This is invalid for all of these file systems, because of how they store metadata.
Again, you're assumiung you, a technically savvy user. End users don't behave like this. By and large they use meaningful file names in a single directory. If you're looking for a document someone else did, it will be in their single directory, not in a common folder for documents relating to that topic. If you don't know who worked on the document, you need to do a broad search based on keywords.
Which shows how little you've thought about the implementation of this system. You only have to make a change if the file metadata changes. In many file systems you already have to write that change in a different location to changes to the file itself (if you don't, your metadata search time goes out the window). If your "locate" database is a relational database, making a change has trivial overhead.
Actually, this isn't what I was meaning. I was referring to the relationship between the data in the FS and in the locate database (or any other metadata search database), and indicating that WinFS (in theory) takes out the step of building a separate database by using the database as the "index" of the file system. Unfortunately in this incarnation of WinFS (the current implementation) MS will not be implementing it quite in that fashion.
But to answer your point ... Win32 systems have had file change notification in their APIs from day 1 (NT 3.1 / Win95 + have FindFirstChangeNotification; NT 3.51 + have ReadDirectoryChangesW).
And that's pretty much what MS is doing by converging a tradition file system with a metadata view.
Of course, WinFS was intended for client operation systems, not servers. And while NTFS could still be improved, it doesn't make a lot of sense to do so: most high data volume applications store their data in structured files, and don't require much from the file system in any place where performance could be signficantly improved.
Re:Again? (Score:4, Interesting)
Fat 16 was limited to 2GB partitions, that affects the normal users.
Now, the fact that if the database file system works the way I imagine it would it will be a bad thing for the normal user's more tech savy friend.
I have spent years explaining to relatives that the same file name in 2 places is 2 different files.
Now I must spend time explaining that if you brows, documents, taxes, and edit file blah it will effect important stuff, file blah and what not.
People will be confused by this I believe. And I also think the the techies saying it is stupid would benifit from this greatly, I know I would love to organize things with tons of logical ways to browse there.
But I am not some overpaid market researcher so what do I know.
Re:Again? (Score:3, Funny)
Difference between FAT32 and NTFS (Score:3, Interesting)
NTFS has tons more advantage than FATxx. The official list can be found here [microsoft.com]. Granted, this benefits the corporate user more than home user.
At the very least, NTFS offers a quicker way to hide porn than FAT32.
Re:I can almost guarantee you.... (Score:5, Funny)
How about Clippy? "I see you're looking for your work files. You're fscked."
Re:Pretty thorough article (Score:4, Interesting)
http://www.cs.wisc.edu/~bolo/shipyard/hpfs.html
Re:Lindus says "Me-Too"!!! (Score:3)
On the other hand I wouldn't mind seeing a standard library for this purpose if one doesn't already exist. I know Gnome has a VFS library. I'm not sure how much of i
Re:I like the idea (Score:3, Insightful)
Another poster mentioned phone numbers. An even better analogy is email addresses. The concept of email addresses and typing them into a TO field isn't a very good concept, and the proof of it is looking at how many people use address books to handle email addresses. Why type in an email address when you can just type in (or select