Michael S (already5chosen@yahoo.com) on 2/3/09 wrote:

>Are the arguments against usage of Alternate Data Streams the same as Linus' arguments against xattrs?

Yes

NTFS alternate data streams are almost full blooded files. That is they can have arbitrary size, data and honor the usual file read/write operations.

What matters is they are different enough to break tools not written with them in mind. Tools to view the file system like the dir command or the Explorer desktop don't show them. Most tools to copy, backup or archive don't know about them. They also break when the data is copied to other filesystems without the feature. There are an awful lot of network effects in basic things like file storage.

Why should everyone need all new tools and learn about new kinds of files? The files we have work fine. They have names, they store data, they honor things like read, write and seek. NTFS alternate data streams, like most extended attribute schemes, amount to feature creep. The filesystem people are looking for something new and sexy they can add to distinguish their work.

Simple is everything in this business. The Unix style filesystem is a big success because of its simplicity as a hierarchical name space with byte stream data attached. Even the classic extra features like links see only intermittent use while the core directory tree structure is widely used in applications and other operating systems.

There is room for new filesystems to advance without breaking with the old. Making large numbers of small files efficient opens new uses. All those extended attributes can and should be files as that is what they are, names with data attached. ReiserFS took a run at this though the end was tragic. I don't know enough of its technical merits to say how close it came but the project did make it a central goal. There is a lot of work in new filesystems going on these days and it appears they have a lot of respect for scaling, both up in size and down to efficiency on numerous small files. It will be exciting to see how well they work in practice.

I think it would be a nice feature if every file could function as a directory and every directory could hold data like a file. If a program understands the name as holding data it does open/read/write and gets data. If it wants to attach extra attributes it does a chdir into the file as a directory and makes a new named file inside. Copy is recursive and displaying the file in a directory listing or GUI file manager shows it has more named files inside.

Still, even this simple idea has big network effect problems. Only a new filesystem could do that and it might even affect how the operating system understands things. Maybe the OS deep inside understands files and directories as safely different, good bet actually. Even if it were all nice and smooth on one system it would all go poof once one copies the data to another filesystem which doesn't have the feature. There you could store the data by making a conventional name for the default data associated with directories. Having done so we get back to Go and realize we could have just done that from the beginning and done the entire thing in user space. When given a name of something to open looking for data try to open it, if it is a file great, if it is a directory open the file with [conventional default name] inside it.

It all comes back to programs which don't know about each other still managing to cooperate enough to not damage each others data. You would need a bunch of conventions for them to follow. The usual editor habit of updating a file by writing an entirely new one, unlink old and rename new would make a mess. All of the usual copy and delete operations would have to default to being recursive.

The only way any of this happens is if something widely used does it. A desktop environment could maybe get enough traction to seed the growth, maybe. What is there to attach to existing files which is worth the trouble? Icons and thumbnails? It isn't security data as none of this is secure.

What data do we need to attach to files? Without a compelling answer to that none of this matters.