Re: File systems are semantically impoverished compared to database

Lou Grinzo (lgrinzo@stny.lrun.com)
Fri, 25 Jun 1999 10:55:07 -0400


Let me try again, since Albert and I seem to be talking around
each other.

Albert Cahalan writes:

From: "Albert D. Cahalan" <acahalan@cs.uml.edu>
Date: Thu, 24 Jun 1999 22:52:09 -0400 (EDT)
Subject: Re: File systems are semantically impoverished compared to database

Lou Grinzo writes:

> namely the desired level of functionality. As I see it, there are
> two interesting levels of feature we're talking about:
>
> First, the ability to mash a bunch of file streams into a single
> disk file, ala MS's COM structured storage. This is clearly
> something that can be done wholly in user space, and would

No, not like that at all. Structured storage adds an extra allocation
and namespace layer above the filesystem. This works in userspace,
but the performance is poor.

You can get the performance today by using directories, but that
destroys usability. Documents need to appear as files to the user,
so they must appear as files to traditional POSIX software.
It isn't enough to patch a file manager. You'd have to bloat
everything in the whole system, perhaps losing the performance!
The kernel can examine inodes and other structures directly,
so it can avoid much of the bloat.

This is precisely the meta-point I was attempting to make: Albert is making
a lot of assumptions about how this should or should not work. I agree
with him that compound files must look like single files to traditional
software. But I don't think that means that the support should be in
the kernel as opposed to user space. What are we assuming about
how this support will be used? How will the end user see his/her
system change when this support is added? If the goal is to provide
nothing more than a compound storage facility for applications, then
a relatively simple user-mode solution is strongly preferable, and the
support can be contained in a shared library. If the compound files
are to be treated through various interfaces (system calls, UI) as
directories under at least some conditions, then this is a whole other
issue, and much more complicated. This is why I originally mentioned
making albods self-identifying. If the system is to do something special
with them in response to certain sys calls, then it has to distinguish
them from normal binary files.

My point is NOT to argue for or against any particular side so much as
it is to ask just what we're trying to accomplish and what the benefits
to the users and application developers will be. I'm NOT implying that
people are running around coding with no idea of what they're doing.
But this is a big design issue that Linux will likely have to live with
for a long time. I hope that for this kind of change, the developers
involved will take the time to do a requirements analysis.

IMO, compound files can be a very useful thing, but introducing them
brings up some tough questions, like: To what extent does the system
make these look like directories to programs and the user? How
extensive should the support be for them? (IMO, if you add compound
files, of any flavor, you immediately add the requirement to manipulate them
with traditional commands. Whether this is done automagically by sinking
the support in the kernel, or by keeping it in user space modifying all the
individual commands is another issue entirely.) How much of a
performance penalty and code complexity penalty, for both Linux and
applications, is reasonable for code that does and doesn't use this
feature? Is it acceptable to support this feature only on native Linux
FS's, or does it have to work on every currently supported FS? Now
that Linux has NTFS support, how does this interact with NTFS's
file streams? For that matter, should Linux now support COM
structured storage files from any Windows partition in the same way
as its own albods?

In short, I think this is a perfect example of something that should be
designed from the outside in, considering first and foremost what changes
are desirable to the user's and application's view of the system.

Lou

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/