> Just for clarification I'm not for obfuscating the kernel by having it
> handle "resources" which are easily handled in user space.
> On Fri, 09 Jul 1999, Alexander_Maryanchick%STORACTIVE@storactive.com
> wrote: > There were some questions about my message.
>> I took the liberty to combine them to reduce email traffic.
>> My apologies if you are offended.
> I'll follow your lead.
Looks like you have VERY wrong idea about "user-space solution". There are few.
Microsoft way: filesystem in file. But it's NOT discussed here ! Here was
dicussed other implemementation: use directory as albod and files in this
directory as forks. THIS version need changes in mv.
>> >> 2. Why in the kernel?
>> > Why does the kernel necessarily help?
>> The problem of userspace realizations is that
>> a. thay can not store resources together with data.
> Someone already said, yes they can. Please give me an example where you
> can't. What's your definition of a resource? Information about the
> data? That's easily and commonly stored with the data.
Not if you have 100Mb of data and 100Mb of resources and want to drop half of
them :-) Not easily nor fast :-/ Due unability to add information in the middle
of file you are forced to implement filesystem-in-file ...
>> b. thay must index files by full names.
> Like the other respondant, I don't understand what you mean.
So I am :-(
>> >> a. *Huge* perfomance losses (10 - 1000%)
>> > This point is not defended very well:
>> - scanning /etc/magic for 1000 files in directory is *seconds*.
> for i in 1 2 3 4 5 6 7 8 9 a; do
> for j in 1 2 3 4 5 6 7 8 9 a; do
> for k in 1 2 3 4 5 6 7 8 9 a; do
> /usr/games/fortune >$i$j$k
> done
> done
> done
> time file * /dev/null
> 0.82user 0.06system 0:00.92elapsed 95%CPU (0avgtext+0avgdata 0maxresident)k
> 0inputs+0outputs (1146major+176minor)pagefaults 0swaps
> Doesn't look like "seconds" to me. Looks like less than a second.
> This also doesn't mean the 'file' could not be optimized.
No, it can not. That is `file' CAN be optimized, but in ANY case `file' will
need MANY times more operations to determine file type then extensions parsing
or metadata opening...
>> - parsing extensions (DOS sort of metadata) is milliseconds.
> This isn't the "Unix way." I don't think anyone will argue with this.
> I don't think this is an acceptable alternative.
Unfortunatelly it is THE ONLY [barely] acceptable alternative :-(
>> - following @albod/type is microseconds.
> I'd like to see some evidence. How is the kernel going to store this
> resource data?
Just like any other files :-)
> How much data do you personally think is going to be put in resources
> if they were available?
No matter. May be 10bytes, may be 10GiB -- reiserfs was developed exactly
to solve this problem... @albod/type will be just few bytes, while
@albod/audiocomment will be few MiBs...
> I think it could be rather large and end up being many times the size of
> the data itself.
Sometimes -- yes.
> Does the kernel store the resource in the same "file" as the data?
Not, of course :-) This will lead to the same problems as userspace solution...
> If so, how is this different than a user space program storing it just like
> they do now? If not, where oh where would the kernel store it?
Somewhere on disk. No matter. In separate file, in inode (if metadata is small
enough) or somewhere else. Why you are not interested where two files from the
same directory are stored on disk ?
> Will there be a percentage of every filesystem reserved for resources?
What for ?
> How much will this be? What happens when that space runs out? Does the
> kernel dynamically allocate this storage for resources? Doesn't sound
> like this would be "microseconds" to me.
reiserfs will try to keep files for one albod close on disk and will prefetch
them. So in most cases it'll be microseconds. Not always. In unlucky cases it'll
be milliseconds...
> And how to represent type? Is the kernel supposed to have some
> registry that maps a type number to a name? Are you saying store
> actual text as the type? Who makes the determination of what the
> standard number of name for a given type is?
IANA, as usual :-)
> Is there going to be a standards body created that approves new types?
There are alredy is (see RFC2048).
> Are we going to shove this on the IANA?
We need register "vendors" (KDE, GNOME, etc) with IANA, yes. Then you can use
`application/vnd.gnome.gnumeric.spreadshit' just fine without additional
registration in IANA just fine. gnome part must be registered in IANA,
gnumeric in some GNOME's database and spreadshit in some Gnumeric's database.
You'll need something like this anyway...
> Sounds rediculous, but we are talking about puting
> resources in files that will be transferred over the Internet. There
> has to be some standard format and protocol for transferring such
> files. Otherwise, they resources will only be on your Linux box and
> you could never share data with others.
Someone must be first :-) Why not Linux ? On legacy OS's you'll be forced to
use old-fashioned approach (filesystem-in-file)...
> A simple suggestion - take this through the standards process before
> implementing it in Linux.
You need TWO WORKING INTEROPERABLE INDEPENDENT implementations anyway :-)
BEFORE standard will be accepted :-)
> Or, at least do a complete write up so that we can see exactly how you are
> proposing to do what you want with resources.
Reiser is doing it just now AFAIK.
> It's not like you have to get MY approval. I haven't as of
> yet contributed one line as a kernel developer. But from an outsider's
> viewpoint I think there would be MUCH less controversy and much more
> agreement if there was a complete white paper write up of what is being
> proposed.
There will be code :-)
> This is not some simple addition to the kernel that someone could
> simply send a patch in for and be done with it. It needs to be
> architected before implementation. Obfuscating the kernel - Just Say
> No.
> If it is eventually integrated into the kernel PLEASE make it an option
> so that people can turn it off.
What for ? How you'll read ZIP floppies with albods without support in kernel ?
If you dislike programs with albods usage -- do not use them...
>> >> - create cache database (it will duplicate file names
>> >> => waste disk space)
>> > Hmm I already give 128MB to my swapspace.
>> > Why not not use a little of that?
>> To give 50 Mb for 'file types' cache only???
>> IMHO it's much uglier than fixing findutils.
> Well, as mentioned earlier where the hell is the kernel going to store
> the resources. If it's in traditional files then there is no need for
> the kernel to get involved.
If fact you need at least new flag for marking directories as albods...
Not sore if something above this is REALLY needed. Most programs for Linux
available in source form so may be simplest possible way is just fixing them.
> If it's in some special area of the filesystem, then I think you're going
> to need much more than 50Mb for all this resource crap given the size of
> today's hard drives.
>> >> b. Security holes
>> >> ACLs in tarball?
>> > This is a good point. However, it may be solvable with a much simpler
>> > kernel mechanism.
>> a. What is simpler: one powerful mechanism or 100 simple mechanisms?
> You seem to want to force everyone to use one complex mechanism that
> may not fit everyone's needs. I'd much rather see app developers have
> the freedom to do what they want.
They can not use unimplemented mechanism :-/
> Sure, they could ignore the resource fork stuff even if it is included
> in the kernel and do things in the "traditional" way. But, just having that
> "one powerful mechanism" will unduly effect their performance.
reiserfs is [almost always] faster then ext2fs :-)
>> b. What simpler kernel mechanism?
>> If you store somewhere additional data - you must rewrite mv, aren't you?
> Earlier you said that userspace programs could not handle this concept.
> Now you are complaining that mv would have to be rewritten if you
> DON'T (as in DO NOT) put resource fork capabilities in the kernel?
> It's the exact opposite. If you have userspace programs handle the
> compund document, resource forks, whatever you want to call them, then
> the file will be a "standard file" as far as mv, tar, ftp, etc are
> concerned.
No it will not. The whole idea is to use directories as compound documents.
Without kernel help you can not hide directories/files difference from mv :-((
I'm not sure if it's RERALLY needed though.
> If you put compound document, resource forks, or whatever
> you want to call them facilities into the kernel THEN you would have to
> rewrite mv, tar, ftp, etc OR come up with totally new transfer and
> storage protocols AND have to get everyone else in the world who
> communicates on the Internet to agree to your new standards AND write
> new programs to handle these new protocols to replace mv, tar, ftp, etc.
Not at all. You just need some magic to create one file from albod :-)
I'm not sure if such complexity is belond to kernel. Or even to userspace lib !
>>> I honestly can't believe that this is going on for so long. I don't
>> > think applications are already too slow. What applications?
>> Personally I am not satisfied with file managers perfomance.
>> 'DOS Navigator' is 1000 times faster than kfm determines file types.
>> (DOS uses file extensions)
> I don't know 'DOS Navigator.' Is it a graphical application? kfm is.
> Does it run under a structured window system like X or MSWindows or
> does it have it's own custom made GUI? Does it support drag and drop
> and all the other things that kfm does?
`DOS Navigator' has drag and drop, custom-made structured windowing system.
Graphic application... Not (AFAIK, anyway). I know that graphical applications
are slower but THAT slower :-/
> For some reason I don't think this is a "fair" comparison. If you want
> to use a DOS like file manager I think there are a handful available
> that run quite well in an XTerm - try mc.
I use mc, thank you. But I'm can not userstood why we must keep kfm and other
oprograms slow, unrealiable and unflexible just to keep "holy cow" untouched :-/
>> > I don't see any problems with the current way of doing things.
>> You are lucky! But some of us use sluggish file managers, etc. :-(
> Then pick a different file manager! Don't mess around with the kernel
> just because you don't like the standard file manager for your window
> system. Or lend a hand to make kfm faster. I do believe that the
> source is available :-)
He tries to do it MUCH faster :-) And there ARE way to make it MUCH faster.
You'll just need small modification to kernel.
>> > parsing conf files is slow? Where'd anybody get that idea from.
>> I did not benchmarks: I only yawned and listened to the HDD.
> May be you need a new hard drive :-)
If you can say where I'll be able to buy new hard driver 1000 times faster
then my current one (and NOT 1000 times more expensive :-) I'll shut up.
>> > How often do you parse conf files?
>> a. Initialization of * program.
>> For comparison: BeOS starts (with full GUI) in 6 seconds.
> Then use BeOS
> The only program that takes a little too long for my taste is Netscape
> Navigator. I'd attribute this to feature creep and a horrible code
> base more than because the kernel doesn't handle resource forks. I
> understand that Mozilla is turning out to be quite fast indeed!
Last time I tried Mozilla (M7) it was WAY SLOWER then Netscape (it's hard to
believe, I know). But it's unrelated problem, of course...
>> b. loading xpms.
> Er, this may be a not-good file format. Why not store the data in a
> binary format? I don't think that your dislike of the pixmap format
> standard for X is a good reason to mess with the kernel.
> Write a new standard.
>> c. logging dynamic data.
> Don't have a clue what you mean here.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/