Re: NFS read performance ugliness

Zlatko Calusic (Zlatko.Calusic@CARNet.hr)
03 Nov 1998 15:31:30 +0100


alan@lxorguk.ukuu.org.uk (Alan Cox) writes:

> > > Use 2.0.x. The 2.1.x NFS code needs several major modifications before it will
> > > perform acceptably against Sun equipment - notably write gathering support.
> > >
> > > Alan
> > >
> > Before I proceed, I must mention that 2.0.x kernels are mostly even
> > worse than 2.1.x (I tried many times in last year or two). Biggest
> > write speed I got is 50KB/sec, and I remember that at misterious
> > occasions I could achieve cca 200KB/sec with 2.1.x (in some older
> > revision).
>
> With 8K blocks you should be seeing well over 50K/second, some of it
> is hidden by the asynchronous writeback in 2.1.x. The real problem
> is that NFS is synchronous write, and Sun machines use 8K blocks on
> the disk (the BSD derived UFS/vm stuff requires page sized blocks), A half
> page synchronous write is incredibly machine expensive.
>
> Because 2.1.x uses the page cache to generate requests it never generates
> requests over 4K in size, even if r/wsize is set higher (unless you are
> running on an 8K page host...)
>

Huh, this at least explains why changing [r|w]size options doesn't
have any impact on performance.

>
> > sign is offset. So we can easily see that Sun stacks 32768 bytes worth
> > of data. Everything is cool.
>
> Yep

Even though it says in the man page that when using v2 NFS protocol,
it defaults to 8192 bytes. :)

man mount_nfs (on Sun):

...
rsize=n Set the read buffer size to n bytes.
The default value is 32768 when using
Version 3 of the NFS protocol. The
default can be negotiated down if the
server prefers a smaller transfer size.
When using Version 2, the default value
is 8192.
...

Same for wsize.

>
> > Strange is that packet order is reversed (in time) if I read dumps
>
> [itsa performance trick]

Nice. I think I understand now, when I meditate on the topic.

>
> > correctly. First last fragment is received, then middle one, and in
> > the end, fragment that contains NFS (IP,UDP?) headers. Could this be a
> > bug in Linux NFS implementation?
>
> No
>
> > Also, I notice, no matter what rsize, wsize options I use, dump always
> > looks like that. Something like rsize,wsize=4096 is hardcoded and
> > can't be changed?
>
> Yes, thats one of the remaining NFS problems, its constrained at 4K because
> it doesnt write gather
>

Now, it looks like this is a big problem, since it is not already
implemented (and we could use it, definitely). Is it so?

Pardon my ignorance, I'll try (just a one) stupid question.
How come we can't copy few pages worth of data to one network buffer?

-- 
Posted by Zlatko Calusic           E-mail: <Zlatko.Calusic@CARNet.hr>
---------------------------------------------------------------------
	     It really bothers me when people cut me o...

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/