> OK, I agree this opens up a lot of new possibilities. Even more
> if you include a size_t length parameter. I was thinking it was
> going to be a clone of NTs web-server-syscall, which as far as I
> know is limited to sending entire files verbatim onto a socket.
sendfile is most useful when you can append or prepend data constructed from
an iovec.
> > fd-in - file descriptor to disk file (? is this needed as a restriction)
> > fd-out - file descriptor to any object
> > options - eg, close on send completion, sync/async. etc.
>
> All these options aren't pretty. I would hope you could do
> without them. If sendfile blocks until it is complete, there
> seems little point in having a close-on-send, and if you look
> at Deans website, you can see that the question of closing a
> TCP/IP socket properly is non-trivial, and probably not for
> kernel-space-only.
In the case of web servers and proxy caches, I don't think close on send is
worth while, because it defeats the purpose and the (possibly very great)
win experienced with HTTP/1.1.
I am perhaps overstating pipelining in HTTP/1.1, because I've measured this
on active proxies and 40% of all requests come from single connections, but
this value is likely to reduce as browsers get smarter.
(As is, by default, IE 4 and stuff don't do HTTP/1.1 though proxies because
the piece of shit MS-Proxy 2 doesn't support this, which means the rest of
the world gets penalized because M$ can't write decent code as per usual).
Dean knows about 50 times what I do in this area, so its probably best he
jumps in and points and where I'm wrong.
> Possibly. But according to Squid's own doc, the primary problems
> in Squid performance are
>
> 1) Not enough memory
> 2) Too slow disks
I think this is for squid-1.1. squid-1.2 is a bit different and probably
would benefit from sendfile. It already supports async io (via threads).
In reality I do find squid-1.2 really memory greedy, but memory is cheap
nowdays so put in 1-2GB and live with it.
> Yes, but is it the bulk of web content by server-CPU-time?
Again, Dean would know better here...
but on loaded servers I've tested, plenty of time is spent copying images
(because they are large) from disk out to the network.
> There are so many other likely bottlenecks, like disk bandwidth (sendfile
> doesn't help here as far as I can see)
I keep hearing disk bandwidth is an issue, but I've yet to see it. Disk seek
times are a problem, but disk IO usually isn't.
Anyhow, sendfile could help here by eliminating the costly copy to/from
userspace, which is the real limiting factor if you have really fast disks,
really fast network and are trying to stream the data out.
> CPU speed for cgi, bandwidth to the net, bandwidth on your Ethernet, etc.
> Linux/Apache can easily saturate a 10mbits Ethernet already, how many
> people actually have a faster connection to the net?
IntrAnet's can and do use 100mb/s, server can have multiple interfaces. I
used to have a machine setting on three T3s.
> A T3 is tens of thousands of dollars a month. Compared with this,
> a couple of Linux boxes, which give you redundancy as well are
> nothing.
More boxes - more to go wrong, more to administer, etc.
And as I said, people have fast internal networks, and there are
applications like NFS which could benefit (although to a much lesser extent
with knfsd).
> That's cool, as long as we make something that the application
> writers can use. If we depart from Posix, we'd better get it right.
POSIX doesn't specific lots of APIs we already have. sendfile is available
on other hardware, and if we can support it cheaply, why not? (Bloat
arguments off the list please).
-Chris
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu