A critical part of input packet processing is finding (if any) the
socket assosciated with the connection on the local machine based upon
the packet address/port combination we see.
Currently the Linux TCP has a one entry cache, unless you are only
talking to one host over one set of ports that doesn't get hit that
often. Thus we often go to af_inet.c:get_sock() This hashes into our
256 entry per-protocol sock_array, which is currently hashed on the
foreign port number for TCP.
I wrote a little "hack" to compare this hashing scheme to the one
FreeBSD's stack uses. I didn't like the results.
Essentially the program takes as input a file which is in the format
of /proc/net/tcp (easy as hell to parse ;-), creates a hash table and
then prints some statistics about how well things hashed. It has a
verbose mode as well that logs each hash table insertion during the
run. The test program can be found at:
vger.rutgers.edu:/pub/linux/Net/socklist.c
It's simple, it's stupid, but it got me the information I wanted to
see in this case.
FreeBSD hashes all connections based upon destination address/port and
the local port.
I took a random sample while the mailer was lightly active on
vger.rutgers.edu with a bit over 60 active TCP sockets open. The
sample data is available in the same place as socklist.c as
"vger.tcplist". This is a completely arbitrary sampling.
For this sample data socklist told me:
TCP sockets added 67, hash table size 256
Linux STATISTICS: empty[229] single[21] collision[6] deepest_collision[22]
TCP sockets added 67, hash table size 128
FreeBSD STATISTICS: empty[76] single[39] collision[13] deepest_collision[4]
There were a few (say 3 or 4) incoming ftp connections, and 2 or 3 web
connections at this point as well.
I imagine a highly web bound site would give different distributions,
as would an active news server (udp connections primarily right?) or
heavy ftp server.
Either way I'm not too happy with those numbers and I'm going to hack
on making it better.
I'd like to get more test data, in particular from heavy web sites, so
if you know someone who runs a heavy web or ftp Linux server, ask them
for /proc/net/tcp snapshots during high activity and send them to me,
I'd certainly appreciate it.
Of course there are many other things than just this socket lookup
that are important, one thing at a time. But the next thing I do want
to look at is new connection costs, and then the effects of TIME_WAIT
TCP timers being processed.
---------------------------------------------////
Yow! 11.26 MB/s remote host TCP bandwidth & ////
199 usec remote TCP latency over 100Mb/s ////
ethernet. Beat that! ////
-----------------------------------------////__________ o
David S. Miller, davem@caip.rutgers.edu /_____________/ / // /_/ ><