Re: dynamic hash table allocation

Chuck Lever (cel@monkey.org)
Thu, 17 Jun 1999 21:45:39 -0400 (EDT)


On Fri, 18 Jun 1999, Andrea Arcangeli wrote:
> I also rewrote from scratch the buffer dynamic allocation. With my code
> you can choose at compile time (not a config option but a #define in
> buffer.c) the number of buffers you want to take hashed in the same bucket
> supposing the hashfn is perfect and all the memory is alloced in buffers.

thanks for the feedback! i have trouble with your method for a few
reasons.

first, as dave miller pointed out, i wanted to make all the various hash
init routines look like each other so that eventually someone could split
out the common code, as he suggested. i'm not sure i'm ready to do that,
because i would like to be sure the new logic actually works, first, and
is really generalizable.

so, i don't see any advantage to building special logic into the buffer
hash, especially because it is dwindling in importance. the dentry and
page caches are much more significant to performance.

next, *theoretically* a hash function could approach a scenario where all
the buckets were equally filled, but in practice i don't think that will
ever occur, especially with small tables like these. there will always
be a normal distribution of bucket sizes; that's the nature of statistical
hash functions. the distribution becomes narrow as the size of the table
approaches infinity -- but i don't think we need to worry about that case!
:)

third, why would you want to *vary* the optimal bucket size? that's
always going to be one or two. the hash table's size is probably most
dependent on how much memory is on the machine, anyway.

lastly, i think the kernel would be in deep cookies if *all* available
memory was allocated in the buffer cache. making the table so large is
overkill, IMHO. besides, how can you tell what buffer sizes the system
will need? 2K? 1K? 512 bytes? that will have direct bearing on how
many buffers there are.

so, in summary, i don't think any exact calculation will always generate
an optimal buffer hash table size. guessing is all we can do.

- Chuck Lever

--
corporate:	<chuckl@netscape.com>
personal:	<chucklever@netscape.net> or <cel@monkey.org>

The Linux Scalability project: http://www.citi.umich.edu/projects/linux-scalability/

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/