Buffer cache hints

Richard Gooch (rgooch@atnf.csiro.au)
Fri, 6 Sep 1996 15:00:43 +1000

Hi, all. I have a process which reads a large file (nearly half the
size of RAM) and swap-copies it into a malloced array of the same
size. This results in a lot of disc head movement as pages are read
from the file and written to the swap partition (when I can afford it,
I'll get that second SCSI disc and move swap to that). This behaviour
appears to be due to the kernels attempt to keep bits of the file I
have read in the buffer cache, so malloced pages tend to be thrown
out. Since I know that my process will read the file sequentially and
then close it, this seems to be sub-optimal. A few solutions come to
mind:

1) give malloced pages a higher priority than disc buffer pages. This
has the disadvantage of crippling disc access performance for
frequently used pages (like libc :-)

2) providing a hint to the kernel that I will be accessing a file in
a sequential fashion, and the kernel then reclaims the oldest pages in
the file in preference to writing malloced pages to swap. Since I may
either use read() or mmap() to read the file, this scheme should work
for both those cases

3) mlocking the malloced pages, but that's very unfriendly (and good
luck if my file happens to be bigger than RAM)

I like (2) the best (this does not appear to be implemented), but
perhaps there are problems with this, or someone can see a better way
to do it. The current MM scheme does not work well with big data.

Regards,

Richard....