Re: maximum memory limit

From: Jamie Lokier (lk@tantalophile.demon.co.uk)
Date: Mon Jul 03 2000 - 08:32:13 EST


Stephen C. Tweedie wrote:
> libc can easily mmap /dev/zero in chunks of multiple MB at a time if
> it wants to, and can then dole out that memory as if it was a huge
> piece of the heap without kernel involvement.

Several allocators already do that (including GCC's), but Glibc doesn't
AFAIK.

> You cannot have an arbitrarily growable heap, AND an arbitrarily
> growable stack, AND have the kernel correctly guess where to place
> mmaps.
>
> One answer may be to start mmaps down near the heap boundary, and to
> teach glibc to be more willing to use mmap() for even small mallocs.
> That may break custom malloc libraries but should give the best
> results for code which uses the standard glibc malloc: it doesn't
> artificially restrain the stack or the mmap area. Anything
> relying directly on [s]brk will be affected, of course.

There are lots of custom malloc libraries. If you're going to teach
Glibc something anyway, why not add a new mmap flag?

One flag I think would be quite useful is MAP_NOCLOBBER|MAP_FIXED: there
are times when I'd like to be able to _try_ mapping something at a fixed
address, but fail if there is something mapped there already. Currently
it's necessary to parse /proc/self/maps, and that's far from thread
safe.

The obvious use is for pre-relocated shared libraries, which can run at
any address but will load faster and share more pages if loaded at a
specific address. (Especially if they're non-PIC).

As a natural extension, a map flag which says "try to map at the
supplied address, but if there is an object there search for a big
enough hole above the supplied address" would simultaneously be useful
for malloc optimisations like you're suggesting, and pre-relocated
shared libraries. That would be MAP_NOCLOBBER without MAP_FIXED.

-- Jamie

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Fri Jul 07 2000 - 21:00:12 EST