I can see a very serious problem with this. Just how can you tell
the difference between a hardware failure, buffer overrun, ignored
return value from malloc, (any other of a number of ways you can
generate a seg fault through programming errors), and bumping into
one of these over-committed memory areas?
It would seem prudent to at least track the amount of virtual memory
that has been committed and not allow that figure to exceed the amount
available (say the sum of the phys ram and swap space). In fact, I
thought this is what was being done.
BSD does something similar to this (though not all that well) in that
all memory allocations have their swap space allocated at request time.
Any request for which swap space cannot be assigned is failed. This
is efficient speed-wise, but very inneficient in terms of resources,
as it does not allow for a system with less swap space than RAM to use
all of it's RAM.
It would seem to me to be fairly simple and inexpensive to simply keep
track of the current total commitment for each process, and a sum for
the system, and fail any allocation that pushes the system into an
overcommited state. This is not foolproof of course, eg if swap space
is removed from the system, then you could end up overcommitted, but
it seems to me that we would want a system that is running out of virtual
memory to fail gracefully, by failing allocation requests, rather than
having it fail in some other fashion, say by getting seg faults in
processes that are accessing memory that has been allocated to them.
thanks,
Brett
>Date: Tue, 18 Feb 1997 23:17:29 -0500
>From: "Theodore Y. Ts'o" <tytso@MIT.EDU>
> Date: Tue, 18 Feb 1997 14:27:44 -0700 (MST)
> From: Jody Matsushima <matsushj@unidata.ucar.edu>
>
> > We are trying to do some software development on our machine
> > that runs RedHat Linux (Intel - Pentium). One of the
> > developers has asked me if there is any kind of fix for a
> > problem with malloc. The problem seems to be that when you
> > use the malloc command, you can allocate memory that is not
> > available to allocate. Has anybody else reported this? If
> > you need more information, I can get it to you.
>This is not a bug; it's a feature. Linux does not do hard allocation of
>memory, because its not efficient. A very large amount of memory could
>potentially be used is in fact never used. For example, when a process
>fork()s, the data pages are not copied right away, but marked
>copy-on-write, and only copied when they are modified. If they are
>never modified, the extra data page is never needed. If however Linux
>were to guarantee that every single memory access was always guaranteed
>to succeed, it would have to reserve room for all of those data pages,
>even if 99.99999% of the cases they would never be used.
>In general, this is very, very, very rarely a problem. It would perhaps
>useful for you to comment on *why* you think this is a problem. What is
>it specifically that you are trying to do?
> - Ted