Re: Memory overcommitting (was Re: http://www.redhat.com/redhat/)

bofh@snoopy.virtual.net.au
Thu, 20 Feb 97 19:29:44 +1000


>I can see a very serious problem with this. Just how can you tell the
>difference between a hardware failure, buffer overrun, ignored return
>value from malloc, (any other of a number of ways you can generate a seg
>fault through programming errors), and bumping into one of these
>over-committed memory areas?

Have a look at the amount of free swap space, if it's close to zero...

>It would seem prudent to at least track the amount of virtual memory that
>has been committed and not allow that figure to exceed the amount
>available (say the sum of the phys ram and swap space). In fact, I
>thought this is what was being done.

>BSD does something similar to this (though not all that well) in that all
>memory allocations have their swap space allocated at request time. Any
>request for which swap space cannot be assigned is failed. This is
>efficient speed-wise, but very inneficient in terms of resources, as it
>does not allow for a system with less swap space than RAM to use all of
>it's RAM.

This would be OK as a tunable in /proc/sys, but I doubt that it's desirable for everyone. Most programs allocate lots of memory that they don't need, and requiring physical disk space for all that unused memory will just cause problems on small systems.

>It would seem to me to be fairly simple and inexpensive to simply keep
>track of the current total commitment for each process, and a sum for the
>system, and fail any allocation that pushes the system into an
>overcommited state. This is not foolproof of course, eg if swap space is
>removed from the system, then you could end up overcommitted, but it seems

I think that you would just have to decline the swapoff in that situation. Otherwise processes would immidiately start failing after the swapoff (imagine that you're logged in as root and after doing swapoff you haven't got enough memory available to run "ps", "kill", or "swapon", imagine that all memory is being used by daemon programs and it won't be freed by people logging out)...

Russell Coker