Re: Memory overcommitting

Brian E. Gallew (geek+@CMU.EDU)
Wed, 19 Feb 1997 17:20:02 -0500 (EST)


> >I can see a very serious problem with this. Just how can you tell
> >the difference between a hardware failure, buffer overrun, ignored
> >return value from malloc, (any other of a number of ways you can
> >generate a seg fault through programming errors), and bumping into
> >one of these over-committed memory areas?

I hate to point this out, but not only has this issue been discussed to death
before ( a couple of times if memory serves), but *industry* believes that the
allocation method that linux uses is neither a bad idea or hazardous.

To hit on the specifics:
1) hardware failure: This is covered in system logs/hardware. True, on a
linux box there generally isn't much hardware support for error
detection/correction/prediction, nevertheless, it is always assumed that the
hardware is the least likely point of failure for unproven systems.
2) buffer overrun/ignored malloc error/other programming holes: While it's
*nice* for the OS to protect you from your mistakes, that isn't it's job. It's
job is to protect every other process from mistakes in one. (Isn't it nice
when people tell you the answer is just not to make any mistakes??? :)

To give yet another OS example (and support my original statement), DYNIX/ptx
has no less than 3 memory allocation modes, depending on whether performance or
stability is more important. Pre-allocation, pre-commission, and optimistic.

Pre-allocation: The old berkeley thing. No process is allowed to execute
without sufficient swap to contain it. The image is written to swap before
execution begins. Safest, slowest, and poorest use of resources.

Pre-commission: No process is allowed to execute unless there is sufficient VM.
Swap is only written upon demand. Evidently IRIX and Solaris support this
mode. *Almost* as safe as pre-allocation, lots faster, and better use of
resources.

Optimistic: The Linux Way (tm). It is assumed that processes will often
overallocate. Least safe, fastest, and best usage of resources.

For those of you who dislike The Linux Way (tm), grab the source, write you
*own* allocators, and provides patches for compile-time choosing among methods.
If you aren't willing to write it yourself, then sit down and be quiet.

=====================================================================
| Please do not shoot at the thermonuclear weapons! -- Deacon |
=====================================================================
| Finger geek@andrew.cmu.edu for my public key. |
=====================================================================