Re: Memory intensive processes

Mark H. Wood (mwood@mhw.oit.iupui.edu)
Thu, 12 Dec 1996 12:11:17 -0500 (EST)


On Wed, 11 Dec 1996, William Burrow wrote:

>
> On Wed, 11 Dec 1996, Richard B. Johnson wrote:
>
> > It is apparent that you didn't understand the well-proven concept. The
> > idea is that, unless actually written, the zero-filled memory doesn't have
> > to exist at all! This is called "demand-zero" paging and has been used
> > in VAXen forever.

But don't the (possibly hundreds of thousands of) PTEs still have to
exist? They can all point to the same page, but the hardware still wants
them to be there. A virtual address beyond the end of the current page
table is invalid. Those PTEs add up to a lot of memory. That's one
reason for the larger (in some models, *much* larger) pages in the AXP chip.

> System Kennunglinux (whatever) wrote me and stated that mmap()ing
> /dev/zero does cause Linux to use a zero page. This answer is not
> portable, however (ie algorithms from other systems that do not have
> /dev/zero will not have this feature, and software from Linux won't port
> easily to other systems that don't have /dev/zero).
>
> > Note that I don't advocate copying Digital's mistakes. However we can
> > learn a lot about performance by understanding how these things work. One
> > of the problems with "Digital security" is that if you extend a file, the
> > data is written to zero before you are allowed to access that file. That
> > takes a lot of CPU cycles and hurts performance.
>
> Hmm, I thought I read in a DEC system manual that files could contain
> junk from previous files. Perhaps this zeroing a file was a paranoia option?

By default a volume is INITIALIZEd to enable "high-water marking": you
cannot allocate disk blocks to a file and the read them without first
having written them. You can disable this (SET VOLUME
/NOHIGHWATER_MARKING) if you don't like this. You can also set
erase-on-delete (write erasure bit-patterns on deleted blocks) at the
volume and file level, or DSE a whole piece of medium when you
INITIALIZE it. /ERASE is the more costly of the two and is *not* the
default.

> > page. On VAXen, it knows that it doesn't even have to be written to the swap
> > device because it can always re-read the program file!
>
> Linux can do this as well. Executables are never written to swap from
> what I understand. eg Turn off swap (swapoff -a) and fill RAM. Sleeping
> processes will show up as swapped and activating them will cause disk
> activity.

I seem to recall that VMS charges a process for its page tables as well
as user pages. Does Linux also do this? (I know, I know, UTSL....)

Mark H. Wood, Lead System Programmer MWOOD@INDYVAX.IUPUI.EDU
Those who will not learn from history are doomed to reimplement it.