Re: Avoiding *mandatory* overcommit...

From: Linda Walsh (law@sgi.com)
Date: Mon Apr 03 2000 - 12:13:17 EST


Pavel Machek wrote:
> > You still haven't answered the question. After startup, why is the kernel
> > allocating memory if not by user or user-proxy (program) request?
>
> Because network "ping" packet came?
>
> Because user plugged in 10 more usb mice?

---
	The former wouldn't cause an alloc.  The transient ping buffer already exists -- 
either that or the ping command would appear to never hear from the remote system.

The 2nd is allocation performed by user action. The mice in question would never get initialized or be usable. > > oldest->P2->P3 ... ->I->I'. > > Which fails if you have just 5 pages total! No matter how much swap > space you have, with 2 memory pages total, nothing can help you and > you are deadlocked. That was my point. > Pavel --- So you somehow managed to get the kernel loaded. Now you only have 5 or 2 pages left over total to run user programs in?? Uh...you are really not going to have much performance, but the 2 page question solvable.

Assume a MOVS instruction spans pages 0 and 1 of your Virtual address space (V0,V1). The source data spans Virtual address space V2 and V3 with the target spanning V4 and V5. Assume you only have 2 pages of Physical memory (P0 and P1). Replacement policy is LRU (Least Recently Used). No pages are in memory at the start, but P1 was LRU. Show the Paging activity for a 32-bit word at offset 0x3fe in both source and destination pages:

T P0 P1 ------------------ 0 X X 1 V0 X 2 V0 V1 ;instruction loaded and decoded -- it stays in the CPU now ** 3 V2 V1 ;load first source page 4 V2 V4 ;load first dest. page and copy 1st byte 5 V2 V4 ;copy 2nd byte (pages are already in memory 6 V3 V4 ;load 2nd source page 7 V3 V5 ;load 2nd dest page and copy 3rd byte 8 V3 V5 ;4th byte is copied.

** -- once an instruction is decoded it is internal to an x86 CPU, so no more instruction fetches need to be performed. Even if they had to be refetched on each cycle, however, the problem is still solvable -- just more steps.

The paging issue is solvable for physical pages > 0 and no overcommit.

If you have overcommited, and your target area had been just recently extended by brk by 4 bytes, then 2 things can occur. 1) There is still open virtual memory in which case there is no problem -- the virtual page is mapped into VM and life goes on or 2) there is no room in VM for the page to be mapped. At which point the program fails. If the program was 'VMWare' (which often will be the largest memory hog in the system, not running at UID==root), and quite possibly having just been started, then your 'guest' OS will be killed by a very unclean shutdown with possible corruption of the internal hard disk.

This is the beauty of not-overcommitting. If you have run with sufficient stack to start, then, while your machine may horribly thrash in an OOM situation, it will eventually complete -- it won't randomly fail. Failures will be at discrete points in user programs (like an exit due to insufficient resources) vs. random points where the OS has killed a program.

-l

-- Linda A Walsh | Trust Technology, Core Linux, SGI law@sgi.com | Voice: (650) 933-5338

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Fri Apr 07 2000 - 21:00:10 EST