Re: Found the commit that causes the OOMs

[Minchan Kim]

On Tue, 30 Jun 2009 20:57:47 +0100
David Howells <dhowells@xxxxxxxxxx> wrote:

> Minchan Kim <minchan.kim@xxxxxxxxx> wrote:
> 
> > David. Doesn't it happen OOM if you revert my patch, still?
> 
> It does happen, and indeed happens in v2.6.30, but requires two adjacent runs
> of msgctl11 to trigger, rather than usually triggering on the first run.  If
> you interpolate the rest of LTP between the iterations, it doesn't seem to
> happen at all on v2.6.30.  My guess is that with the rest of LTP interpolated,
> there's either enough time for some cleanup or something triggers a cleanup
> (the swapfile tests perhaps?).
> 
> > Befor I go to the trip, I made debugging patch in a hurry.  Mel and I
> > suspect to put the wrong page in lru list.
> > 
> > This patch's goal is that print page's detail on active anon lru when it
> > happen OOM.  Maybe you could expand your log buffer size.
> 
> Do you mean to expand the dmesg buffer?  That's probably unnecessary: I capture
> the kernel log over a serial port into a file on another machine.
> 
> > Could you show me the information with OOM, please ?
> 
> Attached.  It's compressed as there was rather a lot.
> 
> David
> ---

Hi, David. 

Sorry for late response.

I looked over your captured data when I got home but I didn't find any problem
in lru page moving scheme.
As Wu, Kosaki and Rik discussed, I think this issue is also related to process fork bomb. 

When I tested msgctl11 in my machine with 2.6.31-rc1, I found that:

2.6.31-rc1		
real	0m38.628s
user	0m10.589s
sys	1m12.613s

vmstat

allocstall 3196

2.6.31-rc1-revert-mypatch

real	1m17.396s
user	0m11.193s
sys	4m3.803s 

vmstat

allocstall 584

Sometimes I got OOM, sometime not in with 2.6.31-rc1.

Anyway, the current kernel's test took a rather short time than my reverted patch. 
In addition, the current kernel has small allocstall(direct reclaim)

As you know, my patch was just to remove calling shrink_active_list in case of no swap.
shrink_active_list function is a big cost function.
The old shrink_active_list could throttle to fork processes by chance. 
But by removing that function with my patch, we have a high probability to make process fork bomb. Wu, KOSAKI and Rik, does it make sense? 

So I think you were just lucky with a unnecessary routine.
Anyway, AFAIK, Rik is making throttling page reclaim. 
I think it can solve your problem. 

-- 
Kind regards,
Minchan Kim
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/