Re: Kswapd in 3.2.0-rc5 is a CPU hog

[Dave Chinner]

On Tue, Dec 27, 2011 at 01:56:58PM +0900, KAMEZAWA Hiroyuki wrote:
> On Tue, 27 Dec 2011 12:57:31 +0900
> Minchan Kim <minchan@xxxxxxxxxx> wrote:
> > The scenario I think is as follows,
> > 
> > 1. dd comsumes memory in NORMAL zone
> > 2. dd enter direct reclaim and wakeup kswapd
> > 3. kswapd reclaims some memory in NORMAL zone until it reclaims high wamrk
> > 4. schedule
> > 5. dd consumes memory again in NORMAL zone
> > 6. kswapd fail to reclaim memory by high watermark due to 5.
> > 7. loop again, goto 3.
> > 
> > The point is speed between reclaim VS memory consumption.
> > So kswapd cannot reach a point which enough pages are in NORMAL zone.
> > 
> > > 
> > > BTW. I'm sorry if I miss something ...Why only kswapd reclaims memory
> > > while 'dd' operation ? (no direct relcaim by dd.)
> > > Is this log record cpu hog after 'dd' ?
> > 
> > If above scenario is right, dd couldn't enter direct reclaim to reclaim memory.
> > 
> 
> I think you're right. IIUC, kswapd's behavior is what we usually see.
> 
> Hmm, if I understand correctly,
> 
>  - dd's speed down is caused by kswapd's cpu consumption.
>  - kswapd's cpu consumption is enlarged by shrink_slab() (by perf)
>  - kswapd can't stop because NORMAL zone is small.
>  - memory reclaim speed is enough because dd can't get enough cpu.
> 
> I wonder reducing to call shrink_slab() may be a help but I'm not sure
> where lock conention comes from...

There is no lock contention. It's simply the overhead of grabbing a
passive reference to every superblock in the machine 3 times every
100uS.

FWIW, I don't think kswapd should be polling the shrinkers this
often when there are no/very few pages available to be freed from
the slab caches. There are many more shrinkers now than there were
in the past, so the overhead of polling all shrinkers very quickly
is significant.

FWIW, when we move to locality aware shrinkers, the polling overhead
is going to be even higher, so either the VM needs to call
shrink_slab less aggressively, or shrink_slab() needs to have some
method of reducing shrinker polling frequency.....

Cheers,

Dave.
-- 
Dave Chinner
david@xxxxxxxxxxxxx
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/