Re: [PATCH 3/4] Count the number of times zone_reclaim() scans andfails

From: Mel Gorman
Date: Wed Jun 10 2009 - 06:40:38 EST


On Wed, Jun 10, 2009 at 10:10:28AM +0800, Wu Fengguang wrote:
> On Wed, Jun 10, 2009 at 01:01:43AM +0800, Mel Gorman wrote:
> > On NUMA machines, the administrator can configure zone_reclaim_mode that
> > is a more targetted form of direct reclaim. On machines with large NUMA
> > distances for example, a zone_reclaim_mode defaults to 1 meaning that clean
> > unmapped pages will be reclaimed if the zone watermarks are not being met.
> >
> > There is a heuristic that determines if the scan is worthwhile but it is
> > possible that the heuristic will fail and the CPU gets tied up scanning
> > uselessly. Detecting the situation requires some guesswork and experimentation
> > so this patch adds a counter "zreclaim_failed" to /proc/vmstat. If during
> > high CPU utilisation this counter is increasing rapidly, then the resolution
> > to the problem may be to set /proc/sys/vm/zone_reclaim_mode to 0.
> >
> > Signed-off-by: Mel Gorman <mel@xxxxxxxxx>
> > ---
> > include/linux/vmstat.h | 3 +++
> > mm/vmscan.c | 4 ++++
> > mm/vmstat.c | 3 +++
> > 3 files changed, 10 insertions(+), 0 deletions(-)
> >
> > diff --git a/include/linux/vmstat.h b/include/linux/vmstat.h
> > index ff4696c..416f748 100644
> > --- a/include/linux/vmstat.h
> > +++ b/include/linux/vmstat.h
> > @@ -36,6 +36,9 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,
> > FOR_ALL_ZONES(PGSTEAL),
> > FOR_ALL_ZONES(PGSCAN_KSWAPD),
> > FOR_ALL_ZONES(PGSCAN_DIRECT),
> > +#ifdef CONFIG_NUMA
> > + PGSCAN_ZONERECLAIM_FAILED,
> > +#endif
>
> I'd rather to refine the zone accounting (ie. mapped tmpfs pages)
> so that we know whether a zone scan is going to be fruitless. Then
> we can get rid of the remedy patches 3 and 4.
>

This patch is not a remedy patch as such. tmpfs might not be the only
trigger case for thie zone_reclaim() excessive scan problem. In the
event it's occuring, we want to be able to pinpoint better why we are
spinning at 100% CPU. It's to reduce the debug time if/when this problem
is encountered.

On the mapped tmpfs page accounting, I mentioned the problems I see with
this in another mail. It would alter a number of paths, particularly the
page cache add/remove paths to maintain the counters we need to avoid
tmpfs in this slow NUMA-specific path. I'm hoping that can be avoided.

> We don't have to worry about swap cache pages accounted as file pages.
> Since there are no double accounting in NR_FILE_PAGES for tmpfs pages.
>
> We don't have to worry about MLOCKED pages, because they may defeat
> the estimation temporarily, but after one or several more zone scans,
> MLOCKED pages will go to the unevictable list, hence this cause of
> zone reclaim failure won't be persistent.
>
> Any more known accounting holes?
>

Not that I'm aware of but if/when they show up, I'd like to be able to
detect the situation easily, hence this patch.

> Thanks,
> Fengguang
>
> > PGINODESTEAL, SLABS_SCANNED, KSWAPD_STEAL, KSWAPD_INODESTEAL,
> > PAGEOUTRUN, ALLOCSTALL, PGROTATED,
> > #ifdef CONFIG_HUGETLB_PAGE
> > diff --git a/mm/vmscan.c b/mm/vmscan.c
> > index e862fc9..8be4582 100644
> > --- a/mm/vmscan.c
> > +++ b/mm/vmscan.c
> > @@ -2489,6 +2489,10 @@ int zone_reclaim(struct zone *zone, gfp_t gfp_mask, unsigned int order)
> > ret = __zone_reclaim(zone, gfp_mask, order);
> > zone_clear_flag(zone, ZONE_RECLAIM_LOCKED);
> >
> > + if (!ret) {
> > + count_vm_events(PGSCAN_ZONERECLAIM_FAILED, 1);
> > + }
> > +
> > return ret;
> > }
> > #endif
> > diff --git a/mm/vmstat.c b/mm/vmstat.c
> > index 1e3aa81..02677d1 100644
> > --- a/mm/vmstat.c
> > +++ b/mm/vmstat.c
> > @@ -673,6 +673,9 @@ static const char * const vmstat_text[] = {
> > TEXTS_FOR_ZONES("pgscan_kswapd")
> > TEXTS_FOR_ZONES("pgscan_direct")
> >
> > +#ifdef CONFIG_NUMA
> > + "zreclaim_failed",
> > +#endif
> > "pginodesteal",
> > "slabs_scanned",
> > "kswapd_steal",
> > --
> > 1.5.6.5
>

--
Mel Gorman
Part-time Phd Student Linux Technology Center
University of Limerick IBM Dublin Software Lab
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/