Re: [PATCH 00/22] Cleanup and optimise the page allocator V7

From: Zhang, Yanmin
Date: Mon Apr 27 2009 - 03:58:27 EST

On Wed, 2009-04-22 at 14:53 +0100, Mel Gorman wrote:
> Here is V7 of the cleanup and optimisation of the page allocator and
> it should be ready for wider testing. Please consider a possibility for
> merging as a Pass 1 at making the page allocator faster. Other passes will
> occur later when this one has had a bit of exercise. This patchset is based
> on mmotm-2009-04-17 and I've tested it successfully on a small number of
> machines.
We ran some performance benchmarks against V7 patch on top of 2.6.30-rc3.
It seems some counters in kernel are incorrect after we run some ffsb (disk I/O benchmark)
and swap-cp (a simple swap memory testing by cp on tmpfs). Free memory is bigger than
total memory.

[ymzhang@lkp-st02-x8664 ~]$ uname -a
Linux lkp-st02-x8664 2.6.30-rc3-mgpage #1 SMP Thu Apr 23 16:09:43 CST 2009 x86_64 x86_64 x86_64 GNU/Linux
[ymzhang@lkp-st02-x8664 ~]$ free
total used free shared buffers cached
Mem: 8166564 18014398497625640 20022908 0 2364424 247224
-/+ buffers/cache: 18014398495013992 22634556
Swap: 0 0 0
[ymzhang@lkp-st02-x8664 ~]$ cat /proc/meminfo
MemTotal: 8166564 kB
MemFree: 20022916 kB
Buffers: 2364424 kB
Cached: 247224 kB
SwapCached: 0 kB
Active: 2414520 kB
Inactive: 206168 kB
Active(anon): 4316 kB
Inactive(anon): 4932 kB
Active(file): 2410204 kB
Inactive(file): 201236 kB



[ymzhang@lkp-ne01 ~]$ uname -a
Linux lkp-ne01 2.6.30-rc3-mgpage #1 SMP Thu Apr 23 15:04:27 CST 2009 x86_64 x86_64 x86_64 GNU/Linux
[ymzhang@lkp-ne01 ~]$ free
total used free shared buffers cached
Mem: 6116356 18014398509340432 6257908 0 609804 1053512
-/+ buffers/cache: 18014398507677116 7921224
Swap: 15631204 0 15631204
[ymzhang@lkp-ne01 ~]$ cat /proc/meminfo
MemTotal: 6116356 kB
MemFree: 6257948 kB
Buffers: 609804 kB
Cached: 1053512 kB
SwapCached: 0 kB
Active: 723152 kB



Or a simple kernel source cp/rm/cp:
[ymzhang@lkp-ne01 linux-2.6.30-rc3_melgorman]$ uname -a
Linux lkp-ne01 2.6.30-rc3-mgpage #1 SMP Thu Apr 23 15:04:27 CST 2009 x86_64 x86_64 x86_64 GNU/Linux
[ymzhang@lkp-ne01 linux-2.6.30-rc3_melgorman]$ free
total used free shared buffers cached
Mem: 6116356 1309940 4806416 0 82184 1259072
-/+ buffers/cache: 18014398509450668 6147672
Swap: 15631204 0 15631204
[ymzhang@lkp-ne01 linux-2.6.30-rc3_melgorman]$ cat /proc/meminfo
MemTotal: 6116356 kB
MemFree: 4806724 kB
Buffers: 82184 kB
Cached: 1259072 kB
SwapCached: 0 kB
Active: 477396 kB
Inactive: 872388 kB
Active(anon): 8704 kB
Inactive(anon): 0 kB
Active(file): 468692 kB
Inactive(file): 872388 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 15631204 kB
SwapFree: 15631204 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 8632 kB
Mapped: 4140 kB
Slab: 174504 kB
SReclaimable: 154976 kB


>
> The performance improvements are in a wide range depending on the exact
> machine but the results I've seen so fair are approximately;
>
> kernbench: 0 to 0.12% (elapsed time)
> 0.49% to 3.20% (sys time)
> aim9: -4% to 30% (for page_test and brk_test)
> tbench: -1% to 4%
> hackbench: -2.5% to 3.45% (mostly within the noise though)
> netperf-udp -1.34% to 4.06% (varies between machines a bit)
> netperf-tcp -0.44% to 5.22% (varies between machines a bit)
>
> I haven't sysbench figures at hand, but previously they were within the -0.5%
> to 2% range.
>
> On netperf, the client and server were bound to opposite number CPUs to
> maximise the problems with cache line bouncing of the struct pages so I
> expect different people to report different results for netperf depending
> on their exact machine and how they ran the test (different machines, same
> cpus client/server, shared cache but two threads client/server, different
> socket client/server etc).
>
> I also measured the vmlinux sizes for a single x86-based config with
> CONFIG_DEBUG_INFO enabled but not CONFIG_DEBUG_VM. The core of the .config
> is based on the Debian Lenny kernel config so I expect it to be reasonably
> typical.
>
> text kernel
> 3355726 mmotm-20090417
> 3355718 0001-Replace-__alloc_pages_internal-with-__alloc_pages_.patch
> 3355622 0002-Do-not-sanity-check-order-in-the-fast-path.patch
> 3355574 0003-Do-not-check-NUMA-node-ID-when-the-caller-knows-the.patch
> 3355574 0004-Check-only-once-if-the-zonelist-is-suitable-for-the.patch
> 3355526 0005-Break-up-the-allocator-entry-point-into-fast-and-slo.patch
> 3355420 0006-Move-check-for-disabled-anti-fragmentation-out-of-fa.patch
> 3355452 0007-Calculate-the-preferred-zone-for-allocation-only-onc.patch
> 3355452 0008-Calculate-the-migratetype-for-allocation-only-once.patch
> 3355436 0009-Calculate-the-alloc_flags-for-allocation-only-once.patch
> 3355436 0010-Remove-a-branch-by-assuming-__GFP_HIGH-ALLOC_HIGH.patch
> 3355420 0011-Inline-__rmqueue_smallest.patch
> 3355420 0012-Inline-buffered_rmqueue.patch
> 3355420 0013-Inline-__rmqueue_fallback.patch
> 3355404 0014-Do-not-call-get_pageblock_migratetype-more-than-ne.patch
> 3355300 0015-Do-not-disable-interrupts-in-free_page_mlock.patch
> 3355300 0016-Do-not-setup-zonelist-cache-when-there-is-only-one-n.patch
> 3355188 0017-Do-not-check-for-compound-pages-during-the-page-allo.patch
> 3355161 0018-Use-allocation-flags-as-an-index-to-the-zone-waterma.patch
> 3355129 0019-Update-NR_FREE_PAGES-only-as-necessary.patch
> 3355129 0020-Get-the-pageblock-migratetype-without-disabling-inte.patch
> 3355129 0021-Use-a-pre-calculated-value-instead-of-num_online_nod.patch
>
> Some patches were dropped in this revision because while I believe they
> improved performance, they also increase the text size so they need to
> be revisited in isolation to show they actually help things and by how
> much. Other than that, the biggest changes were cleaning up accidental
> functional changes identified by Kosaki Motohiro. Massive credit to him for a
> very defailed review of V6, to Christoph Lameter who reviewed earlier versions
> quite heavily and Pekka who kicked through V6 in quite a lot of detail.
>
> arch/ia64/hp/common/sba_iommu.c | 2
> arch/ia64/kernel/mca.c | 3
> arch/ia64/kernel/uncached.c | 3
> arch/ia64/sn/pci/pci_dma.c | 3
> arch/powerpc/platforms/cell/ras.c | 2
> arch/x86/kvm/vmx.c | 2
> drivers/misc/sgi-gru/grufile.c | 2
> drivers/misc/sgi-xp/xpc_uv.c | 2
> include/linux/gfp.h | 27 -
> include/linux/mm.h | 1
> include/linux/mmzone.h | 11
> include/linux/nodemask.h | 15 -
> kernel/profile.c | 8
> mm/filemap.c | 2
> mm/hugetlb.c | 8
> mm/internal.h | 11
> mm/mempolicy.c | 2
> mm/migrate.c | 2
> mm/page_alloc.c | 555 ++++++++++++++++++++++++--------------
> mm/slab.c | 11
> mm/slob.c | 4
> mm/slub.c | 2
> net/sunrpc/svc.c | 2
> 23 files changed, 424 insertions(+), 256 deletions(-)
>
> Changes since V6
> o Remove unintentional functional changes when splitting into fast and slow paths
> o Drop patch 7 for zonelist filtering as it modified when zlc_setup() is called
> for the wrong reasons. The patch that avoids calling it for non-NUMA machines is
> still there which has the bulk of the saving. cpusets is relatively small
> o Drop an unnecessary check for in_interrupt() in gfp_to_alloc_flags()
> o Clarify comment on __GFP_HIGH == ALLOC_HIGH
> o Redefine the watermark mask to be expessed in terms of ALLOC_MARK_NOWATERMARK
> o Use BUILD_BUG_ON for checking __GFP_HIGH == ALLOC_HIGH
> o Drop some patches that were not reducing text sizes as expected
> o Remove numa_platform from slab
>
> Change since V5
> o Rebase to mmotm-2009-04-17
>
> Changes since V4
> o Drop the more controversial patches for now and focus on the "obvious win"
> material.
> o Add reviewed-by notes
> o Fix changelog entry to say __rmqueue_fallback instead __rmqueue
> o Add unlikely() for the clearMlocked check
> o Change where PGFREE is accounted in free_hot_cold_page() to have symmetry
> with __free_pages_ok()
> o Convert num_online_nodes() to use a static value so that callers do
> not have to be individually updated
> o Rebase to mmotm-2003-03-13
>
> Changes since V3
> o Drop the more controversial patches for now and focus on the "obvious win"
> material
> o Add reviewed-by notes
> o Fix changelog entry to say __rmqueue_fallback instead __rmqueue
> o Add unlikely() for the clearMlocked check
> o Change where PGFREE is accounted in free_hot_cold_page() to have symmetry
> with __free_pages_ok()
>
> Changes since V2
> o Remove brances by treating watermark flags as array indices
> o Remove branch by assuming __GFP_HIGH == ALLOC_HIGH
> o Do not check for compound on every page free
> o Remove branch by always ensuring the migratetype is known on free
> o Simplify buffered_rmqueue further
> o Reintroduce improved version of batched bulk free of pcp pages
> o Use allocation flags as an index to zone watermarks
> o Work out __GFP_COLD only once
> o Reduce the number of times zone stats are updated
> o Do not dump reserve pages back into the allocator. Instead treat them
> as MOVABLE so that MIGRATE_RESERVE gets used on the max-order-overlapped
> boundaries without causing trouble
> o Allow pages up to PAGE_ALLOC_COSTLY_ORDER to use the per-cpu allocator.
> order-1 allocations are frequently enough in particular to justify this
> o Rearrange inlining such that the hot-path is inlined but not in a way
> that increases the text size of the page allocator
> o Make the check for needing additional zonelist filtering due to NUMA
> or cpusets as light as possible
> o Do not destroy compound pages going to the PCP lists
> o Delay the merging of buddies until a high-order allocation needs them
> or anti-fragmentation is being forced to fallback
>
> Changes since V1
> o Remove the ifdef CONFIG_CPUSETS from inside get_page_from_freelist()
> o Use non-lock bit operations for clearing the mlock flag
> o Factor out alloc_flags calculation so it is only done once (Peter)
> o Make gfp.h a bit prettier and clear-cut (Peter)
> o Instead of deleting a debugging check, replace page_count() in the
> free path with a version that does not check for compound pages (Nick)
> o Drop the alteration for hot/cold page freeing until we know if it
> helps or not
>

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/