Transparent Hugepage Support #31
From: Andrea Arcangeli
Date: Thu Oct 14 2010 - 20:44:55 EST
http://www.linux-kvm.org/wiki/images/9/9e/2010-forum-thp.pdf
http://git.kernel.org/?p=linux/kernel/git/andrea/aa.git;a=shortlog
first: git clone git://git.kernel.org/pub/scm/linux/kernel/git/andrea/aa.git
or first: git clone --reference linux-2.6 git://git.kernel.org/pub/scm/linux/kernel/git/andrea/aa.git
later: git fetch; git checkout -f origin/master
The tree is rebased and git pull won't work.
http://www.kernel.org/pub/linux/kernel/people/andrea/patches/v2.6/2.6.36-rc/7transparent_hugepage-31/
http://www.kernel.org/pub/linux/kernel/people/andrea/patches/v2.6/2.6.36-rc7/transparent_hugepage-31.gz
Diff #30 -> #31:
b/ksmd-khugepaged-freeze | 35 ++++-
Sleep in wait_event_freezable.
b/pte_alloc_trans_splitting | 24 +++-
Fix build problem with INTEL_TXT=y. (problem found by Naoya Horiguchi
<n-horiguchi@xxxxxxxxxxxxx>)
b/transhuge_mapcount_debug | 34 +++++
Add two VM_BUG_ON to verify invariants.
b/transparent-hugepage-per-node-meminfo | 54 +++++++++
Provide per-node meminfo from David Rientjes <rientjes@xxxxxxxxxx>.
b/transparent_hugepage | 19 +--
Fix memleak in case of fork vs split_huge_page race.
Move a VM_BUG_ON inside a spinlock to avoid false positives.
b/transparent_hugepage_vmstat-anon_vma-chain | 12 --
Fix /proc/vmstat when TRANSPARENT_HUGEPAGE=n (problem found by Anton
Blanchard <anton@xxxxxxxxx>).
b/vma_adjust_trans_huge | 161 +++++++++++++++++++++++++++
A race condition could occur if qemu-kvm extends the vma when at least
one ptes before vma-extension wasn't null, then khugepaged could
collapse an hugepage before the KSM madvise run. Unlike most madvise
that can work on anonymous memory (like MADV_DONTNEED calling
zap_page_range that in turn will call split_huge_page_pmd) there is no
pte mangling done whatsoever by the ksm_madvise, it just splits the
vma in a way that can't fit an hugepage anymore, but without calling
split_huge_page. That would then lead to a later fork to run
copy_huge_pmd twice on the same hugepage (so boosting the mapcount
twice). That would later trip a BUG_ON in split_huge_page when process
quits as the number of hugepmd found by the rmap walk would be lower
than the page_mapcount. No memory corruption or data corruption could
happen and in fact this would result in memleak if there wasn't such
BUG_ON. split_huge_page is very strict, changes for something to go
wrong unnoticed are very low.
For triggering it takes a combination of khugepaged just at the wrong
time plus a combination of KSM madvise, and finally fork.
Similar invariant breakage (leading to copy_huge_pmd running twice on
the same hugepage) could have happened in case things like munmap run
in the middle of a vma would succeed the first split_vma but fail the
second one (no pte mangling would be invoked and the effect of the
first succeeded split_vma wouldn't be rolled back).
Fixing this from inside vma_adjust takes care of every one of these
cases with a one liner change to mmap.c (that is optimized away at
compile time if TRANSPARENT_HUGEPAGE=n).
compaction-migration-warning | 25 ----
free_pages-count | 60 ----------
free_pages-drain_all_pages | 62 ----------
free_pages-vmstat | 156 --------------------------
swapin-race-conditions | 152 -------------------------
merged upstream.
Full diffstat:
Documentation/vm/transhuge.txt | 283 ++++
arch/alpha/include/asm/mman.h | 2
arch/mips/include/asm/mman.h | 2
arch/parisc/include/asm/mman.h | 2
arch/powerpc/mm/gup.c | 12
arch/x86/include/asm/kvm_host.h | 1
arch/x86/include/asm/paravirt.h | 23
arch/x86/include/asm/paravirt_types.h | 6
arch/x86/include/asm/pgtable-2level.h | 9
arch/x86/include/asm/pgtable-3level.h | 23
arch/x86/include/asm/pgtable.h | 149 ++
arch/x86/include/asm/pgtable_64.h | 28
arch/x86/include/asm/pgtable_types.h | 3
arch/x86/kernel/paravirt.c | 3
arch/x86/kernel/tboot.c | 2
arch/x86/kernel/vm86_32.c | 1
arch/x86/kvm/mmu.c | 60
arch/x86/kvm/paging_tmpl.h | 4
arch/x86/mm/gup.c | 28
arch/x86/mm/pgtable.c | 66
arch/xtensa/include/asm/mman.h | 2
drivers/base/node.c | 21
fs/Kconfig | 2
fs/exec.c | 44
fs/proc/meminfo.c | 14
fs/proc/page.c | 14
include/asm-generic/mman-common.h | 2
include/asm-generic/pgtable.h | 130 +
include/linux/compaction.h | 13
include/linux/gfp.h | 14
include/linux/huge_mm.h | 170 ++
include/linux/khugepaged.h | 66
include/linux/kvm_host.h | 4
include/linux/memory_hotplug.h | 14
include/linux/mm.h | 114 +
include/linux/mm_inline.h | 19
include/linux/mm_types.h | 3
include/linux/mmu_notifier.h | 66
include/linux/mmzone.h | 1
include/linux/page-flags.h | 36
include/linux/sched.h | 1
include/linux/swap.h | 2
kernel/fork.c | 12
kernel/futex.c | 55
mm/Kconfig | 38
mm/Makefile | 1
mm/compaction.c | 48
mm/huge_memory.c | 2291 ++++++++++++++++++++++++++++++++++
mm/hugetlb.c | 69 -
mm/ksm.c | 52
mm/madvise.c | 8
mm/memcontrol.c | 138 +-
mm/memory-failure.c | 2
mm/memory.c | 198 ++
mm/memory_hotplug.c | 14
mm/mempolicy.c | 14
mm/migrate.c | 12
mm/mincore.c | 7
mm/mmap.c | 7
mm/mmu_notifier.c | 20
mm/mprotect.c | 20
mm/mremap.c | 8
mm/page_alloc.c | 31
mm/pagewalk.c | 1
mm/rmap.c | 115 -
mm/sparse.c | 4
mm/swap.c | 117 +
mm/swap_state.c | 6
mm/swapfile.c | 2
mm/vmscan.c | 89 -
mm/vmstat.c | 1
virt/kvm/iommu.c | 2
virt/kvm/kvm_main.c | 56
73 files changed, 4486 insertions(+), 411 deletions(-)
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/