[PATCH v2 0/8] convert hugetlb fault functions to folios
From: Sidhartha Kumar
Date: Wed Jan 25 2023 - 12:06:18 EST
============== CHANGE LOG ===========================
v1 -> v2
- Add r-b to patches 1,3
- Refactor hugetlbfs_pagecache_present() to use rcu +
page_cache_next_miss() per Matthew in patch 2.
- Change function name from putback_active_hugetlb_folio()
to folio_putback_active_hugetlb() per Matthew in patch 3.
- Squashed v1 patches 4-5 into one patch to make for easier
review as did not need to handle null page case from
alloc_huge_page().
Refactor hugetlbfs_fallocate() to use same rcu +
page_cache_next_miss() that Matthew suggested for
hugetlbfs_pagecache_present() per Mike in patch 4.
Initialize h_cg variable to NULL in alloc_hugetlb_folio() to
suppress compiler warning when CONFIG_CGROUP_HUGETLB is not set
per lkp@xxxxxxxxx in patch 4.
Changed variable page_in_pagecache to folio_in_pagecache in
hugetlb_mcopy_atomic_pte() as the variable is set to true once
the folio variable is added to the page cache in patch 4.
- Fixed wrong change from get_page(page) ->
folio_get(pagecache_folio) reported by Gerald Schaefer in
patch 7.
============== OVERVIEW ===========================
This series converts the hugetlb page faulting functions to operate on
folios. These include hugetlb_no_page(), hugetlb_wp(),
copy_hugetlb_page_range(), and hugetlb_mcopy_atomic_pte().
patch 1-3:
- convert prerequisite helper functions to folios.
patch 4:
- add a folio variable to the hugetlb fault functions to complete
conversion to folios.
patch 5-7:
- convert three functions to take in a folio rather than a page as
all callers now use folios.
patch 8:
-update documentation that references alloc_huge_page
============== TEST COVERAGE ============================
Linux Test Project Hugetlb Test Fault Coverage
[opc@sidhakum-devel kernel (master)]$ sudo ./funccount 'hugetlb_no_page'
FUNC COUNT
hugetlb_no_page 7796
[opc@sidhakum-devel kernel (master)]$ sudo ./funccount 'hugetlb_wp'
FUNC COUNT
hugetlb_wp 4623
Using fallocate commands to create files on hugeltbfs
[opc@sidhakum-devel kernel (master)]$ sudo ./funccount 'hugetlbfs*'
FUNC COUNT
hugetlbfs_fallocate 1
Userfaultfd selftest
./userfaultfd hugetlb 256 50 /dev/hugepages/hugefile
[opc@sidhakum-devel kernel (master)]$ sudo ./funccount 'hugetlb_mcopy_atomic_pte'
FUNC COUNT
hugetlb_mcopy_atomic_pte 6240
[opc@sidhakum-devel kernel (master)]$ sudo ./funccount 'copy_hugetlb_page_range'
FUNC COUNT
copy_hugetlb_page_range 3
============== PERFORMANCE ============================
using bpftrace to track time spent in fault functions over 10 rounds of
the LTP hugetlb tests
pre-patch:
@hugetlb_wp_nsecs:
[256, 512) 3675 |@@@@@@@@@@ |
[512, 1K) 18875 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[1K, 2K) 1366 |@@@ |
[2K, 4K) 77 | |
[4K, 8K) 12 | |
[8K, 16K) 10 | |
[16K, 32K) 2 | |
[32K, 64K) 0 | |
[64K, 128K) 0 | |
[128K, 256K) 0 | |
[256K, 512K) 0 | |
[512K, 1M) 0 | |
[1M, 2M) 0 | |
[2M, 4M) 0 | |
[4M, 8M) 0 | |
[8M, 16M) 0 | |
[16M, 32M) 0 | |
[32M, 64M) 0 | |
[64M, 128M) 0 | |
[128M, 256M) 0 | |
[256M, 512M) 0 | |
[512M, 1G) 0 | |
[1G, 2G) 0 | |
[2G, 4G) 1 | |
@hugetlb_no_page_nsecs:
[64, 128) 1 | |
[128, 256) 0 | |
[256, 512) 67 | |
[512, 1K) 66 | |
[1K, 2K) 65 | |
[2K, 4K) 198 | |
[4K, 8K) 97 | |
[8K, 16K) 3 | |
[16K, 32K) 4 | |
[32K, 64K) 678 | |
[64K, 128K) 3401 |@ |
[128K, 256K) 96746 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[256K, 512K) 107 | |
[512K, 1M) 0 | |
[1M, 2M) 1 | |
[2M, 4M) 0 | |
[4M, 8M) 1 | |
[8M, 16M) 0 | |
[16M, 32M) 0 | |
[32M, 64M) 0 | |
[64M, 128M) 0 | |
[128M, 256M) 0 | |
[256M, 512M) 0 | |
[512M, 1G) 0 | |
[1G, 2G) 0 | |
[2G, 4G) 283 | |
post-patch:
@hugetlb_wp_nsecs:
[256, 512) 22282 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[512, 1K) 16242 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ |
[1K, 2K) 1579 |@@@ |
[2K, 4K) 76 | |
[4K, 8K) 18 | |
[8K, 16K) 30 | |
[16K, 32K) 4 | |
@hugetlb_no_page_nsecs:
[256, 512) 92 | |
[512, 1K) 46 | |
[1K, 2K) 83 | |
[2K, 4K) 188 | |
[4K, 8K) 80 | |
[8K, 16K) 13 | |
[16K, 32K) 8 | |
[32K, 64K) 2332 |@ |
[64K, 128K) 7992 |@@@ |
[128K, 256K) 106347 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[256K, 512K) 143 | |
[512K, 1M) 1 | |
[1M, 2M) 0 | |
[2M, 4M) 0 | |
[4M, 8M) 0 | |
[8M, 16M) 0 | |
[16M, 32M) 0 | |
[32M, 64M) 0 | |
[64M, 128M) 0 | |
[128M, 256M) 0 | |
[256M, 512M) 0 | |
[512M, 1G) 0 | |
[1G, 2G) 0 | |
[2G, 4G) 606 | |
rebased on 01/24/23 mm-unstable
Sidhartha Kumar (8):
mm/hugetlb: convert hugetlb_install_page to folios
mm/hugetlb: convert hugetlbfs_pagecache_present() to folios
mm/hugetlb: convert putback_active_hugepage to take in a folio
mm/hugetlb: convert hugetlb fault paths to use alloc_hugetlb_folio()
mm/hugetlb: convert restore_reserve_on_error to take in a folio
mm/hugetlb: convert hugetlb_add_to_page_cache to take in a folio
mm/hugetlb: convert hugetlb_wp() to take in a folio
Documentation/mm: update hugetlbfs documentation to mention
alloc_hugetlb_folio
Documentation/mm/hugetlbfs_reserv.rst | 21 +-
.../zh_CN/mm/hugetlbfs_reserv.rst | 14 +-
fs/hugetlbfs/inode.c | 40 +--
include/linux/hugetlb.h | 16 +-
include/linux/rmap.h | 2 +-
mm/hugetlb.c | 279 +++++++++---------
mm/mempolicy.c | 6 +-
mm/migrate.c | 8 +-
mm/rmap.c | 6 +-
9 files changed, 196 insertions(+), 196 deletions(-)
--
2.39.1