[RFC PATCH 2/7] mm/gup: handle memory.high overage while populating

From: Shakeel Butt

Date: Sat Nov 21 2026 - 21:48:14 EST


mlock(), mlockall(), mmap() with MAP_POPULATE or MAP_LOCKED, and
MADV_POPULATE_READ/WRITE can fault in a lot of memory in one call,
with mmap_lock held. When their charges can block, the charge path
throttles them for memory.high with mmap_lock still held. In a VM with
THP on, mlock() of an 80M MADV_HUGEPAGE range under memory.high=64M
made another thread wait up to 17.2s to mmap(), and a task in another
cgroup wait up to 15.3s to read the process's /proc/<pid>/cmdline.
THP charges in VMAs without MADV_HUGEPAGE can't block under the
default defrag setting, so they are not throttled at all: mlock(512M)
reached 513M.

Throttle these calls with mmap_lock dropped instead. The populate
loops, __mm_populate() and madvise_populate(), now fault in at most
2MiB per GUP call. Between calls, if more than a batch of overage is
pending, they drop mmap_lock, handle it, and go on. GUP itself doesn't
change. One fault can still charge a whole folio.

With this patch, mlock(512M) stays at 84M, and populating 2G with no
limit takes as long as before. The mmap_lock stall goes away when a
later patch removes the synchronous step.

Signed-off-by: Shakeel Butt <shakeel.butt@xxxxxxxxx>
---
mm/gup.c | 9 +++++++--
mm/internal.h | 27 +++++++++++++++++++++++++++
mm/madvise.c | 11 +++++++++--
3 files changed, 43 insertions(+), 4 deletions(-)

diff --git a/mm/gup.c b/mm/gup.c
index 7605da0e06d8..d93d89f6aad0 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -1951,6 +1951,7 @@ int __mm_populate(unsigned long start, unsigned long len, int ignore_errors)
end = start + len;

for (nstart = start; nstart < end; nstart = nend) {
+ populate_handle_over_high(mm, &locked);
/*
* We want to fault in pages for [nstart; end) address range.
* Find first corresponding VMA.
@@ -1976,9 +1977,13 @@ int __mm_populate(unsigned long start, unsigned long len, int ignore_errors)
/*
* Now fault in a range of pages. populate_vma_page_range()
* double checks the vma flags, so that it won't mlock pages
- * if the vma was already munlocked.
+ * if the vma was already munlocked. Do one chunk at a time,
+ * see POPULATE_CHUNK_SIZE. On error, nend skips the rest of
+ * the VMA.
*/
- ret = populate_vma_page_range(vma, nstart, nend, &locked);
+ ret = populate_vma_page_range(vma, nstart,
+ populate_chunk_end(nstart, nend),
+ &locked);
if (ret < 0) {
if (ignore_errors) {
ret = 0;
diff --git a/mm/internal.h b/mm/internal.h
index de4593b3e3b6..12f7b4b1254f 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -948,6 +948,33 @@ extern long populate_vma_page_range(struct vm_area_struct *vma,
unsigned long start, unsigned long end, int *locked);
extern long faultin_page_range(struct mm_struct *mm, unsigned long start,
unsigned long end, bool write, int *locked);
+
+/*
+ * __mm_populate() and madvise_populate() fault in at most
+ * POPULATE_CHUNK_SIZE at a time, and handle memory.high overage between
+ * chunks with mmap_lock dropped, so other tasks don't wait for them.
+ */
+#define POPULATE_CHUNK_SIZE SZ_2M
+
+static inline unsigned long populate_chunk_end(unsigned long start,
+ unsigned long end)
+{
+ return min(end, round_down(start, POPULATE_CHUNK_SIZE) +
+ POPULATE_CHUNK_SIZE);
+}
+
+static inline void populate_handle_over_high(struct mm_struct *mm,
+ int *locked)
+{
+ if (!mem_cgroup_over_high_pending())
+ return;
+ if (*locked) {
+ mmap_read_unlock(mm);
+ *locked = 0;
+ }
+ mem_cgroup_handle_over_high(GFP_KERNEL);
+}
+
bool mlock_future_ok(const struct mm_struct *mm, bool is_vma_locked,
unsigned long bytes);

diff --git a/mm/madvise.c b/mm/madvise.c
index 2d54cb9863ea..c77924e52d79 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -992,8 +992,15 @@ static long madvise_populate(struct madvise_behavior *madv_behavior)
long pages;

while (start < end) {
- /* Populate (prefault) page tables readable/writable. */
- pages = faultin_page_range(mm, start, end, write, &locked);
+ /*
+ * Populate (prefault) page tables readable/writable, at most a
+ * chunk at a time, see POPULATE_CHUNK_SIZE.
+ */
+ pages = faultin_page_range(mm, start,
+ populate_chunk_end(start, end),
+ write, &locked);
+ if (pages >= 0)
+ populate_handle_over_high(mm, &locked);
if (!locked) {
mmap_read_lock(mm);
locked = 1;
--
2.53.1-Meta