Re: [PATCH 00/60] Private NUMA nodes
From: Yury Norov
Date: Sun Nov 22 2026 - 11:51:16 EST
On Fri, Nov 20, 2026 at 01:08:51PM -0500, Gregory Price via B4 Relay wrote:
> This is v6 of the Private NUMA node series. The series falls into four
> parts, each building on the one before:
>
> 1-18 mm: introduce N_MEMORY_COMMON
I walked through this part, mostly. It looks like you can substantially
decrease the patch count if you keep N_MEMORY meaning unchanged, and
add another type, like N_MEMORY_NOALLOC.
> 19-26 restrict folio-walking MM services to common memory
> 27-44 N_MEMORY_PRIVATE, kept out of cpusets, mempolicy, accounting
> and the page allocator's ordinary paths
> 45-61 private node features, memory hotplug of private memory,
> dax/kmem private mode, selftests and documentation
>
> Patches 1-26 change no behavior on any system and can be reviewed and
> merged on their own. Without private nodes, every path in patches 27-61
> behaves as before.
>
> The full stack is also available at:
> https://github.com/gourryinverse/linux/tree/scratch/gourry/managed_nodes/private_nodes_v6
>
> Motivation
> ==========
>
> Some memory should be visible as a NUMA node without becoming part of
> the general allocation pool: HBM, coherent GPU or accelerator memory,
> or device memory a driver hands out itself.
>
> Today such memory is either ordinary System RAM, where any allocation
> can fall back onto it and every MM service manages it, or ZONE_DEVICE,
> where the buddy allocator and the MM services are not available at all.
>
> Private nodes sit in between. Their memory is managed by the buddy
> allocator, but ordinary allocations never land on it, and generic MM
> services leave it alone unless the node opts back in.
>
> Model
> =====
>
> N_MEMORY = N_MEMORY_COMMON + N_MEMORY_PRIVATE
>
> - N_MEMORY keeps meaning "this node has memory".
> - N_MEMORY_COMMON means ordinary allocations and kernel services
> may use it.
> - Allocation isolation comes from the zonelists: a private node is
> on no fallback zonelist and has an empty __GFP_THISNODE zonelist.
> It is reached only through its own private zonelist.
> - Services that walk existing folios check folio_is_common_memory()
> where they already skip ZONE_DEVICE.
> - A private node can opt into compaction, reclaim and userspace
> placement through per-node features. The services are written in
> terms of their own node state (N_MEMORY_RECLAIM, ...), so services
> don't need to be "aware" of anything but themselves.
>
> The node state use / filtering model goes as such:
> 1) If you're walking the normal zonelists, you're already filtered
> 2) If you're walking nodelists to touch memory, filter on
> N_MEMORY_COMMON by default (replacing N_MEMORY).
> 3) If you're walking structures to operate on folios / pages,
> check folio_is_common_memory() (replacing zone_device checks)
> 4) If the service allows private memory interactions, use the
> N_MEMORY_<FEATURE> bit corresponding to the service.
>
> e.g.: reclaim checks N_MEMORY_FEATURE when walking N_MEMORY
> or when walking the private memory zonelist.
>
> N_MEMORY_COMMON (patches 1-18)
> ==============================
>
> N_MEMORY has two meanings today:
> 1) this node has memory
> 2) ordinary allocations and MM services can use this node's memory
>
> The two are the same today. They stop being the same once a NUMA node
> with memory isn't part of the general allocation pool.
>
> N_MEMORY_COMMON takes the second meaning, and the N_MEMORY users that
> mean it are converted. N_MEMORY continues reporting memory topology.
>
> The rule for choosing between them:
>
> N_MEMORY the node has memory: topology, per-node metadata and
> statistics, cpuset.mems.
> N_MEMORY_COMMON ordinary allocations or a kernel service may use the
> node's memory: zonelists, allocation and migration
> targets, per-node service threads, and sizing or
> accounting meant for ordinary allocations.
>
> Code that walks the existing zonelist needs no check. The zonelists
> are built from N_MEMORY_COMMON, so they already hold only memory the
> allocation may use.
>
> Unless a system actually has a private node, N_MEMORY and
> N_MEMORY_COMMON are the same set, so behavior should be unchanged.
>
> 1-3 node-state helpers, N_MEMORY_COMMON itself, and the
> `has_common_memory` sysfs ABI.
> 4 zonelists: the fallback lists hold common nodes, and a node's
> __GFP_THISNODE list is built only for a common node
> 5-11 allocation and migration targets
> 12-13 kswapd and kcompactd
> 14-18 sizing and accounting meant for ordinary allocations
>
> has_common_memory lists the same nodes as has_memory until memory
> outside N_MEMORY_COMMON exists. It is added now so that userspace, such
> as libnuma, can start choosing allocation targets from it, and so that
> the meaning is fixed before anything makes the two lists differ.
>
> Folio-walking MM services (patches 19-26)
> =========================================
>
> Patches 1-18 decide where memory comes from: zonelists, allocation
> targets and per-node services follow N_MEMORY_COMMON. Some MM services
> instead reach folios that already exist, wherever they are: through
> page tables (KSM, khugepaged, NUMA hinting, madvise, mbind() and
> migrate_pages(), move_pages()) or through physical addresses (DAMON).
>
> These services skip ZONE_DEVICE folios with folio_is_zone_device() and
> page_is_zone_device().
>
> Patch 19 generalizes those checks into folio_is_common_memory() and
> page_is_common_memory(), which reject ZONE_DEVICE folios and folios
> on a node outside N_MEMORY_COMMON.
>
> The rule: a service that walks folios checks folio_is_common_memory()
> where it would otherwise check folio_is_zone_device().
>
> Rather than add new hooks to page-table and pfn walkers, this reuses the
> places that already opt ZONE_DEVICE memory out.
>
> 19 folio_is_common_memory() and page_is_common_memory()
> 20-26 ZONE_DEVICE checks in folio-walking services become
> page/folio_is_common_memory(). One component per patch.
>
> N_MEMORY_PRIVATE (patches 27-44)
> ================================
>
> Patches 27-44 add the memory that makes N_MEMORY and N_MEMORY_COMMON
> differ. A private node has online System RAM that ordinary allocations
> do not use and MM services leave alone, so neither an allocation that
> names no node nor a node-local kernel allocation can land on it.
>
> Most of these patches add the explicit opt-outs that the N_MEMORY ->
> N_MEMORY_COMMON conversion did not cover, largely concentrated in
> cpuset, mempolicy and migration.
>
> In the page allocator, each node gets a private zonelist holding its
> own zones only if it is private. MM code reaches private memory only
> through that zonelist (ALLOC_ZONELIST_PRIVATE plus a nodemask).
>
> A private request whose node becomes ineligible relaxes onto common
> memory, and exhausting private memory fails the allocation (SIGBUS on
> fault) instead of invoking the OOM killer (this can change in the
> future with additional work).
>
> Nothing in patches 27-44 creates private memory. Memory hotplug of
> private memory, with dax/kmem as its first user, comes in patches
> 56-57, after the feature bits that let a private node opt into
> reclaim, compaction and userspace placement.
>
> 27-28 N_MEMORY_PRIVATE, exported as has_private_memory.
> A per-node feature mask on the pgdat.
> 29-30 accounting: MemTotal/MemFree/other limits ignore private mem.
> the auto-movable online policy ignores private memory
> 31-35 cpusets and mempolicy: private nodes are inherited only on
> request, never rebound onto, never spread over, and never
> migrate_pages() destinations
> 36 explicit reclaim and compaction requests refuse private nodes
> 37-44 page allocator:
> 37-38 the reserves and watermarks, and contiguous allocation,
> skip/refuse private memory
> 39-40 the private zonelist and its selection by MM callers
> 41 a private request whose node becomes ineligible relaxes
> onto common memory to ensure forward progress
> 42 exhausting private memory fails with SIGBUS, like an
> exhausted HugeTLB pool
> 43 an alloc_flags-aware folio allocator (Brendan Jackman)
> 44 a bind policy whose nodes became private falls back to
> common memory
>
> Private node features and providers (patches 45-61)
> ===================================================
>
> Patches 1-44 keep every generic MM service off private memory.
>
> Some private memory wants a few of them back: HBM or coherent GPU
> memory that userspace manages with mbind() and migrate_pages() needs
> reclaim and compaction, while staying out of incidental allocation
> (no fallback onto it, only explicit placement).
>
> Patches 45-55 add three opt-in feature bits and corresponding node
> states:
>
> N_MEMORY_COMPACTION kcompactd, proactive and explicit compaction
> N_MEMORY_RECLAIM kswapd, memcg and explicit reclaim
> N_MEMORY_MEMPOLICY mbind() and explicit migration may target it
>
> The states cover what the kernel does on its own initiative. Direct
> reclaim and compaction serve an allocation within its own zonelist, so
> an allocation from a private node may reclaim and compact it either way.
>
> RECLAIM requires COMPACTION, and MEMPOLICY requires both. Userspace
> control (mbind, migration) generates fragmentation, and as a result
> private allocations would fail unpredictably without reclaim/compaction.
>
> Relcaim and compaction are required by mempolicy because otherwise
> order-N allocations cause unpredictable out of memory conditions due
> to fragmentation.
>
> Demotion and NUMA balancing support are deferred to a later series
> but can be trivially support to allow demotions from HBM to DRAM
> as a way of cheap tiering.
>
> Every common node carries all three, so the existing N_MEMORY_COMMON
> users that mean "reclaim" or "compaction" move to the new states.
>
> Service support:
>
> 45-46 N_MEMORY_COMPACTION, and compaction switched to it
> 47-48 N_MEMORY_RECLAIM, and reclaim switched to it
> 49 madvise reclaim advice only for reclaim-capable memory
> 50 memcg reclaim also visits private nodes that permit reclaim
> 51 N_MEMORY_MEMPOLICY
> 52 explicit migration acts on N_MEMORY_MEMPOLICY memory
> 53 N_MEMORY_MEMPOLICY memory counts as general-purpose.
> MemAvailable (free and file pages) and dirty limits count
> only common memory.
> 54 MPOL_F_PRIVATE_NODES: mbind() flag that binds a mapping to
> one private node.
> 55 migrate_pages() support for N_MEMORY_MEMPOLICY nodes
>
> Creating a private node
>
> 56 memory hotplug of private memory, with a feature mask
> 57 dax/kmem private mode
>
> Testing and Documentation:
>
> 58-60 selftests: private kmem devices and services, memory policy
> bindings, migration and exhaustion, and cpuset interaction
> 61 Documentation/mm/numa_private_nodes.rst
>
> MPOL_F_PRIVATE_NODES is new uapi. Without it, private nodes in an
> mbind() mask are ignored, similar to nodes outside the cpuset.
>
> libnuma and numactl need a matching change to reach private nodes.
>
> There is no sysfs list of the nodes that accept private bindings.
> userspace can probe with MPOL_F_PRIVATE_NODES. Otherwise - we can
> discuss how service support should be discoverable to userspace.
>
> Memory hotplug is the only source of private memory right now.
>
> All System RAM on a node shares one configuration: hotplug with a
> different one fails with -EBUSY.
>
> Costs and known limits
> ======================
>
> - ZONELIST_PRIVATE is a third zonelist per pgdat; with NODES_SHIFT=10
> that is about 80KB per possible node, though a private zonelist
> holds at most one node's zones.
>
> local-only zonelists can be space-optimized if this matters.
>
> - Free page reporting reports free private pages like any others.
>
> - Tasks are OOM-scored by all of their resident memory,
> private pages included.
>
> Changes since v5 ("Private Memory NUMA Nodes")
> ==============================================
>
> - Reordered so the N_MEMORY audit (patches 1-26) comes first and can
> be reviewed (and merged) on its own if desired.
> - Capabilities became memory features with node states, so a service
> can be written in terms of itself: reclaim walks N_MEMORY_RECLAIM,
> compaction N_MEMORY_COMPACTION.
> - The capability set is pared down to COMPACTION, RECLAIM and
> MEMPOLICY for the first run. DEMOTION, NUMA_BALANCING, and
> compressed RAM support are deferred.
> - The private zonelist is now local-only. A fallback variant can be
> added if a use case needs one.
> - cpuset and OOM semantics are worked out:
> - cpuset.mems governs private nodes
> - private-only request inherits the parent's common nodes.
> - Private bindings are an explicit uapi flag, MPOL_F_PRIVATE_NODES,
> rather than inferred from the nodemask.
> - Exhausting private placement fails the fault with SIGBUS, as an
> exhausted HugeTLB pool does, instead of invoking the OOM killer.
> OOM support can be added later, but this keeps initial support
> reasonable simple.
>
> Signed-off-by: Gregory Price (Meta) <gourry@xxxxxxxxxx>
> ---
> Gregory Price (Meta) (60):
> nodemask: add node-state iterators
> mm: publish and clear node memory states via helpers
> mm: introduce the common-memory node state
> mm/page_alloc: build zonelists from common memory
> mm/mempolicy: keep interleave and tmpfs mount policies on common memory
> mm/memory_hotplug: migrate offlined pages to common memory
> mm/memory-tiers: restrict demotion topology to common memory
> mm/migrate: reject move_pages() destinations outside common memory
> mm/damon: reject migration destinations outside common memory
> sched/numa: restrict NUMA migration targets to common memory
> fork: check cached stack locality only on common memory nodes
> mm/vmscan: restrict reclaim to common memory
> mm/compaction: restrict compaction to common memory
> mm: base hash distribution on common memory nodes
> mm/page_alloc: count common memory nodes for kswapd recovery
> mm/oom_kill: compare OOM constraints against common memory
> PM: hibernate: size the image from common memory
> mm/khugepaged: base the recommended min_free_kbytes on common memory
> mm: add folio_is_common_memory() and page_is_common_memory()
> mm/ksm: restrict merging to common memory
> mm/khugepaged: restrict collapse to common memory
> mm/mempolicy: don't NUMA-hint folios outside common memory
> mm/damon: restrict DAMON-native folio operations to common memory
> mm/madvise: restrict reclaim advice to common memory
> mm/mempolicy: queue only common memory for mbind() and migrate_pages()
> mm/migrate: leave pages outside common memory in move_pages()
> mm: add the private-memory node state
> mm: select memory features per node
> mm: keep private memory out of the system memory totals
> mm/memory_hotplug: keep private memory out of the auto-movable ratio
> cgroup/cpuset: inherit common memory when a nodemask is private only
> cgroup/cpuset: keep spread page cache off private nodes
> cgroup/cpuset: ignore private nodes when matching OOM victims
> mm/mempolicy: don't rebind policies onto private nodes
> mm/mempolicy: keep private nodes out of migrate_pages() destinations
> mm: refuse explicit reclaim and compaction of private nodes
> mm/page_alloc: keep private memory out of reclaim reserves
> mm/page_alloc: refuse contiguous allocation from private memory
> mm/page_alloc: add a private-node zonelist
> mm/page_alloc: let MM callers select the local private zonelist
> mm/page_alloc: relax a private request whose node becomes ineligible
> mm: fail private placement on exhaustion instead of OOM killing
> mm/page_alloc: fall back when a bind policy's nodes become private
> mm: add N_MEMORY_COMPACTION node state
> mm/compaction: restrict compaction to N_MEMORY_COMPACTION
> mm: add N_MEMORY_RECLAIM node state
> mm/vmscan: restrict reclaim to N_MEMORY_RECLAIM instead of N_MEMORY_COMMON
> mm/madvise: restrict reclaim advice to reclaim-capable memory
> mm/vmscan: reclaim private nodes for memcg
> mm: add N_MEMORY_MEMPOLICY node state
> mm: restrict explicit migration to N_MEMORY_MEMPOLICY
> mm: count memory with userspace placement as general purpose
> mm/mempolicy: add MPOL_F_PRIVATE_NODES for private node bindings
> mm/migrate: route explicit NUMA migration to private nodes
> mm/memory_hotplug: online and offline private memory nodes
> dax/kmem: add a private memory mode
> selftests/dax: test private kmem devices and services
> selftests/dax: test memory policy on private nodes
> selftests/dax: test cpuset interaction with private nodes
> Documentation/mm: describe private memory nodes
>
> Documentation/ABI/stable/sysfs-devices-node | 27 +
> Documentation/ABI/testing/sysfs-bus-dax | 19 +
> Documentation/admin-guide/cgroup-v1/cpusets.rst | 7 +
> Documentation/admin-guide/cgroup-v2.rst | 13 +-
> .../admin-guide/mm/numa_memory_policy.rst | 22 +
> Documentation/mm/index.rst | 1 +
> Documentation/mm/numa_private_nodes.rst | 223 ++++++++
> Documentation/mm/physical_memory.rst | 25 +-
> drivers/base/node.c | 9 +-
> drivers/dax/kmem.c | 55 +-
> include/linux/memory_hotplug.h | 2 +-
> include/linux/mmzone.h | 6 +-
> include/linux/nodemask.h | 74 ++-
> include/linux/sched.h | 2 +
> include/linux/swap.h | 4 +-
> include/uapi/linux/mempolicy.h | 4 +-
> kernel/cgroup/cpuset.c | 85 ++-
> kernel/fork.c | 2 +-
> kernel/power/snapshot.c | 15 +-
> kernel/sched/fair.c | 2 +-
> mm/compaction.c | 10 +-
> mm/damon/core.c | 5 +-
> mm/damon/ops-common.c | 7 +-
> mm/damon/vaddr.c | 2 +-
> mm/huge_memory.c | 2 +-
> mm/internal.h | 71 +++
> mm/khugepaged.c | 20 +-
> mm/ksm.c | 8 +-
> mm/madvise.c | 6 +-
> mm/memory-tiers.c | 23 +-
> mm/memory.c | 5 +
> mm/memory_hotplug.c | 120 ++--
> mm/mempolicy.c | 165 ++++--
> mm/migrate.c | 15 +-
> mm/mm_init.c | 20 +-
> mm/mmap.c | 6 +-
> mm/nommu.c | 4 +-
> mm/oom_kill.c | 7 +-
> mm/page-writeback.c | 7 +-
> mm/page_alloc.c | 199 ++++++-
> mm/page_alloc.h | 22 +-
> mm/show_mem.c | 49 +-
> mm/vmscan.c | 58 +-
> tools/testing/selftests/dax/.gitignore | 3 +
> tools/testing/selftests/dax/Makefile | 10 +
> tools/testing/selftests/dax/config | 6 +
> tools/testing/selftests/dax/kmem_private.sh | 94 ++++
> tools/testing/selftests/dax/kmem_private_cpuset.sh | 70 +++
> tools/testing/selftests/dax/kmem_private_policy.sh | 58 ++
> .../selftests/dax/kmem_private_policy_tool.c | 620 +++++++++++++++++++++
> tools/testing/selftests/dax/kmem_private_rebind.sh | 301 ++++++++++
> .../selftests/dax/kmem_private_rebind_tool.c | 495 ++++++++++++++++
> .../testing/selftests/dax/kmem_private_services.sh | 97 ++++
> tools/testing/selftests/dax/private_node_common.sh | 268 +++++++++
> tools/testing/selftests/dax/settings | 2 +-
> 55 files changed, 3249 insertions(+), 203 deletions(-)
> ---
> base-commit: 15f114dac9a7bde1796f0e54e22726fd31cfa8a5
> change-id: 20261120-private_nodes_v6-64d04596e126
>
> Best regards,
> --
> ~Gregory
>