Re: [RFC PATCH v0 2/2] mm: sched: Batch-migrate misplaced pages

[David Hildenbrand]

On 22.05.25 19:30, Zi Yan wrote:
> On 22 May 2025, at 13:21, David Hildenbrand wrote:
>
> > On 22.05.25 18:38, Zi Yan wrote:
> > > On 22 May 2025, at 12:26, David Hildenbrand wrote:
> > >
> > > > On 22.05.25 18:24, Zi Yan wrote:
> > > > > On 22 May 2025, at 12:11, David Hildenbrand wrote:
> > > > >
> > > > > > On 21.05.25 10:02, Bharata B Rao wrote:
> > > > > > > Currently the folios identified as misplaced by the NUMA
> > > > > > > balancing sub-system are migrated one by one from the NUMA
> > > > > > > hint fault handler as and when they are identified as
> > > > > > > misplaced.
> > > > > > >
> > > > > > > Instead of such singe folio migrations, batch them and
> > > > > > > migrate them at once.
> > > > > > >
> > > > > > > Identified misplaced folios are isolated and stored in
> > > > > > > a per-task list. A new task_work is queued from task tick
> > > > > > > handler to migrate them in batches. Migration is done
> > > > > > > periodically or if pending number of isolated foios exceeds
> > > > > > > a threshold.
> > > > > >
> > > > > > That means that these pages are effectively unmovable for other purposes (CMA, compaction, long-term pinning, whatever) until that list was drained.
> > > > > >
> > > > > > Bad.
> > > > >
> > > > > Probably we can mark these pages and when others want to migrate the page,
> > > > > get_new_page() just looks at the page's target node and get a new page from
> > > > > the target node.
> > > >
> > > > How do you envision that working when CMA needs to migrate this exact page to a different location?
> > > >
> > > > It cannot isolate it for migration because ... it's already isolated ... so it will give up.
> > > >
> > > > Marking might not be easy I assume ...
> > >
> > > I guess you mean we do not have any extra bit to indicate this page is isolated,
> > > but it can be migrated. My point is that if this page is going to be migrated
> > > due to other reasons, like CMA, compaction, why not migrate it to the target
> > > node instead of moving it around within the same node.
> >
> > I think we'd have to identify that
> >
> > a) This page is isolate for migration (could be isolated for other
> >     reasons)
> >
> > b) The one responsible for the isolation is numa code (could be someone
> >     else)
> >
> > c) We're allowed to grab that page from that list (IOW sync against
> >     others, and especially also against), to essentially "steal" the
> >     isolated page.
>
> Right. c) sounds like adding more contention to the candidate list.
> I wonder if we can just mark the page as migration candidate (using
> a page flag or something else), then migrate it whenever CMA,
> compaction, long-term pinning and more look at the page.

I mean, all these will migrate the page either way, no need to add 
another flag for that.

I guess what you mean, indicating that the migration destination should 
be on a different node than the current one.

Well, and for the NUMA scanner (below) to find which pages to migrate.

... to be this raises some questions: like, if we don't migrate 
immediately, could that information ("migrate this page") actually now 
be wrong? I guess a way to obtain the destination node would suffice: if 
the destination node matches, no need to migrate from that NUMA scanner.

In addition,
> periodically, the migration task would do a PFN scanning and migrate
> any migration candidate. I remember Willy did some experiments showing
> that PFN scanning is very fast.

PFN scanning can be faster than walking lists, but I suspect it depends 
on how many pages there really are to be migrated ... and some other 
factors :)

--
Cheers,

David / dhildenb