Re: [v1 resend 00/12] THP support for zone device page migration

[Matthew Brost]

On Fri, Jul 18, 2025 at 01:57:13PM +1000, Balbir Singh wrote:
> On 7/18/25 09:40, Matthew Brost wrote:
> > On Fri, Jul 04, 2025 at 09:34:59AM +1000, Balbir Singh wrote:
> ...
> >>
> >> The nouveau dmem code has been enhanced to use the new THP migration
> >> capability.
> >>
> >> Feedback from the RFC [2]:
> >>
> > 
> > Thanks for the patches, results look very promising. I wanted to give
> > some quick feedback:
> > 
> 
> Are you seeing improvements with the patchset?
> 

We're nowhere near stable yet, but basic testing shows that CPU time
from the start of migrate_vma_* to the end drops from ~300µs to ~6µs on
a 2MB GPU fault. A lot of this improvement is dma-mapping at 2M
grainularity for the CPU<->GPU copy rather than mapping 512 4k pages
too.

> > - You appear to have missed updating hmm_range_fault, specifically
> > hmm_vma_handle_pmd, to check for device-private entries and populate the
> > HMM PFNs accordingly. My colleague François has a fix for this if you're
> > interested.
> > 
> 
> Sure, please feel free to post them. 
> 
> > - I believe copy_huge_pmd also needs to be updated to avoid installing a
> > migration entry if the swap entry is device-private. I don't have an
> > exact fix yet due to my limited experience with core MM. The test case
> > that triggers this is fairly simple: fault in a 2MB device page on the
> > GPU, then fork a process that reads the page — the kernel crashes in
> > this scenario.
> > 
> 
> I'd be happy to look at any traces you have or post any fixes you have
> 

I've got it so the kernel doesn't explode but still get warnings like:

[ 3564.850036] mm/pgtable-generic.c:54: bad pmd ffff8881290408e0(efffff80042bfe00)
[ 3565.298186] BUG: Bad rss-counter state mm:ffff88810a100c40 type:MM_ANONPAGES val:114688
[ 3565.306108] BUG: non-zero pgtables_bytes on freeing mm: 917504

I'm basically just skip is_swap_pmd clause if the entry is device
private, and let the rest of the function execute. This avoids
installing a migration entry—which isn't required and cause the
crash—and allows the rmap code to run, which flips the pages to not
anonymous exclusive (effectively making them copy-on-write (?), though
that doesn't fully apply to device pages). It's not 100% correct yet,
but it's a step in the right direction.

Matt

> Thanks for the feedback
> Balbir Singh