Re: [PATCH] mm: limit THP alignment – performance gain observed in AI inference workloads

[Vlastimil Babka]

On 7/1/25 20:49, Zi Yan wrote:
>>> This is very useful information and it's appreciated! Let's not drown this
>>> out with restatements of stuff already covered.
>>>
>>>> ⚙️ 5. mTHP note
>>>> Although this patch doesn’t target mTHP directly, I believe a similar logic
>>>> tweak could apply there too — especially with shmem-backed workloads (common
>>>> in model servers using shared tensor memory). I’d be happy to help test any
>>>> changes proposed there to derive the consequent results.
>>> Dev - could we hold off on any effort to do something like this until I've
>>> had a chance to refactor THP somewhat? This is already a mess and I'd like
>>> to avoid us piling on more complexity.
>>>
>>> We can revisit this at a later stage.
>>
>> Yes of course. I had run a small benchmark on a quick dumb patch I wrote and I
>> don't see any measurable perf improvement, probably because the highest THP order
>> getting chosen is always PMD size.
> 
> I think mTHP is much more complicated, since mTHP has many sizes.
> Trying to adjust VMA alignments to get mTHP might not work well, since
> you never know what sizes new VMAs are going to have.

Yes I agree it's more complicated. In case there would be a stream of
allocations of varying small-ish sizes, aligning each of them to its
smallest applicable mTHP could create gaps that wouldn't exist if we ignored
the alignment and just find any free area and in the end merge it to an
existing one. Basically we'd risk recreating the issue with gaps.

Sticking to one size (2MB) mitigates this to some extent. Unfortunately even
after my fix the heuristics might be prone to gaps:

- all allocations not multiple of 2MB - will merge freely

- all allocations multiple of 2MB - the alignment heuristic will kick in,
but as a result allocations should still merge as all boundaries are 2MB
alignned

- allocations alternate between multiple of 2MB and non-multiple of 2MB -
this will still create gaps

Note we already had a report about ebizzy regressing due to my commit [1]
and I suspect it might be due to this kind of scenario. A proper
investigation would be useful but I didn't get to it.

Maybe the solution is to first check if unaligned search gives us a range
that will merge with adjacent area, and only try the alignment heuristics if
it doesn't. This will still fail if mmap() is followed by e.g. mprotect() or
madvise() that will change an initially un-mergeable area to a mergeable
one. I have no ideas around that though. Just some thoughts to consider for
anyone wanting to change things here further :)

[1] https://lore.kernel.org/all/019401db769f%24961e7e20%24c25b7a60%24@xxxxxxxxx/

> IMHO, it might be better to align VMA to PMD or the largest mTHP size
> (for example, on ARM64 with 64KB base page, PMD THP is 512MB, a 2MB
> mTHP sounds more reasonable there) if possible and enable
> VMA merging as much as possible for future huge page collapse.
> mTHP can be used to fill the non faulted holes in VMAs if necessary.
> 
>>
>> Out of curiosity, where do you plan to do the refactoring?
> 
> 
> Best Regards,
> Yan, Zi
>