Re: [PATCH v3 3/4] mm: prevent KSM from breaking VMA merging for new VMAs

[Vlastimil Babka]

On 6/20/25 14:48, Lorenzo Stoakes wrote:
> Hi Andrew,
> 
> Sending a fix-patch for this commit due to a reported syzbot issue which
> highlighted a bug in the implementation.
> 
> I discuss the syzbot report at [0].
> 
> [0]: https://lore.kernel.org/all/a55beb72-4288-4356-9642-76ab35a2a07c@lucifer.local/
> 
> There's a very minor conflict around the map->vm_flags vs. map->flags change,
> easily resolvable, but if you need a respin let me know.
> 
> I ran through all mm self tests included the newly introduced one in 4/4 and all
> good.
> 
> Thanks, Lorenzo
> 
> ----8<----
> From 4d9dde3013837595d733b5059c2d6474261654d6 Mon Sep 17 00:00:00 2001
> From: Lorenzo Stoakes <lorenzo.stoakes@xxxxxxxxxx>
> Date: Fri, 20 Jun 2025 13:21:03 +0100
> Subject: [PATCH] mm/vma: correctly invoke late KSM check after mmap hook
> 
> Previously we erroneously checked whether KSM was applicable prior to
> invoking the f_op->mmap() hook in the case of not being able to perform
> this check early.
> 
> This is problematic, as filesystems such as hugetlb, which use anonymous
> memory and might otherwise get KSM'd, set VM_HUGETLB in the f_op->mmap()
> hook.
> 
> Correct this by checking at the appropriate time.
> 
> Reported-by: syzbot+a74a028d848147bc5931@xxxxxxxxxxxxxxxxxxxxxxxxx
> Closes: https://lore.kernel.org/all/6853fc57.a00a0220.137b3.0009.GAE@xxxxxxxxxx/
> Signed-off-by: Lorenzo Stoakes <lorenzo.stoakes@xxxxxxxxxx>

I've checked the version in mm tree, LGTM.

Reviewed-by: Vlastimil Babka <vbabka@xxxxxxx>