Re: [PATCH 0/4] mm: ksm: prevent KSM from entirely breaking VMA merging

From: David Hildenbrand
Date: Mon May 19 2025 - 07:54:15 EST


On 19.05.25 10:51, Lorenzo Stoakes wrote:
When KSM-by-default is established using prctl(PR_SET_MEMORY_MERGE), this
defaults all newly mapped VMAs to having VM_MERGEABLE set, and thus makes
them available to KSM for samepage merging. It also sets VM_MERGEABLE in
all existing VMAs.

However this causes an issue upon mapping of new VMAs - the initial flags
will never have VM_MERGEABLE set when attempting a merge with adjacent VMAs
(this is set later in the mmap() logic), and adjacent VMAs will ALWAYS have
VM_MERGEABLE set.

Just to clarify, you mean that VM_MERGEABLE is set later, during __mmap_new_vma()->ksm_add_vma()->__ksm_add_vma(), and we are already past vma_merge_new_range(), correct?

--
Cheers,

David / dhildenb