Re: [patch] mm, madvise: fail with ENOMEM when splitting vma will hit max_map_count

[Anshuman Khandual]

On 01/25/2017 04:02 AM, David Rientjes wrote:
> If madvise(2) advice will result in the underlying vma being split and
> the number of areas mapped by the process will exceed
> /proc/sys/vm/max_map_count as a result, return ENOMEM instead of EAGAIN.
> 
> EAGAIN is returned by madvise(2) when a kernel resource, such as slab,
> is temporarily unavailable.  It indicates that userspace should retry the
> advice in the near future.  This is important for advice such as
> MADV_DONTNEED which is often used by malloc implementations to free
> memory back to the system: we really do want to free memory back when
> madvise(2) returns EAGAIN because slab allocations (for vmas, anon_vmas,
> or mempolicies) cannot be allocated.
> 
> Encountering /proc/sys/vm/max_map_count is not a temporary failure,
> however, so return ENOMEM to indicate this is a more serious issue.  A
> followup patch to the man page will specify this behavior.

But in the due course there might be other changes in number of VMAs of
the process because of unmap() or merge() which could reduce the total
number of VMAs and hence this condition may not exist afterwards. In
that case EAGAIN still makes sense.