Re: [RFC 08/16] vfio/type1: Cache locked_vm to ease mmap_lock contention

[Jason Gunthorpe]

On Wed, Jan 05, 2022 at 07:46:48PM -0500, Daniel Jordan wrote:
> padata threads hold mmap_lock as reader for the majority of their
> runtime in order to call pin_user_pages_remote(), but they also
> periodically take mmap_lock as writer for short periods to adjust
> mm->locked_vm, hurting parallelism.
> 
> Alleviate the write-side contention with a per-thread cache of locked_vm
> which allows taking mmap_lock as writer far less frequently.
> 
> Failure to refill the cache due to insufficient locked_vm will not cause
> the entire pinning operation to error out.  This avoids spurious failure
> in case some pinned pages aren't accounted to locked_vm.
> 
> Cache size is limited to provide some protection in the unlikely event
> of a concurrent locked_vm accounting operation in the same address space
> needlessly failing in case the cache takes more locked_vm than it needs.

Why not just do the pinned page accounting once at the start? Why does
it have to be done incrementally?

Jason