Re: [PATCH 02/27] sched/isolation: Introduce housekeeping per-cpu rwsem

[Waiman Long]

On 6/23/25 1:39 PM, Tejun Heo wrote:
> Hello,
>
> On Mon, Jun 23, 2025 at 01:34:58PM -0400, Waiman Long wrote:
> > On 6/20/25 11:22 AM, Frederic Weisbecker wrote:
> > > The HK_TYPE_DOMAIN isolation cpumask, and further the
> > > HK_TYPE_KERNEL_NOISE cpumask will be made modifiable at runtime in the
> > > future.
> > >
> > > The affected subsystems will need to synchronize against those cpumask
> > > changes so that:
> > >
> > > * The reader get a coherent snapshot
> > > * The housekeeping subsystem can safely propagate a cpumask update to
> > >     the susbsytems after it has been published.
> > >
> > > Protect against readsides that can sleep with per-cpu rwsem. Updates are
> > > expected to be very rare given that CPU isolation is a niche usecase and
> > > related cpuset setup happen only in preparation work. On the other hand
> > > read sides can occur in more frequent paths.
> > >
> > > Signed-off-by: Frederic Weisbecker <frederic@xxxxxxxxxx>
> > Thanks for the patch series and it certainly has some good ideas. However I
> > am a bit concern about the overhead of using percpu-rwsem for
> > synchronization especially when the readers have to wait for the completion
> > on the writer side. From my point of view, during the transition period when
> > new isolated CPUs are being added or old ones being removed, the reader will
> > either get the old CPU data or the new one depending on the exact timing.
> > The effect the CPU selection may persist for a while after the end of the
> > critical section.
> >
> > Can we just rely on RCU to make sure that it either get the new one or the
> > old one but nothing in between without the additional overhead?
> So, I had a similar thought - ie. does this need full interlocking so that
> when the modification operation can wait for existing users to drain? It'd
> be nice to explain that part a bit more. That said, percpu_rwsem read path
> is pretty cheap, so if that is a requirement, I doubt the overhead
> difference between RCU access and percpu read locking would make meaningful
> difference.
>
> Thanks.

The percpu-rwsem does have a cheaper read side compared with rwsem for 
typical use case where writer update happens sparingly. However, when 
the writer has successful acquired the write lock, the readers do have 
to wait until the writer issues a percpu_up_write() call before they can 
proceed. It is the delay introduced by this wait that I am worry about. 
Isolated partitions are typically set up to run RT applications that 
have a strict latency requirement. So any possible latency spike should 
be avoided.

Cheers,
Longman

>