Re: [RFC PATCH 0/7] locking/rtqspinlock: Realtime queued spinlocks

[Steven Rostedt]

On Wed, 4 Jan 2017 15:02:23 -0500
Waiman Long <longman@xxxxxxxxxx> wrote:

> On 01/04/2017 10:55 AM, Steven Rostedt wrote:
> > On Wed, 4 Jan 2017 10:25:14 -0500
> > Waiman Long <longman@xxxxxxxxxx> wrote:
> >
> >    
> >> I know that in -RT kernel, all the non-raw spinlocks are replaced by
> >> rtmutex which is a sleeping lock. This can have a real performance
> >> impact on systems with more than a few cores. The rtmutex isn't fair either.  
> > We do fine on 80+ CPUs. Is that enough cores for you ;-)  
> 
> It depends on what you mean fine. I haven't done the actual benchmark
> yet, but I believe that a -RT system with 80+ CPUs will feel noticeably
> slower for non-RT tasks than a system with non-RT kernel. I am not
> saying that the system will slow down significantly, but the slow down
> will certainly be noticeable.

Really matters what the work load is. Yes, java apps will be much
slower, because java runs 1000s of threads, and the real bottle neck is
with the read-writer locks (the mmap rwsem causes issues here).

But, really, I've done kernel builds on these large boxes running -rt,
and those still do well.

> 
> > Note, it's not a true sleeping lock, because of the adaptive nature.
> > That is, it spins unless the owner of the lock is sleeping, in which
> > case, it too will sleep (why spin waiting for a task that isn't
> > running). But if the owner is running, it will spin too.  
> 
> I think this is a recent change by Davidlohr. However, the spinning is
> limited to the top waiter. The rests will still go to sleep.

No, adaptive spinlocks were there for some time. It's copyright in 2008
by Gregory Haskins, Sven Dietrich and Peter Morreale.

As for the top waiter, that could probably be optimized to not do so.
Good project to try out and see how that speeds things up.

> >
> > Note, I also want to get rt_mutex into the kernel first for all sleeping
> > locks. That is, get the logic in before we convert spin_locks to
> > sleeping locks.

> 
> The rtmutex code is already in the upstream kernel. It is just that
> there isn't any code yet that can let you flip a switch (a config
> option) and change all sleeping locks to use rtmutex.

The pi code is upstream, but it's only used by futexes and maybe even
rcu boosting. But normal kernel mutexes don't have pi.

-- Steve