Re: [PATCH] locking/rwsem: don't let a read lock steal from a queued writer

From: Peter Zijlstra

Date: Thu Nov 19 2026 - 10:57:07 EST


On Thu, Nov 05, 2026 at 08:24:27PM +0000, Lorenzo Stoakes (ARM) wrote:
> Since commit 1a728dff855a ("locking/rwsem: Enable reader optimistic lock
> stealing"), a racing read lock can steal an rwsem from under a queued
> writer whose handoff timeout hasn't expired yet.
>
> This is most problematic when an rwsem read lock is released then
> reacquired in order to relieve lock contention, as instead it will simply
> have the lock handed right back to it ahead of the intended beneficiary.
>
> Naturally, this behaviour rather defeats the object of doing this kind of
> thing, so this is less than ideal.
>
> Fix this by updating the optimistic read lock stealing code in
> rwsem_down_read_slowpath() to explicitly check whether the first queued
> lock is a waiting writer, and to not allow the theft if so.
>
> When a steal is disallowed, keep the existing sem->wait_lock held when
> jumping to the queue code to avoid useless lock acquisition/release.
>
> In order to perform this check, sem->wait_lock is acquired whenever there
> are queued waiters in the steal code, rather than the previous behaviour of
> only doing so when the reader count is 1.
>
> Since failing to steal a read lock unconditionally results in acquisition
> of the sem->wait_lock, this doesn't seem likely to be a significant
> overhead.
>
> Also update the comment describing optimistic read lock acquisition
> conditions to include this newly introduced behaviour.
>
> Care is taken to ensure correct ordering of lock acquisition, lock waking
> and marking the lock reader-owned.
>
> Signed-off-by: Lorenzo Stoakes (ARM) <ljs@xxxxxxxxxx>

So conceptually this is indeed what I meant -- ACK on that respect.

It would however be good if we can get this ran through some CIs to see
if there's some obvious regressions and whatnot.

> ---
> kernel/locking/rwsem.c | 30 ++++++++++++++++++------------
> 1 file changed, 18 insertions(+), 12 deletions(-)
>
> diff --git a/kernel/locking/rwsem.c b/kernel/locking/rwsem.c
> index b9c180ac1eee..cdb3dd6260bf 100644
> --- a/kernel/locking/rwsem.c
> +++ b/kernel/locking/rwsem.c
> @@ -59,7 +59,8 @@
> * A fast path reader optimistic lock stealing is supported when the rwsem
> * is previously owned by a writer and the following conditions are met:
> * - rwsem is not currently writer owned
> - * - the handoff isn't set.
> + * - the handoff isn't set
> + * - a writer isn't at the head of the wait queue.
> */
> #define RWSEM_READER_OWNED (1UL << 0)
> #define RWSEM_NONSPINNABLE (1UL << 1)
> @@ -1034,31 +1035,36 @@ rwsem_down_read_slowpath(struct rw_semaphore *sem, long count, unsigned int stat
> * Reader optimistic lock stealing.
> */
> if (!(count & (RWSEM_WRITER_LOCKED | RWSEM_FLAG_HANDOFF))) {
> - rwsem_set_reader_owned(sem);
> - lockevent_inc(rwsem_rlock_steal);
> -
> - /*
> - * Wake up other readers in the wait queue if it is
> - * the first reader.
> - */
> - if ((rcnt == 1) && (count & RWSEM_FLAG_WAITERS)) {
> + if (count & RWSEM_FLAG_WAITERS) {
> raw_spin_lock_irq(&sem->wait_lock);
> - if (sem->first_waiter)
> + first = sem->first_waiter;
> + /* A writer at the head of queue takes precedence. */
> + if (first && first->type == RWSEM_WAITING_FOR_WRITE)
> + goto queue_locked;
> + rwsem_set_reader_owned(sem);
> + /* If first reader, wake other readers in queue. */
> + if (first && rcnt == 1)
> rwsem_mark_wake(sem, RWSEM_WAKE_READ_OWNED,
> &wake_q);
> raw_spin_unlock_irq(&sem->wait_lock);
> - wake_up_q(&wake_q);
> + } else {
> + rwsem_set_reader_owned(sem);
> }
> +
> + lockevent_inc(rwsem_rlock_steal);
> + if (!wake_q_empty(&wake_q))
> + wake_up_q(&wake_q);
> return sem;
> }
>
> queue:
> + raw_spin_lock_irq(&sem->wait_lock);
> +queue_locked:
> waiter.task = current;
> waiter.type = RWSEM_WAITING_FOR_READ;
> waiter.timeout = jiffies + RWSEM_WAIT_TIMEOUT;
> waiter.handoff_set = false;
>
> - raw_spin_lock_irq(&sem->wait_lock);
> first = sem->first_waiter;
> if (!first) {
> /*
>
> ---
> base-commit: 05441d697f7f9a927264f4058cbbc6a03dfcac9d
> change-id: 20261105-rwsem-dont-steal-from-queued-writer-04b11d9d406b
>
> Cheers,
> --
> Lorenzo Stoakes (ARM) <ljs@xxxxxxxxxx>
>