Re: [PATCH v2] locking/rwbase: Prevent indefinite writer starvation

[Sebastian Andrzej Siewior]

On 2023-01-19 11:02:20 [+0000], Mel Gorman wrote:
> > - Once the writer removes READER_BIAS, it forces the reader into the
> >   slowpath.
> 
> Removed in __rwbase_write_trylock IIUC

And added back in case try trylock failed via __rwbase_write_unlock().
The RTmutex is unlocked and the READER_BIAS is "returned".

> >   At that time the writer does not own the wait_lock meaning
> >   the reader _could_ check the timeout before writer had a chance to set
> >   it. The worst thing is probably that if jiffies does not have the
> >   highest bit set then it will always disable the reader bias here.
> >   The easiest thing is probably to check timeout vs 0 and ensure on the
> >   writer side that the lowest bit is always set (in the unlikely case it
> >   will end up as zero).
> > 
> 
> I am missing something important. On the read side, we have
> 

Look at this side by side:

                writer                                                       reader

| static int __sched rwbase_write_lock(struct rwbase_rt *rwb,
|                                      unsigned int state)
| {
|         /* Force readers into slow path */
|         atomic_sub(READER_BIAS, &rwb->readers);


|                                                               static int __sched __rwbase_read_lock(struct rwbase_rt *rwb,
|                                                                                                     unsigned int state)
|                                                               {       
|                                                                       struct rt_mutex_base *rtm = &rwb->rtmutex;
|                                                                       int ret;                         
|                                                               
|                                                                       raw_spin_lock_irq(&rtm->wait_lock);

Reader has the lock, writer will wait.

|                                                                       /*
|                                                                        * Allow readers, as long as the writer has not completely
|                                                                        * acquired the semaphore for write.
|                                                                        */
|                                                                       if (atomic_read(&rwb->readers) != WRITER_BIAS) {

here, the timeout value is not yet populated by the writer so the reader
compares vs 0.

|                                                                               atomic_inc(&rwb->readers);
|                                                                               raw_spin_unlock_irq(&rtm->wait_lock);
|                                                                               return 0;
|                                                                       }
|                                                              

|         raw_spin_lock_irqsave(&rtm->wait_lock, flags);
|         if (__rwbase_write_trylock(rwb))
|                 goto out_unlock;
|

Hope this makes it easier.

Sebastian