Re: [PATCH 2/5] ipc/mqueue.c: Update/document memory barriers

[Manfred Spraul]

On 10/11/19 6:55 PM, Davidlohr Bueso wrote:
> On Fri, 11 Oct 2019, Manfred Spraul wrote:
>
> > Update and document memory barriers for mqueue.c:
> > - ewp->state is read without any locks, thus READ_ONCE is required.
>
> In general we relied on the barrier for not needing READ/WRITE_ONCE,
> but I agree this scenario should be better documented with them.

After reading core-api/atomic_ops.rst:

> _ONCE() should be used. [...] Alternatively, you can place a barrier.

So both approaches are ok.

Let's follow the "should", i.e.: all operations on the ->state variables 
to READ_ONCE()/WRITE_ONCE().

Then we have a standard, and since we can follow the "should", we should 
do that.

> Similarly imo, the 'state' should also need them for write, even if
> under the lock -- consistency and documentation, for example.
Ok, so let's convert everything to _ONCE. (assuming that my analysis 
below is incorrect)
> In addition, I think it makes sense to encapsulate some of the
> pipelined send/recv operations, that also can allow us to keep
> the barrier comments in pipelined_send(), which I wonder why
> you chose to remove. Something like so, before your changes:
I thought that the simple "memory barrier is provided" is enough, so I 
had removed the comment.


But you are right, there are two different scenarios:

1) thread already in another wake_q, wakeup happens immediately after 
the cmpxchg_relaxed().

This scenario is safe, due to the smp_mb__before_atomic() in wake_q_add()

2) thread woken up but e.g. a timeout, see ->state=STATE_READY, returns 
to user space, calls sys_exit.

This must not happen before get_task_struct acquired a reference.

And this appears to be unsafe: get_task_struct() is refcount_inc(), 
which is refcount_inc_checked(), which is according to lib/refcount.c 
fully unordered.

Thus: ->state=STATE_READY can execute before the refcount increase.

Thus: ->state=STATE_READY needs a smp_store_release(), correct?

> diff --git a/ipc/mqueue.c b/ipc/mqueue.c
> index 3d920ff15c80..be48c0ba92f7 100644
> --- a/ipc/mqueue.c
> +++ b/ipc/mqueue.c
> @@ -918,17 +918,12 @@ SYSCALL_DEFINE1(mq_unlink, const char __user *, 
> u_name)
> Â* The same algorithm is used for senders.
> Â*/
>
> -/* pipelined_send() - send a message directly to the task waiting in
> - * sys_mq_timedreceive() (without inserting message into a queue).
> - */
> -static inline void pipelined_send(struct wake_q_head *wake_q,
> +static inline void __pipelined_op(struct wake_q_head *wake_q,
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ struct mqueue_inode_info *info,
> -ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ struct msg_msg *message,
> -ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ struct ext_wait_queue *receiver)
> +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ struct ext_wait_queue *this)
> {
> -ÂÂÂ receiver->msg = message;
> -ÂÂÂ list_del(&receiver->list);
> -ÂÂÂ wake_q_add(wake_q, receiver->task);
> +ÂÂÂ list_del(&this->list);
> +ÂÂÂ wake_q_add(wake_q, this->task);
> ÂÂÂÂ/*
> ÂÂÂÂ * Rely on the implicit cmpxchg barrier from wake_q_add such
> ÂÂÂÂ * that we can ensure that updating receiver->state is the last
> @@ -937,7 +932,19 @@ static inline void pipelined_send(struct 
> wake_q_head *wake_q,
> ÂÂÂÂ * yet, at that point we can later have a use-after-free
> ÂÂÂÂ * condition and bogus wakeup.
> ÂÂÂÂ */
> -ÂÂÂ receiver->state = STATE_READY;
> +ÂÂÂÂÂÂÂ this->state = STATE_READY;
> +}
> +
> +/* pipelined_send() - send a message directly to the task waiting in
> + * sys_mq_timedreceive() (without inserting message into a queue).
> + */
> +static inline void pipelined_send(struct wake_q_head *wake_q,
> +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ struct mqueue_inode_info *info,
> +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ struct msg_msg *message,
> +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ struct ext_wait_queue *receiver)
> +{
> +ÂÂÂ receiver->msg = message;
> +ÂÂÂ __pipelined_op(wake_q, info, receiver);
> }
>
> /* pipelined_receive() - if there is task waiting in sys_mq_timedsend()
> @@ -955,9 +962,7 @@ static inline void pipelined_receive(struct 
> wake_q_head *wake_q,
> ÂÂÂÂif (msg_insert(sender->msg, info))
> ÂÂÂÂÂÂÂ return;
>
> -ÂÂÂ list_del(&sender->list);
> -ÂÂÂ wake_q_add(wake_q, sender->task);
> -ÂÂÂ sender->state = STATE_READY;
> +ÂÂÂ __pipelined_op(wake_q, info, sender);
> }
>
> static int do_mq_timedsend(mqd_t mqdes, const char __user *u_msg_ptr,

I would merge that into the series, ok?

--

ÂÂÂ Manfred