Re: [PATCH v3] smp: Wake ksoftirqd on PREEMPT_RT instead do_softirq().

[Christoph Hellwig]

On Tue, Feb 08, 2022 at 05:54:07PM +0100, Sebastian Andrzej Siewior wrote:
> The softirq implementation on PREEMPT_RT does not provide do_softirq(). The
> softirq can not be handled directly here because migration_cpu_stop() is
> invoked with disabled preemption/ interrupts.
> 
> A known user of scheduling softirqs from a remote function call is the block
> layer. It won't happen on PREEMPT_RT because it doesn't make sense for
> latency/ performance reasons and is disabled. Nevertheless this should
> be handled in case of a new user pops up rather than simply ignoring it.
> 
> Waking ksoftirqd unconditionally can be problematic if softirqs were already
> pending but not yet handled. This can happen since the migration thread
> is running at a high priority and able to preempt a threaded-interrupt.
> The woken-up ksoftirqd would catch-up all pending (and later raised)
> softirqs which is not desired on PREEMPT_RT since it is no longer
> handled where it has been originally raised. This in turn delays the
> actual processing until a SCHED_OTHER task can run.
> 
> Wake the softirq thread on PREEMPT_RT if a remote function call raised
> softirqs. Add warning in this case since this condition is not desired.
> 
> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@xxxxxxxxxxxxx>
> ---
> v2…v3:
>    - Only wake ksoftirqd if the softirqs were raised wthin
>      flush_smp_call_function_queue().
>    - Add a warning in the wake case.
> v1…v2: Drop an empty line.
> 
>  kernel/smp.c |   21 ++++++++++++++++++---
>  1 file changed, 18 insertions(+), 3 deletions(-)
> ---
> --- a/kernel/smp.c
> +++ b/kernel/smp.c
> @@ -691,10 +691,25 @@ void flush_smp_call_function_from_idle(v
>  	cfd_seq_store(this_cpu_ptr(&cfd_seq_local)->idle, CFD_SEQ_NOCPU,
>  		      smp_processor_id(), CFD_SEQ_IDLE);
>  	local_irq_save(flags);
> -	flush_smp_call_function_queue(true);
> -	if (local_softirq_pending())
> -		do_softirq();
> +	if (!IS_ENABLED(CONFIG_PREEMPT_RT)) {

I still absolutely hate these pointless negations in simple if
statements.