Re: SCX_ENQ_IMMED potentially leaving dispatched tasks lingering on local DSQs

[Kuba Piecuch]

On Thu Apr 23, 2026 at 7:29 PM UTC, Tejun Heo wrote:
> Hello,
>
> On Thu, Apr 23, 2026 at 07:12:18PM +0000, Kuba Piecuch wrote:
> ...
>> I think these two cases show that we don't necessarily have to migrate a task
>> for wakeup_preempt() to be warranted.
>
> I see.
>
>> So to cover all cases (that I'm aware of), we need four checks:
>> 
>> * wakeup_preempt() in dispatch_to_local_dsq() in the case of rq != src_rq &&
>>   src_rq == dst_rq
>> 
>> * wakeup_preempt() at the end of move_remote_task_to_local_dsq()
>> 
>> * wakeup_preempt() somewhere on the scx_dsq_move() path, but only if we're
>>   moving the task to the local DSQ of a remote CPU (?)
>> 
>> * nr_immed check before returning RETRY_TASK
>> 
>> > Adding wakeup_preempt() to local_dsq_post_enq() for every insertion would
>> > work too, but I'd rather keep it in sync with how core sched handles
>> > move_queued_task() and only add it where it's actually needed.
>> 
>> Does having four separate cases to handle, not all of which involve task
>> migration, change that calculus somewhat?
>> I believe all of these cases will be handled if we add wakeup_preempt()
>> to local_dsq_post_enq().
>
> If we call wakeup_preempt() unconditionally, we'd be calling
> wakeup_preempt_scx() on every dispatch, which isn't too appealing. We can
> add an ENQ flag to mark these sites specifically but I'm not sure whether
> that's necessarily better. The invovled code paths are inherently subtle and
> finicky, so might as well just add the calls where they're necessary. What
> do you think?

I don't like the idea of maintaining the separate checks, precisely because
of how subtle and finnicky they are. I worry that it will be difficult to keep
all cases covered as the codebase evolves.

As an optimization, we could skip wakeup_preempt() in local_dsq_post_enq() if
rq->next_class == &ext_sched_class. That should make the performance impact
negligible in the common case, WDYT?

Thanks,
Kuba