Re: [PATCH v2] sched_ext: Clarify CPU context for running/stopping callbacks

[Changwoo Min]

Hi Andrea,

On 4/24/25 14:26, Andrea Righi wrote:
> Hi Changwoo,
>
> On Thu, Apr 24, 2025 at 08:06:47AM +0900, Changwoo Min wrote:
> > Hi Andrea,
> >
> > On 4/24/25 06:02, Andrea Righi wrote:
> > > The ops.running() and ops.stopping() callbacks can be invoked from a CPU
> > > other than the one the task is assigned to, particularly when a task
> > > property is changed, as both scx_next_task_scx() and dequeue_task_scx() may
> > > run on CPUs different from the task's target CPU.
> >
> > The same goes to ops.quiescent() too since ops.quiescent() is also
> > called from dequeue_task_scx().
>
> Yeah, I was a bit conflicted about mentioning this for ops.runnable() and
> ops.quiescent() as well, since it's more obvious in those cases that
> they're executed outside the context of the "current CPU", since the task
> isn't running on any CPU yet, or it's no longer running. In the end, I
> decided to update only ops.running() and ops.stopping(), where it's less
> clear that the task's CPU may not match the current CPU.

That makes sense. Thanks for the clarification!

-- Changwoo

> Thanks for taking a look!
> -Andrea
>
> > Reviewed-by: Changwoo Min <changwoo@xxxxxxxxxx>
> >
> > Regards,
> > Changwoo Min
> >
> > > This behavior can lead to confusion or incorrect assumptions if not
> > > properly clarified, potentially resulting in bugs (see [1]).
> > >
> > > Therefore, update the documentation to clarify this aspect and advise
> > > users to use scx_bpf_task_cpu() to determine the actual CPU the task
> > > will run on or was running on.
> > >
> > > [1] https://github.com/sched-ext/scx/pull/1728
> > >
> > > Cc: Jake Hillion <jake@xxxxxxxxxxxxx>
> > > Cc: Changwoo Min <changwoo@xxxxxxxxxx>
> > > Signed-off-by: Andrea Righi <arighi@xxxxxxxxxx>
> > > ---
> > >    kernel/sched/ext.c | 18 ++++++++++++++++++
> > >    1 file changed, 18 insertions(+)
> > >
> > > Changes in v2:
> > >    - clarify the scenario a bit more in the code comments
> > >    - link to v1: https://lore.kernel.org/all/20250423190059.270236-1-arighi@xxxxxxxxxx/
> > >
> > > diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c
> > > index ac79067dc87e6..a83232a032aa4 100644
> > > --- a/kernel/sched/ext.c
> > > +++ b/kernel/sched/ext.c
> > > @@ -368,6 +368,15 @@ struct sched_ext_ops {
> > >    	 * @running: A task is starting to run on its associated CPU
> > >    	 * @p: task starting to run
> > >    	 *
> > > +	 * Note that this callback may be called from a CPU other than the
> > > +	 * one the task is going to run on. This can happen when a task
> > > +	 * property is changed (i.e., affinity), since scx_next_task_scx(),
> > > +	 * which triggers this callback, may run on a CPU different from
> > > +	 * the task's assigned CPU.
> > > +	 *
> > > +	 * Therefore, always use scx_bpf_task_cpu(@p) to determine the
> > > +	 * target CPU the task is going to use.
> > > +	 *
> > >    	 * See ->runnable() for explanation on the task state notifiers.
> > >    	 */
> > >    	void (*running)(struct task_struct *p);
> > > @@ -377,6 +386,15 @@ struct sched_ext_ops {
> > >    	 * @p: task stopping to run
> > >    	 * @runnable: is task @p still runnable?
> > >    	 *
> > > +	 * Note that this callback may be called from a CPU other than the
> > > +	 * one the task was running on. This can happen when a task
> > > +	 * property is changed (i.e., affinity), since dequeue_task_scx(),
> > > +	 * which triggers this callback, may run on a CPU different from
> > > +	 * the task's assigned CPU.
> > > +	 *
> > > +	 * Therefore, always use scx_bpf_task_cpu(@p) to retrieve the CPU
> > > +	 * the task was running on.
> > > +	 *
> > >    	 * See ->runnable() for explanation on the task state notifiers. If
> > >    	 * !@runnable, ->quiescent() will be invoked after this operation
> > >    	 * returns.