Re: [External] Re: [PATCH v4] sched/core: Adapt WARN_DOUBLE_CLOCK machinery for core-sched

[Hao Jia]

On 2023/5/4 Peter Zijlstra wrote:
> On Thu, Apr 06, 2023 at 02:44:15PM +0800, Hao Jia wrote:
> > diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> > index 0d18c3969f90..c6e2c79152ef 100644
> > --- a/kernel/sched/core.c
> > +++ b/kernel/sched/core.c
> > @@ -429,11 +429,32 @@ void sched_core_put(void)
> >   		schedule_work(&_work);
> >   }
> >   
> > +/*
> > + * Now, we have obtained a core-wide rq->lock, then we need to clear
> > + * RQCF_UPDATED of rq->clock_update_flags of the sibiling CPU
> > + * on this core to avoid the WARN_DOUBLE_CLOCK warning.
> > + */
> > +static inline void sched_core_clear_rqcf_updated(struct rq *rq)
> > +{
> > +#ifdef CONFIG_SCHED_DEBUG
> > +	const struct cpumask *smt_mask;
> > +	int i;
> > +
> > +	if (rq->core_enabled) {
> > +		smt_mask = cpu_smt_mask(rq->cpu);
> > +		for_each_cpu(i, smt_mask) {
> > +			if (rq->cpu != i)
> > +				cpu_rq(i)->clock_update_flags &= (RQCF_REQ_SKIP|RQCF_ACT_SKIP);
> > +		}
> > +	}
> > +#endif
> > +}
> >   #else /* !CONFIG_SCHED_CORE */
> >   
> >   static inline void sched_core_enqueue(struct rq *rq, struct task_struct *p) { }
> >   static inline void
> >   sched_core_dequeue(struct rq *rq, struct task_struct *p, int flags) { }
> > +static inline void sched_core_clear_rqcf_updated(struct rq *rq) { }
> >   
> >   #endif /* CONFIG_SCHED_CORE */
> >   
> > @@ -548,6 +569,7 @@ void raw_spin_rq_lock_nested(struct rq *rq, int subclass)
> >   		if (likely(lock == __rq_lockp(rq))) {
> >   			/* preempt_count *MUST* be > 1 */
> >   			preempt_enable_no_resched();
> > +			sched_core_clear_rqcf_updated(rq);
> >   			return;
> >   		}
> >   		raw_spin_unlock(lock);
>
> This still looks absolutely wrong. The whole RQCF thing is a pin action.

Do you think it is better for us to extend rq_pin_lock() to clean RQCF 
updated than to do it in raw_spin_rq_lock_nested()?

Before doing this, we need to solve the situation where rq_pin_lock() 
and raw_spin_rq_lock() are used separately.

Any suggestion will be very helpful for me.

Thanks,
Hao