Re: [PATCH] sched/fair: sanitize vruntime of entity being placed

From: Zhang Qiao
Date: Sat Jan 28 2023 - 20:27:40 EST


Hi, Chenyu,

在 2023/1/29 1:27, Chen Yu 写道:
> Hi Roman, Qiao,
> On 2023-01-27 at 17:32:30 +0100, Roman Kagan wrote:
>> From: Zhang Qiao <zhangqiao22@xxxxxxxxxx>
>>
>> When a scheduling entity is placed onto cfs_rq, its vruntime is pulled
>> to the base level (around cfs_rq->min_vruntime), so that the entity
>> doesn't gain extra boost when placed backwards.
>>
>> However, if the entity being placed wasn't executed for a long time, its
>> vruntime may get too far behind (e.g. while cfs_rq was executing a
>> low-weight hog), which can inverse the vruntime comparison due to s64
>> overflow. This results in the entity being placed with its original
>> vruntime way forwards, so that it will effectively never get to the cpu.
>>
> Looks interesting,
> case 1:
> se->vruntime = 1, cfs_rq->min_vruntime = ULONG_MAX
> ==> max = 1
> case 2:
> se->vruntime = 1, cfs_rq->min_vruntime = LONG_MAX
> ==> max = LONG_MAX
>
> May I know if the issue you described above is in case 1? We want
> the max to be ULONG_MAX but it returns 1 because of s64
> comparison? Then max = 1 is incorrectly used as se's vruntime?
> Could you please elaborate a little more about this issue?

Yes, the issue is in case 1.

For more detailed discussion, can see https://lkml.org/lkml/2022/12/21/435.

>> To prevent that, ignore the vruntime of the entity being placed if it
>> didn't execute for much longer than the characteristic sheduler time
>> scale.
>>
>> [rkagan: formatted, adjusted commit log, comments, cutoff value]
>> Co-developed-by: Roman Kagan <rkagan@xxxxxxxxx>
>> Signed-off-by: Roman Kagan <rkagan@xxxxxxxxx>
>> ---
>> @zhangqiao22, I took the liberty to put you as the author of the patch,
>> as this is essentially what you posted for discussion, with minor
>> tweaks. Please stamp with your s-o-b if you're ok with it.
>>
>> kernel/sched/fair.c | 15 +++++++++++++--
>> 1 file changed, 13 insertions(+), 2 deletions(-)
>>
>> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
>> index 0f8736991427..d6cf131ebb0b 100644
>> --- a/kernel/sched/fair.c
>> +++ b/kernel/sched/fair.c
>> @@ -4656,6 +4656,7 @@ static void
>> place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial)
>> {
>> u64 vruntime = cfs_rq->min_vruntime;
>> + u64 sleep_time;
>>
>> /*
>> * The 'current' period is already promised to the current tasks,
>> @@ -4685,8 +4686,18 @@ place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial)
>> vruntime -= thresh;
>> }
>>
>> - /* ensure we never gain time by being placed backwards. */
>> - se->vruntime = max_vruntime(se->vruntime, vruntime);
>> + /*
>> + * Pull vruntime of the entity being placed to the base level of
>> + * cfs_rq, to prevent boosting it if placed backwards. If the entity
>> + * slept for a long time, don't even try to compare its vruntime with
>> + * the base as it may be too far off and the comparison may get
>> + * inversed due to s64 overflow.
>> + */
>> + sleep_time = rq_clock_task(rq_of(cfs_rq)) - se->exec_start;
> If I understand correctly, se->exec_start is just updated by enqueue_entity()->update_curr(cfs_rq),

When a task go to sleep, se->exec_start will update at dequeue_entity()->update_curr(cfs_rq).
And enqueue_entity()->update_curr(cfs_rq) just update current se.

Thank,
Qiao.

> then place_entity() in invoked here, I'm not sure if sleep_time above
> could reflect the real sleep time. Maybe something like:
> rq_clock_task(rq_of(cfs_rq)) - se->time_stamp_dequeued ?
>
> thanks,
> Chenyu
> .
>