Re: sched/deadline: Use revised wakeup rule for dl_server

[Christian Loehle]

On 5/9/26 12:42, Andreas Ziegler wrote:
> Hi Christian, Everyone,
> 
> On 2026-05-08 14:13, Christian Loehle wrote:
>> On 5/8/26 13:06, Andreas Ziegler wrote:
>>> Hi Christian,
>>>
>>> On 2026-05-08 09:20, Christian Loehle wrote:
>>>> On 5/8/26 09:09, Andreas Ziegler wrote:
>>>>> Linux kernel version: 6.12
>>>>>   CONFIG_PREEMPT_RT (w/ PREEMPT_RT patch applied)
>>>>> Architecture: aarch64
>>>>> Platform: Raspberry Pi 4
>>>>>
>>>>> Hi everyone,
>>>>>
>>>>> Commit d66792919d4f (sched/deadline: Use revised wakeup rule for dl_server) [1] introduced a marked degradation in scheduling latency for real-time tasks in the presence of heavy I/O load.
>>>>>
>>>>> --- a/kernel/sched/deadline.c
>>>>> +++ b/kernel/sched/deadline.c
>>>>> @@ -1079,7 +1079,7 @@ static void update_dl_entity(struct sched_dl_entity *dl_se)
>>>>>      if (dl_time_before(dl_se->deadline, rq_clock(rq)) ||
>>>>>          dl_entity_overflow(dl_se, rq_clock(rq))) {
>>>>>
>>>>> -        if (unlikely(!dl_is_implicit(dl_se) &&
>>>>> +        if (unlikely((!dl_is_implicit(dl_se) || dl_se->dl_defer) &&
>>>>>                   !dl_time_before(dl_se->deadline, rq_clock(rq)) &&
>>>>>                   !is_dl_boosted(dl_se))) {
>>>>>              update_dl_revised_wakeup(dl_se, rq);
>>>>>
>>>>> This was observed using a modified version of Con Kolivas' interactivity benchmark [2]; kernel bisection eventually pointed to the above mentioned commit.
>>>>>
>>>>> Benchmark results before d66792919d4f:
>>>>>
>>>>> --- Benchmarking simulated cpu of Audio real time in the presence of simulated ---
>>>>> Load    Latency +/- SD   median  max [100n]    Desired CPU  Deadlines met [%]
>>>>> None      76.6 +/- 8.3654    76  166
>>>>> Video      78.5 +/- 3.9433    78  107
>>>>> X      76.4 +/- 8.123     75  157
>>>>> Burn      72.0 +/- 6.4733    71  127
>>>>> Write     255.3 +/- 26.627   252  331
>>>>> Read     226.6 +/- 12.38    227  262
>>>>> Ring      84.2 +/- 6.6207    83  125
>>>>> Compile     225.3 +/- 23.949   222  328
>>>>>
>>>>>      136.8 +/- 78.462        331
>>>>>
>>>>> Benchmark results after d66792919d4f:
>>>>>
>>>>> --- Benchmarking simulated cpu of Audio real time in the presence of simulated ---
>>>>> Load    Latency +/- SD   median  max [100n]    Desired CPU  Deadlines met [%]
>>>>> None      68.4 +/- 9.7864    67  169
>>>>> Video      74.4 +/- 3.724     74   97
>>>>> X      72.0 +/- 6.5681    71  129
>>>>> Burn      66.9 +/- 5.9059    66  117
>>>>> Write    9576.9 +/- 67639    250500418        98.1         98.1
>>>>> Read     209.3 +/- 11.018   209  267
>>>>> Ring      80.5 +/- 8.0993    78  125
>>>>> Compile     239.0 +/- 29.447   234  372
>>>>>
>>>>>     1298.4 +/- 24118       500418
>>>>>
>>>>> Reverting this commit obviously solves the issue for me. I have no idea why this issue appears exclusively with heavy write loads in the background.
>>>>>
>>>>> Is this a scheduler issue, or rather something in the background?
>>>>>
>>>>
>>>> Hi Andreas,
>>>> You're using cpufreq schedutil for your tests I'm assuming?
>>>> Is there a difference in cpufreq behavior (avg cpufreq or OPP residencies?)
>>>> Does the regression also happen on powersave/performance governor?
>>>
>>> Actually this is a very stripped-down system. The 'performance' cpufreq governor is the only one compiled in, the processor cores run on a fixed frequency. CONFIG_PM_OPP is not set.
>>
>> That certainly makes the analysis easier.
>> I couldn't reproduce the issue so far on my system but it does seem like the dl server
>> would get potentially unbounded running time with very frequent
>> starting and stopping of the dlserver (which presumably happens because of
>> the writeback) reset the runtime, which then leads to your 25s observed latency.
>> Peter, how is the revised wakeup rule supposed to behave here?
>>
>>> [snip]
> 
> This seems to be a case of runtime starvation. If I change sched_rt_runtime_us to a smaller value, the benchmark returns reasonable latency values.
> 
> # echo "980000" > /proc/sys/kernel/sched_rt_runtime_us
> 
> I could live with this workaround, since it seems not to impact overall latency values in a noticeable way.
> 

Not a very stable workaround unfortunately :/
While I try to reproduce this, what you're observing should imply that the
background SCHED_NORMAL work is enough to fully utilize the system, right?
interbench Write does 4k (buffered) writes of a 1GB file and then close+open
and repeat, nothing fancy really. Does this actually produce significant CPU
utilization for you? Can you just run the background work and see what that
looks like?
(What you're seeing looks like a bug in any case, just so I'm not going down
a wrong path when trying to reproduce here).