Re: [PATCH 0/2] nohz: Deal with clock reprogram skipping issues v3

From: Ingo Molnar
Date: Fri May 26 2017 - 02:13:30 EST



* Frederic Weisbecker <fweisbec@xxxxxxxxx> wrote:

> On Wed, May 24, 2017 at 09:16:28AM +0200, Ingo Molnar wrote:
> > So the interdiff between your two patches and the 3 commits already queued up is:
> >
> > diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
> > index e3043873fcdc..30253ed0380b 100644
> > --- a/kernel/time/tick-sched.c
> > +++ b/kernel/time/tick-sched.c
> > @@ -150,12 +150,6 @@ static void tick_sched_handle(struct tick_sched *ts, struct pt_regs *regs)
> > touch_softlockup_watchdog_sched();
> > if (is_idle_task(current))
> > ts->idle_jiffies++;
> > - /*
> > - * In case the current tick fired too early past its expected
> > - * expiration, make sure we don't bypass the next clock reprogramming
> > - * to the same deadline.
> > - */
> > - ts->next_tick = 0;
> > }
> > #endif
> > update_process_times(user_mode(regs));
> > @@ -1103,8 +1097,15 @@ static void tick_nohz_handler(struct clock_event_device *dev)
> > tick_sched_handle(ts, regs);
> >
> > /* No need to reprogram if we are running tickless */
> > - if (unlikely(ts->tick_stopped))
> > + if (unlikely(ts->tick_stopped)) {
> > + /*
> > + * In case the current tick fired too early past its expected
> > + * expiration, make sure we don't bypass the next clock reprogramming
> > + * to the same deadline.
> > + */
> > + ts->next_tick = 0;
> > return;
> > + }
> >
> > hrtimer_forward(&ts->sched_timer, now, tick_period);
> > tick_program_event(hrtimer_get_expires(&ts->sched_timer), 1);
> > @@ -1202,12 +1203,17 @@ static enum hrtimer_restart tick_sched_timer(struct hrtimer *timer)
> > */
> > if (regs)
> > tick_sched_handle(ts, regs);
> > - else
> > - ts->next_tick = 0;
> >
> > /* No need to reprogram if we are in idle or full dynticks mode */
> > - if (unlikely(ts->tick_stopped))
> > + if (unlikely(ts->tick_stopped)) {
> > + /*
> > + * In case the current tick fired too early past its expected
> > + * expiration, make sure we don't bypass the next clock reprogramming
> > + * to the same deadline.
> > + */
> > + ts->next_tick = 0;
> > return HRTIMER_NORESTART;
> > + }
> >
> > hrtimer_forward(timer, now, tick_period);
> >
> >
> > ... so the two are not the same - I'd rather not rebase it, I'd like to keep what
> > is working, we had problems with these changes before ...
> >
> > If you'd like the changes in this interdiff to be applied as well, please add a
> > changelog to it and post it as a fourth patch.
> >
> > Thanks,
> >
> > Ingo
>
> So if you like, you can replace the top patch with the following. It's exactly
> the same code, I've only added a comment and a changelog:
>
> ---
> From 72956bf08c3b2e506a5ce5ec4faac9fd6b097307 Mon Sep 17 00:00:00 2001
> From: Frederic Weisbecker <fweisbec@xxxxxxxxx>
> Date: Mon, 15 May 2017 14:56:50 +0200
> Subject: [PATCH] nohz: Reset next_tick cache even when the timer has no regs
>
> The tick IRQ regs can be NULL if hrtimer_interrupt() is called from
> non-interrupt contexts (ex: hotplug CPU down). For such very special
> path we forget to clean the cached next tick deadline. If we are in
> dynticks mode and the actual timer deadline is ahead of us, we might
> perform a buggy bypass of the next clock reprogramming.
>
> In fact since CPU down is the only user I'm aware of, this fix is likely
> unnecessary as dying CPUs already clean their tick deadline cache. But
> given how hard it is to debug such timer cache related issue, we should
> never be short on paranoid measures.
>
> Signed-off-by: Frederic Weisbecker <fweisbec@xxxxxxxxx>
> Cc: Linus Torvalds <torvalds@xxxxxxxxxxxxxxxxxxxx>
> Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
> Cc: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
> Signed-off-by: Ingo Molnar <mingo@xxxxxxxxxx>
> ---
> kernel/time/tick-sched.c | 11 ++++++++++-
> 1 file changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
> index 764d290..ed18ca5 100644
> --- a/kernel/time/tick-sched.c
> +++ b/kernel/time/tick-sched.c
> @@ -1200,8 +1200,17 @@ static enum hrtimer_restart tick_sched_timer(struct hrtimer *timer)
> * Do not call, when we are not in irq context and have
> * no valid regs pointer
> */
> - if (regs)
> + if (regs) {
> tick_sched_handle(ts, regs);
> + } else {
> + /*
> + * IRQ regs are NULL if hrtimer_interrupt() is called from
> + * non-interrupt contexts (ex: hotplug cpu down). Make sure to
> + * clean the cached next tick deadline to avoid buggy bypass of
> + * clock reprog.
> + */
> + ts->next_tick = 0;
> + }
>
> /* No need to reprogram if we are in idle or full dynticks mode */
> if (unlikely(ts->tick_stopped))

Well, this does not answer my question: between latest tip:timers/nohz and the
patches you posted there's a delta, so it's not just a pure rebase.

I can do a rebase to resolve the bisectability problem (which isn't very serious
by the way, only a single commit wide window, right?), but only if 'git diff
old_branch new_branch' comes up empty.

In every other case let's iterate the existing timers/nohz with additional
patches, ok? I'd rather have a finegrained iteration with well-tested intermediate
stages than break things again.

Thanks,

Ingo