Re: [PATCH 1/8] [watchdog] combine nmi_watchdog and softlockup

From: Don Zickus
Date: Wed Apr 28 2010 - 16:28:54 EST


On Wed, Apr 28, 2010 at 02:36:54PM +0200, Frederic Weisbecker wrote:
> On Fri, Apr 23, 2010 at 12:13:29PM -0400, Don Zickus wrote:
> > +void watchdog_overflow_callback(struct perf_event *event, int nmi,
> > + struct perf_sample_data *data,
> > + struct pt_regs *regs)
> > +{
> > + int this_cpu = smp_processor_id();
> > + unsigned long touch_ts = per_cpu(watchdog_touch_ts, this_cpu);
> > + char warn = __get_cpu_var(watchdog_warn);
> > +
> > + if (touch_ts == 0) {
> > + __touch_watchdog();
> > + return;
> > + }
> > +
> > + /* check for a hardlockup
> > + * This is done by making sure our timer interrupt
> > + * is incrementing. The timer interrupt should have
> > + * fired multiple times before we overflow'd. If it hasn't
> > + * then this is a good indication the cpu is stuck
> > + */
> > + if (is_hardlockup(this_cpu)) {
> > + /* only print hardlockups once */
> > + if (warn & HARDLOCKUP)
> > + return;
> > +
> > + if (hardlockup_panic)
> > + panic("Watchdog detected hard LOCKUP on cpu %d", this_cpu);
> > + else
> > + WARN(1, "Watchdog detected hard LOCKUP on cpu %d", this_cpu);
> > +
> > + __get_cpu_var(watchdog_warn) = warn | HARDLOCKUP;
> > + return;
> > + }
> > +
> > + __get_cpu_var(watchdog_warn) = warn & ~HARDLOCKUP;
> > + return;
> > +}
> [...]
> > +static enum hrtimer_restart watchdog_timer_fn(struct hrtimer *hrtimer)
> > +{
> > + int this_cpu = smp_processor_id();
> > + unsigned long touch_ts = __get_cpu_var(watchdog_touch_ts);
> > + char warn = __get_cpu_var(watchdog_warn);
> > + struct pt_regs *regs = get_irq_regs();
> > + int duration;
> > +
> > + /* kick the hardlockup detector */
> > + watchdog_interrupt_count();
> > +
> > + /* kick the softlockup detector */
> > + wake_up_process(__get_cpu_var(softlockup_watchdog));
> > +
> > + /* .. and repeat */
> > + hrtimer_forward_now(hrtimer, ns_to_ktime(get_sample_period()));
> > +
> > + if (touch_ts == 0) {
> > + __touch_watchdog();
> > + return HRTIMER_RESTART;
> > + }
> > +
> > + /* check for a softlockup
> > + * This is done by making sure a high priority task is
> > + * being scheduled. The task touches the watchdog to
> > + * indicate it is getting cpu time. If it hasn't then
> > + * this is a good indication some task is hogging the cpu
> > + */
> > + duration = is_softlockup(touch_ts, this_cpu);
> > + if (unlikely(duration)) {
> > + /* only warn once */
> > + if (warn & SOFTLOCKUP)
> > + return HRTIMER_RESTART;
> > +
> > + printk(KERN_ERR "BUG: soft lockup - CPU#%d stuck for %us! [%s:%d]\n",
> > + this_cpu, duration,
> > + current->comm, task_pid_nr(current));
> > + print_modules();
> > + print_irqtrace_events(current);
> > + if (regs)
> > + show_regs(regs);
> > + else
> > + dump_stack();
> > +
> > + if (softlockup_panic)
> > + panic("softlockup: hung tasks");
> > + __get_cpu_var(watchdog_warn) = warn | SOFTLOCKUP;
> > + } else
> > + __get_cpu_var(watchdog_warn) = warn & ~SOFTLOCKUP;
>
>
> Note these watchdog_warn modifications are racy against the same that
> happens with HARDLOCKUP. You might clear what did the nmi.
>
> The race is harmless enough that we don't care much I think, but that's
> why it would have make sense to separate watchdog_warn tracking space
> between both.

Heh. Good point. I'll respin.

Cheers,
Don

>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/