Re: [RFC PATCH] ARM: smp: Fix the CPU hotplug race with scheduler.

From: Santosh Shilimkar
Date: Mon Jun 20 2011 - 07:52:10 EST

On 6/20/2011 5:10 PM, Russell King - ARM Linux wrote:
On Mon, Jun 20, 2011 at 04:55:43PM +0530, Santosh Shilimkar wrote:
On 6/20/2011 4:43 PM, Russell King - ARM Linux wrote:
On Mon, Jun 20, 2011 at 04:17:58PM +0530, Santosh Shilimkar wrote:
Yes. It's because of interrupt and the CPU active-online

I don't see that as a conclusion from this dump.

Here is the chash log..
[ 21.025451] CPU1: Booted secondary processor
[ 21.025451] CPU1: Unknown IPI message 0x1
[ 21.029113] Switched to NOHz mode on CPU #1
[ 21.029174] BUG: spinlock lockup on CPU#1, swapper/0, c06220c4

That's the xtime seqlock. We're trying to update the xtime from CPU1,
which is not yet online and not yet active. That's fine, we're just
spinning on the spinlock here, waiting for the other CPUs to release

But what this is saying is that the other CPUs aren't releasing it.
The cpu hotplug code doesn't hold the seqlock either. So who else is
holding this lock, causing CPU1 to time out on it.

The other thing is that this is only supposed to trigger after about
one second:

u64 loops = loops_per_jiffy * HZ;
for (i = 0; i< loops; i++) {
if (arch_spin_trylock(&lock->raw_lock))

which from the timings you have at the beginning of your printk lines
is clearly not the case - it's more like 61us.

Are you running with those h/w timer delay patches?

Ok. So loops_per_jiffy must be too small. My guess is you're using an
older kernel without 71c696b1 (calibrate: extract fall-back calculation
into own helper).

I am on V3.0-rc3+(latest mainline) and the above commit is already
part of it.

The delay calibration code used to start out by setting:

loops_per_jiffy = (1<<12);

This will shorten the delay right down, and that's probably causing these
false spinlock lockup bug dumps.

Arranging for IRQs to be disabled across the delay calibration just avoids
the issue by preventing any spinlock being taken.

The reason that CPU#0 also complains about spinlock lockup is that for
some reason CPU#1 never finishes its calibration, and so the loop also
times out early on CPU#0.

I am not sure but what I think is happening is as soon as interrupts
start firing, as part of IRQ handling, scheduler will try to
enqueue softIRQ thread for newly booted CPU since it sees that
it's active and ready. But that's failing and both CPU's
eventually lock-up. But I may be wrong here.

Of course, fiddling with this global variable in this way is _not_ a good
idea while other CPUs are running and using that variable.

We could also do with implementing trigger_all_cpu_backtrace() to get
backtraces from the other CPUs when spinlock lockup happens...

Any pointers on the other question about "why we need to enable
interrupts before the CPU is ready?"


To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at
Please read the FAQ at