Re: [PATCH -v2] use per cpu data for single cpu ipi calls

[Peter Zijlstra]

On Thu, 2009-01-29 at 09:21 -0800, Linus Torvalds wrote:
> On Thu, 29 Jan 2009, Steven Rostedt wrote:
> > 
> > The caller must wait till the LOCK bit is cleared before setting
> > it. When it is cleared, there is no IPI function using it.
> > A spinlock is used to synchronize the setting of the bit between
> > callers. Since only one callee can be called at a time, and it
> > is the only thing to clear it, the IPI does not need to use
> > any locking.
> 
> That spinlock cannot be right. It is provably wrong for so many reasons..
> 
> Think about it. We're talking about a per-CPU lock, which already makes no 
> sense: we're only locking against our own CPU, and we've already disabled 
> preemption for totally unrelated reasons.
> 
> And the only way locking can make sense against our own CPU is if we lock 
> against interrupts - but the lock isn't actually irq-safe, so if you are 
> trying to lock against interrupts, you are (a) doing it wrong (you should 
> disable interrupts, not use a spinlock) and (b) causing a deadlock if it 
> ever happens.


> +                       else {
> +                               data = &per_cpu(csd_data, cpu);
> +                               spin_lock(&per_cpu(csd_data_lock, cpu));
> +                               while (data->flags & CSD_FLAG_LOCK)
> +                                       cpu_relax();
> +                               data->flags = CSD_FLAG_LOCK;
> +                               spin_unlock(&per_cpu(csd_data_lock, cpu));
> +                       }

I think your argument would hold if he did:

  data = &__get_cpu_var(csd_data);

But now he's actually grabbing the remote cpu's csd, and thus needs
atomicy around that remote csd -- which two cpus could contend for.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/