Re: tutorial question: where mb() is needed?

Ryan Moore (rmoore@rmoore.i.seawood.org)
Wed, 6 Jan 1999 22:59:10 -0800 (PST)

On Wed, 6 Jan 1999, Andrea Arcangeli wrote:
> I looked at the new SMP rescheduling code in pre2.2.0. I think to
> understand everything except a wmb(). I don't understand very well why
> wmb() is needed there:
>
> if (sched_data->prev->state != sched_data->prevstate)
> current->need_resched = 1;
>
> /*
> * Release the previous process ...
> *
> * We have dropped all locks, and we must make sure that we
> * only mark the previous process as no longer having a CPU
> * after all other state has been seen by other CPU's. Thus
> * the write memory barrier!
> */
> wmb();
> sched_data->prev->has_cpu = 0;
> #endif /* __SMP__ */
>
> I think to know what mb()/rmb()/wmb() (I also seen that the function names
> are taken by the Alpha asm instruction ;) do, but I don't understand very
> well which are the real world SMP/UP cases we need it...

Memory barriers are important on the Alpha, even in the UP case. The purpose
of a memory barrier is to ensure that all previous memory write operations
have been flushed before performing new write operations. The biggest reason
to do this is to change some kind of flag. Memory barriers aren't really
synchronization mechasims directly, but they are used in the synchronization
process.

> Is there an example somewhere of a minimal multithreaded or device-driver
> code that needs mb() just to see the point?

I saw other drivers where mb() was used. But it wasn't in as much code
as I suspect it should be in.

> And why don't we use SMP-atomic operation (using lock) instead of use
> mb() to avoid races? I know I am missing something somewhere about mb()...

I can't talk reliably about the internals of the schedule, but I can tell
you what's going on in that code sequence.

My guess is that the "has_cpu" flag probably indicates if the process
is running on a CPU. Presumably, if has_cpu is 0, the sceduler running
on another CPU won't look at the other information about that process.
Therefore, if the scheduler on CPU 1 is messing with that structure, then it
needs to be sure all of the updates have happened before setting the "has_cpu"
flag which would allow the scheduler on another processor to look at the
information. Obviously it's important that all of the data about the process
be up-to-date in memory before the scheduler on another CPU takes a closer
look.

Another CPU could see the "has_cpu" flag as zero before all of the other
information in the structure is updated. This is bad.

Presumably all of this code isn't in a locked section for performance
reasons. Instead, the scheduler is relying on the fact that the scheduler
on another processor won't mess with this current process unless the
has_cpu is zero.

Get the idea, yet?

Apologies if this explanation isn't quite right.

--------------------------------
Ryan Moore rmoore@rmoore.i.seawood.org

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/