Re: Lets get this right (WAS RE:MOSIX and kernel mods)

Richard Gooch (rgooch@atnf.csiro.au)
Sun, 7 Mar 1999 18:59:06 +1100


Larry McVoy writes:
> To: Richard Gooch <rgooch@atnf.csiro.au>
> : If you hand me a network with computers and network orders of
> : magnitude faster, do you think I'm going to keep doing the same thing?
> : No way! I'm going to want render times under 30 milliseconds and
> : datasets of a few hundred MBytes. I'm still going to care that
> : grabbing a remote lock is going to be 4+ orders of magnitude more
> : expensive than grabbing a local lock. So I'm still going to do SMP for
> : the local nodes and MPI for the remote nodes, because I need to think
> : about the two cases in a different way. I use locks because they're
> : fast and there are benefits to using them and they're neater; but when
> : locks would be slow, I do things a different way, and there the locks
> : just get in the way: MPI is neater.
>
> I'm really giving myself a bad rep here, because I have to keep
> agreeing with Richard :-) But hey, when he's right, he's right, what
> can I do :-)

<blush>

> Anyway, to make his point even stronger, the message passing
> interfaces like PVM and MPI can also be optimized for the local
> case, in fact, SGI does that - they have MPI libraries that know
> when the "remote" node is actually local and then just use shared
> memory for that message.

Just how smart have they made this optimisation, BTW? For the volume
rendering application, you have two conceptual transfers of data: the
volume data is "shipped" to worker nodes and they in turn "ship"
subimage data back to the scheduling node.

In the threaded case, there is no transfer of data required. The
volume data is obviously shared, but so is the rendered image
buffer. All threads write to that same buffer, but since they write to
different parts of it, there's no conflict.

So what happens for local MPI? Is data copied twice (into and out of a
SHM buffer), once (into a SHM buffer which is passed to the
destination), or not at all? Not coping at all is easy enough if you
know the destination is another thread in the same process, but
doesn't that break the abstraction of receiving a message in a private
buffer?

Granted, in my case it wouldn't matter, but I wonder about the general
case.

Regards,

Richard....

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/