Re: Off topic: Numactl "distance" wrong

From: Andi Kleen
Date: Tue Apr 07 2009 - 03:54:21 EST


On Tue, Apr 07, 2009 at 09:40:56AM +0200, Brice Goglin wrote:
> Andi Kleen wrote:
> >> Well, some user-space application may use these distances
> >> to improve their binding.
> >>
> >
> > I'm not aware of any that does.
>
> We have some people here that would like to use it ideally. But they
> know numa distances is almost never available, so they don't really look
> at using them...

>From my experience and from talking at people they tend to have enough
trouble getting the basic NUMA tunings done, without caring about
such (arcane) details.

>
> > If it's true then the correct place would be to fix the BIOS.
> >
>
> Come on, you know it's not going to happen for 99.9% on the existing
> opteron boxes. We have many hardware quirks in the kernel, I don't see
> why this numa distance problem would not deserve its own work around.

The systems where it makes a large difference typically have them anyways.

Anyways if you really want you can ask Len for a way to override SLIT
tables at boot time (similar to the mechanism for MADTs), but I suspect
he wouldn't be particularly enthuiastic. Also it's a little more tricky
than for normal MADTs because SLIT parsing happens very early.

> By the way, anybody looked at this on Nehalem boxes ?

Current Nehalem boxes are all fully connected, so 10/20
(or sometimes scaled to trigger the zone fallback workaround) is the
correct answer and you don't get any benefits from magic in this area.

-Andi

--
ak@xxxxxxxxxxxxxxx -- Speaking for myself only.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/