[PATCH] NUMA-aware per-cpu allocation

From: Rusty Russell
Date: Fri Jun 04 2004 - 01:21:02 EST


Stolen from x86_64: if we have NUMA, then use alloc_bootmem_node.

Can't hurt, I figure.

In 2.7, my cunning plan is to require contiguous sections, and allocate
similar sections for the dynamic per-cpu allocations, so this won't work
(archs will need to use tricks to create virtually-contiguous memory
from different NUMA nodes). I have a bitrotted implementation.

Meanwhile, this compiles: does it help performance?

Name: Per-CPU Memory On Right Nodes
Status: Untested

Stolen from x86_64: for NUMA machines, allocate the per-cpu memory
using alloc_bootmem_node so it's optimally placed.

diff -urpN --exclude TAGS -X /home/rusty/devel/kernel/kernel-patches/current-dontdiff --minimal .1784-linux-2.6.7-rc2-bk4/include/asm-ppc64/percpu.h .1784-linux-2.6.7-rc2-bk4.updated/include/asm-ppc64/percpu.h
--- .1784-linux-2.6.7-rc2-bk4/include/asm-ppc64/percpu.h 2004-02-04 15:39:11.000000000 +1100
+++ .1784-linux-2.6.7-rc2-bk4.updated/include/asm-ppc64/percpu.h 2004-06-04 14:49:59.000000000 +1000
@@ -1,6 +1,8 @@
#ifndef __ARCH_PPC64_PERCPU__
#define __ARCH_PPC64_PERCPU__

+/* Big sloppy arch needs more per-cpu room than usual */
+#define PERCPU_ENOUGH_ROOM 65536
#include <asm-generic/percpu.h>

#endif /* __ARCH_PPC64_PERCPU__ */
diff -urpN --exclude TAGS -X /home/rusty/devel/kernel/kernel-patches/current-dontdiff --minimal .1784-linux-2.6.7-rc2-bk4/init/main.c .1784-linux-2.6.7-rc2-bk4.updated/init/main.c
--- .1784-linux-2.6.7-rc2-bk4/init/main.c 2004-05-31 09:57:40.000000000 +1000
+++ .1784-linux-2.6.7-rc2-bk4.updated/init/main.c 2004-06-04 14:53:17.000000000 +1000
@@ -338,12 +338,26 @@ static void __init setup_per_cpu_areas(v
size = PERCPU_ENOUGH_ROOM;
#endif

+ /* Multiple allocs for non-NUMA just wastes memory. */
+#ifdef CONFIG_NUMA
+ for (i = 0; i < NR_CPUS; i++) {
+ pg_data_t *pgdat;
+
+ pgdat = NODE_DATA(cpu_to_node(i));
+ ptr = alloc_bootmem_node(pgdat, size);
+ if (!ptr)
+ panic("Cannot allocate cpu data for CPU %ld\n", i);
+ __per_cpu_offset[i] = ptr - __per_cpu_start;
+ memcpy(ptr, __per_cpu_start, __per_cpu_end - __per_cpu_start);
+ }
+#else
ptr = alloc_bootmem(size * NR_CPUS);

for (i = 0; i < NR_CPUS; i++, ptr += size) {
__per_cpu_offset[i] = ptr - __per_cpu_start;
memcpy(ptr, __per_cpu_start, __per_cpu_end - __per_cpu_start);
}
+#endif /* CONFIG_NUMA */
}
#endif /* !__GENERIC_PER_CPU */


--
Anyone who quotes me in their signature is an idiot -- Rusty Russell

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/