Re: Fw: [PATCH] sunrpc: use better NUMA affinities
From: Greg Banks
Date: Fri Jul 29 2011 - 02:05:51 EST
On 29/07/11 15:32, NeilBrown wrote:
Hi Greg,
I saw this patch float past and thought of you... You may not be interested
any more, and it may be a perfectly good patch that does not need any
comment, but I thought I would let you know anyway.
Thanks Neil.
I've trimmed the cc list to limit the number of copies Trond and Bruce get:)
From: Eric Dumazet<eric.dumazet@xxxxxxxxx>
To: Trond Myklebust<Trond.Myklebust@xxxxxxxxxx>
Cc: "J. Bruce Fields"<bfields@xxxxxxxxxxxx>, Neil Brown<neilb@xxxxxxx>,
David Miller<davem@xxxxxxxxxxxxx>, linux-nfs@xxxxxxxxxxxxxxx, netdev
<netdev@xxxxxxxxxxxxxxx>, linux-kernel<linux-kernel@xxxxxxxxxxxxxxx>
Subject: [PATCH] sunrpc: use better NUMA affinities
Use NUMA aware allocations to reduce latencies and increase throughput.
Briefly looking at the patch, it doesn't seem wrong but I'm surprised
it's (still) necessary.
Some years ago at SGI we encountered that same problem; we solved it by
delaying all the allocation of data structures associated with a thread
so that they were performed in the thread itself, after the thread had
been limited to run on a certain set of CPUs. Thus the thread's normal
allocation behaviour resulted in all of it's allocations being from
node-local pages. It was a pretty ugly patch, but it worked and made a
huge difference to NFS throughput on large NUMA boxes.
Later Jeff Layton converted the sunrpc svc startup code to use kthreads
and at the time I read his patches, pointed out this problem, and posted
my patch for comparison
http://linux-nfs.org/pipermail/nfsv4/2008-May/008760.html
I seem to remember coming to the conclusion that Jeff eventually
addressed this problem...am I misremembering or did something regress?
--
Greg.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/