Re: [PATCH 13/35] autonuma: add page structure fields

[Rik van Riel]

On 05/29/2012 01:38 PM, Linus Torvalds wrote:
> On Tue, May 29, 2012 at 9:38 AM, Andrea Arcangeli<aarcange@xxxxxxxxxx>  wrote:
> > On Tue, May 29, 2012 at 03:16:25PM +0200, Peter Zijlstra wrote:
> > > 24 bytes per page.. or ~0.6% of memory gone. This is far too great a
> > > price to pay.
> >
> > I don't think it's too great, memcg uses for half of that and yet
> > nobody is booting with cgroup_disable=memory even on not-NUMA servers
> > with less RAM.
>
> A big fraction of one percent is absolutely unacceptable.

Andrea, here is a quick back of the envelope idea.

In every zone, we keep an array of pointers to pages and
other needed info for knumad.  We do not need as many as
we have pages in a zone, because we do not want to move
all that memory across anyway (especially in larger systems).
Maybe the number of entries can scale up with the square
root of the zone size?

struct numa_pq_entry {
	struct page *page;
	pg_data_t *destination;
};

For each zone, we can have a numa queueing struct

struct numa_queue {
	struct numa_pq_entry *current_knumad;
	struct numa_pq_entry *current_queue;
	struct numa_pq_entry[];
};

Pages can get added to the knumad queue by filling
in a pointer and a destination node, and by setting
a page flag indicating that this page should be
moved to another NUMA node.

If something happens to the page that would cancel
the queuing, we simply clear that page flag.

When knumad gets around to an entry in the array,
it will check to see if the "should migrate" page
flag is still set. If it is not, it skips the entry.

The current_knumad and current_queue entries can
be used to simply keep circular buffer semantics.

Does this look reasonable?

--
All rights reversed
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/