Re: Page Colouring (was: 2.6.0 Huge pages not working as expected)

[Linus Torvalds]

On Sat, 27 Dec 2003, Eric W. Biederman wrote:
> Linus Torvalds <torvalds@xxxxxxxx> writes:
> >
> > Basically: prove me wrong. People have tried before. They have failed. 
> > Maybe you'll succeed. I doubt it, but hey, I'm not stopping you.
> 
> For anyone taking you up on this I'd like to suggest two possible
> directions.
> 
> 1) Increasing PAGE_SIZE in the kernel.

Yes. This is something I actually want to do anyway for 2.7.x. Dan 
Phillips had some patches for this six months ago.

You have to be careful, since you have to be able to mmap "partial pages", 
which is what makes it less than trivial, but there are tons of reasons to 
want to do this, and cache coloring is actually very much a secondary 
concern.

> 2) Creating zones for the different colors.  Zones were not
>    implemented last time, this was tried.

Hey, I can tell you that you _will_ fail.

Zones are actually a wonderful example of the kinds of problems you get
into when you have pages of different types aka "colors". We've had
nothing but trouble trying to balance different zones against each other,
and those problems were in fact _the_ reason for 99% of all the VM
problems in 2.4.x.

Trying to use them for cache colors would be "interesting". 

Not to mention that it's impossible to coalesce pages across zones.

> Both of those should be minimal impact to the complexity
> of the current kernel. 

Minimal? I don't think so. Zones are basically impossible, and page size 
changes will hopefully happen during 2.7.x, but not due to page coloring.

> I don't know where we will wind up but the performance variation's
> caused by cache conflicts in today's applications are real, and easily
> measurable.  Giving the growing increase in performance difference
> between CPUs and memory Amdahl's Law shows this will only grow
> so I think this is worth looking at.

Absolutely wrong.

Why? Because the fact is, that as memory gets further and further away 
from CPU's, caches have gotten further and further away from being direct 
mapped. 

Cache coloring is already a very questionable win for four-way 
set-associative caches. I doubt you can even _see_ it for eight-way or 
higher associativity caches.

In other words: the pressures you mention clearly do exist, but they are 
all driving direct-mapped caches out of the market, and thus making page
coloring _less_ interesting rather than more.

		Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/