Forget fragmentation (was Re: Linux hostile to poverty)

Linus Torvalds (torvalds@transmeta.com)
18 Jul 1998 19:55:47 GMT


In article <Pine.LNX.4.00.9807181848320.11579-100000@vortex.ll.sci.fi>,
Samuli Kaski <samkaski@cs.helsinki.fi> wrote:
>On Sat, 18 Jul 1998, Benjamin Redelings I wrote:
>
>> And I just heard that this whole fragmentation problem is the result of
>> having a dcache, which is why 2.1 NEEDS to worry about fragmentation but
>> 2.0 didn't. But I don't really know what I'm talking about, just overhear
>
>Uhumm and I thought the exact opposite was true. At least I originally
>moved away from 2.0.32 because the box became absolutely unusable after
>5 or so days. I have been running 2.1.[96-105] for upto 30 days and I
>don't see any problems. (sound works, X doesn't get jerky and so on)
>
>As I see it the 2.1 fragmentation issues have seen the daylight in the
>past 2 months or so, can't say exactly when.

Fragmentation isn't the issue. As of 2.1.109 you shouldn't have to
worry, and even if you did it wouldn't show up as bad performance but as
fork() failures (and if you get a fork() failure while rebuilding the
kernel you'd get a failed kernel build, not a slow one).

The issue is that
- Linux uses more memory for caching. Dentries use up memory, and if
you used to be low on memory before, 2.1.x is going to be even lower
on memory. And once you're paging, you aren't going to have good
performance.

In particular, a kernel compile probably first into 8MB on 2.0.x (not
comfortable, but not excatly uncomfortable either). But in 2.1.x the
working set is suddenly larger - and you end up paging.

- The page aging is wrong, at least for low-memory circumstances.
Stephen Tweedie posted an interesting result where disabling page
aging completely made the thing run a lot faster.

Essentially, we use more memory than we used to, and to make matters
worse we don't throw out the right thing when we notice that we're low
on memory.

What 2.1.109 does is:
- fragmentation isn't really an issue any more (except for NFS, where
it has always been an issue, including in 2.0.x). HOWEVER, the amount
of free memory still is an issue - see above.
- much nicer swap-out, and better "no-memory" behaviour ("no-memory
behaviour" is the point where it decides that it's really run out of
swap and needs to start refusing allocations)

So what 2.1.109 does is to make the machine usable under low memory
conditions, but 2.1.109 does _not_ fix the page aging issue, not does it
change the fact that we use more memory for dentries.

(Note that using more memory for dentries is not something that is ever
going to be fixed - it's an integral part of 2.1.x, but as with the page
aging we probably use the _wrong_ memory, so the aging of memory is
again a big issue).

Some of the behaviour you can tune at run-time, and that may actually be
a big thing already. In particular, check out the /proc/sys/vm/freepages
file: it contains three numbers that essentially specify how much memory
the machine tries to keep free.. You can tweak the numbers a bit to see
how that changes the behaviour of your system..

Linus

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.altern.org/andrebalsa/doc/lkml-faq.html