Re: Performance dropout in recent 2.1.x kernels

Zlatko Calusic (Zlatko.Calusic@CARNet.hr)
28 Jan 1998 19:56:31 +0100


Mark Hemment <markhe@nextd.demon.co.uk> writes:

> This message is in MIME format. The first part should be readable text,
> while the remaining parts are likely unreadable without MIME-aware tools.
> Send mail to mime@docserver.cac.washington.edu for more info.
>
> ---1734428081-924026288-885980764=:91
> Content-Type: TEXT/PLAIN; charset=US-ASCII
>
> Hi,
>
> On 27 Jan 1998, Zlatko Calusic wrote:
> > -#define SLAB_MIN_OBJS_PER_SLAB 4
> > +#define SLAB_MIN_OBJS_PER_SLAB 1
> >
> > -#define SLAB_BREAK_GFP_ORDER 2
> > +#define SLAB_BREAK_GFP_ORDER 1
>
> Yes, being the author for the SLAB allocator, I've mentioned this
> work-around several times. The change to SLAB_BREAK_GFP_ORDER (the only
> one which really needs changing) was part of a patch I posted to linux-mm
> last week - which received zero feed-back. The change to SLAB_MIN_OBJS
> _PER_SLAB can hurt the slab's performance quite badly for some slab sizes,
> but it will help those who suffer from numerious networking allocation
> failures.

Yes, I agree.
With unchanged 2.1.x kernels, memory is being fragmented heavily with
ethernet network traffic. I saw it like this:

MTU = 1500
Max packet + sk_buff private data ~ 1700
First >= object size = 2048
GFP order for that object size =2 -> 16384 (4 pages!)

So effectively, to receive packets from ethernet Linux is allocating
and deallocating blocks of four pages!

Changing SLAB_BREAK_GFP_ORDER to 1, reduces this to 2 pages (if I
recall it correctly).

What I wanted is that network buffers fit in only one page.
1700 * 2 < 4096! Simple mathematics. :)

Now, I believe that this killed performance a bit, but really, I
didn't tested it, and I'm very satisfied with 3.8MB/s to disk transfer
rate. At least I was in 2.1.78. I'm still running 2.1.82. :)

Maybe allocating only one page at a time even speeds things a little?
Who knows. When current problems with networking gets solved I'll
benchmark it all and report results.

> (For those who missed this patch, I've attached a description of changes
> and placed the patch on my homepage; http://www.nextd.demon.co.uk/ It
> reduces 'reserved' memory, and reduces the need to re-check the page-cache
> after a possible blocking operation, along with a few other bits and
> pieces - no rocket science).
>

I tried it, but had some problems with startup. Machine did boot, but
had problems with arguments to startup scripts. Eventually I ended in
state something like singleuser, while I was really booting to
multiuser. Also I saw some messages I have never seen in my life,
coming from login process probably. :)

Nevertheless, you incorporated some great ideas in it.

Breaking files_struct is IMHO the greatest one. I assume you have read
my proposition I made month or two ago. Unfortunately I just couldn't
get enough time to work on that, so it's still on my TODO list. I've
been very busy lately.

In the end, one question for you Mark.
I've been thinking lately lot about one idea.

What do you think about changing slab allocator so it calls
get_free_pages directly if request is >= PAGE_SIZE? Second
modification would be to use simple page splitting for requests that
are bigger than PAGE_SIZE/8 and smaller than PAGE_SIZE (with off-slab
bufctls), and small requests (< PAGE_SIZE/8) would be served
unchanged.

It's been bothering me a lot recently, so I would like to hear your
opinion, as you're the Linux SLAB guy. :)

Keep up the good work!

-- 
Posted by Zlatko Calusic           E-mail: <Zlatko.Calusic@CARNet.hr>
---------------------------------------------------------------------
	"640K ought to be enough for anybody." Bill Gates '81