Re: [PATCH 2/3] xvmalloc memory allocator

[Nitin Gupta]

Andrew Morton wrote:
> On Sat, 21 Mar 2009 17:42:52 +0530 Nitin Gupta <nitingupta910@xxxxxxxxx> wrote:
>
> > Andrew Morton wrote:
> > > On Sat, 21 Mar 2009 01:13:42 +0530 Nitin Gupta <ngupta@xxxxxxxxxx> wrote:
> > > But what is regrettable is that xvmalloc appears to be tied to
> > > compressed-swap in some manner.  Is it not possible to split these two
> > > initiatives apart so that neither is dependent upon the other?  Or is
> > > compressed-swap hopelessly crippled without xvmalloc?
> > xvmalloc itself is completely independent of compressed-swap. Infact, its
> > loaded as separate kernel module (xvmalloc.ko)
>
> That sounds good.
>
> > However, this compression project is almost useless without this specialized
> > allocator.
>
> Why?  Important information!!
>
> See, being told all this helps us understand why xvmalloc exists.  Plus
> once we have a good description of _why_ xvmalloc is needed, perhaps we can
> come up with alternatives which are more palatable than merging a whole new
> allocator.  Such as enhancing an existing one.

xvmalloc is needed by compressed swap since:
 - Its O(1)
 - It is very memory efficient
 - It can use "high memory" for allocation

* space efficiency:

 - comparison with SLUB:
http://code.google.com/p/compcache/wiki/AllocatorsComparison
shows that tlsf (allocator on which xvmalloc is based) uses ~40% less memory
than kmalloc() backed by SLUB. Christoph suggested creating multiple slabs of
different sizes for this test -- which will be a more fair comparison as kmalloc
just uses some predefined slabs. I hope to present this data soon.
Also, SLUB is limited to using "low memory" - this is blocker issue for compress
swap project (on 32-bit system with >1G RAM). xvmalloc can use high memory.

 - comparison with SLOB:
In some previous mail in this thread, I explained all the issues that exist
with SLOB that make it unacceptable for use in this project.


> > > (compcache is a terrible name, btw - it isn't a "compressed cache" at all!)
> > I have now heard this many times and my conscious is beginning to hurt now :)
> > I will change it to match name of its block device: ramzswap   sounds better?
>
> Is there anything swap-specific about it?  It's a block device, yes?  I
> should be able to run mkfs.ext2 on it and mount the thing?
No. It can handle page-aligned I/O only. Maybe its not too difficult to extend
it to handle arbitrary I/O. But as a swap device, handling just page-aligned
I/O is good enough.

> > > > Anyways, I will move it to drivers/block.
> > > This sounds like it might be a backward step.
> >
> > I'm bit confused here. Last thing I want to do is block mainline merge
> > because of such issues. Its real pain to maintain these things separately.
>
> This is why I tell myself to never use the word "it" in an email message.
>
> I assumed that you were referring to moving xvmalloc() down into
> drivers/block.  That would be bad, because then xvmalloc() will _never_ be
> usable by anything other than ramzblock <new name!>?

I was also referring to moving xvmalloc to drivers/block. I meant that for
now maybe move it to drivers/block it that can help speed up the merge.
Maybe later if someone else find it useful too then we can work to move
it back to the real place: mm/ :)

Thanks,
Nitin

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/