Re: [PATCH] SLUB use cmpxchg_local

[Mathieu Desnoyers]

* Christoph Lameter (clameter@xxxxxxx) wrote:
> On Mon, 27 Aug 2007, Mathieu Desnoyers wrote:
> 
> > * Christoph Lameter (clameter@xxxxxxx) wrote:
> > > On Mon, 27 Aug 2007, Peter Zijlstra wrote:
> > > 
> > > > So, if the fast path can be done with a preempt off, it might be doable
> > > > to suffer the slow path with a per cpu lock like that.
> > > 
> > > Sadly the cmpxchg_local requires local per cpu data access. Isnt there 
> > > some way to make this less expensive on RT? Acessing cpu local memory is 
> > > really good for performance on NUMA since the data is optimally placed and 
> > > one can avoid/reduce locking if the process stays tied to the processor.
> > > 
> > 
> > On the slow path, in slab_new, we already have to reenable interrupts
> > because we can sleep. If we make sure that whenever we return to an irq
> > disable code path we take the current per-cpu data structure again, can
> > we make the preempt-disable/irq-disabled code paths O(1) ?
> 
> Not sure exactly what you are getting at?
> This would mean running __alloc_pages tied to one processor even though 
> waiting is possible?
> 

Not exactly. What I propose is:

- Running slab_alloc and slab_free fast paths in preempt_disable
  context, using cmpxchg_local.
- Running slab_alloc and slab_free slow paths with irqs disabled.
- Running __alloc_pages in preemptible context, not tied to any CPU.

In this scheme, calling __alloc_pages from slab_alloc would reenable
interrupts and potentially migrate us to a different CPU. We would
therefore have to get once again our per-cpu data structure once we get
back into irq disabled code, because we may be running on a different
CPU. This is actually what the __slab_alloc slow path does:


new_slab:
        new = get_partial(s, gfpflags, node);
        if (new) {
                c->page = new;
                goto load_freelist;
        }

        new = new_slab(s, gfpflags, node);

          ----> within new_slab, we can reenable interrupts for the
                __slab_alloc call.

        if (new) {
                c = get_cpu_slab(s, smp_processor_id());
                if (c->page) {
                        /*
                         * Someone else populated the cpu_slab while we
                         * enabled interrupts, or we have gotten scheduled
                         * on another cpu. The page may not be on the
                         * requested node even if __GFP_THISNODE was
                         * specified. So we need to recheck.
                         */
                        if (node_match(c, node)) {
                                /*
                                 * Current cpuslab is acceptable and we
                                 * want the current one since its cache hot
                                 */
                                discard_slab(s, new);
                                slab_lock(c->page);
                                goto load_freelist;
                        }
                        /* New slab does not fit our expectations */
                        flush_slab(s, c);
                }
                slab_lock(new);
                SetSlabFrozen(new);
                c->page = new;
                goto load_freelist;

So the idea would be to split the code in O(1)
preempt_disable/irq_disable sections and to enable interrupt and check
for current per-cpu data structure when re-entering in irq disabled
code.

Mathieu

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/