Re: [RFC, PATCH] fs: push rcu_barrier() fromdeactivate_locked_super() to filesystems

[Kirill A. Shutemov]

On Fri, Jun 08, 2012 at 02:43:58PM -0700, Linus Torvalds wrote:
> On Fri, Jun 8, 2012 at 2:28 PM, Kirill A. Shutemov
> <kirill.shutemov@xxxxxxxxxxxxxxx> wrote:
> > From: "Kirill A. Shutemov" <kirill.shutemov@xxxxxxxxxxxxxxx>
> >
> > There's no reason to call rcu_barrier() on every deactivate_locked_super().
> > We only need to make sure that all delayed rcu free inodes are flushed
> > before we destroy related cache.
> >
> > Removing rcu_barrier() from deactivate_locked_super() affects some
> > fas paths. E.g. on my machine exit_group() of a last process in IPC
> > namespace takes 0.07538s. rcu_barrier() takes 0.05188s of that time.
> 
> I think we should just delete it.
> 
> kmem_cache_destroy() (at least for SLUB) already has:
> 
>                 if (s->flags & SLAB_DESTROY_BY_RCU)
>                         rcu_barrier();
> 
> in it. But I think it's too late - it gets called *after* we do
> kmem_cache_close(), and I get the feeling that we should do it before.
> 
> Shouldn't that be sufficient? And if other slab allocators don't have
> this, we should add it to them too.
> 
> Hmm?

When I tried SLAB_DESTROY_BY_RCU I've got problem:

[   36.687999] Pid: 3455, comm: rmmod Not tainted 3.5.0-rc1-00130-g48d212a-dirty #40
[   36.688001] Call Trace:
[   36.688012]  [<ffffffff8113367a>] slab_err+0xaa/0xd0
[   36.688020]  [<ffffffff8113515a>] ? __kmalloc+0x10a/0x110
[   36.688026]  [<ffffffff8113647d>] kmem_cache_destroy+0x1dd/0x420
[   36.688056]  [<ffffffffa00f0f25>] btrfs_destroy_cachep+0x15/0x60 [btrfs]
[   36.688076]  [<ffffffffa013cac3>] exit_btrfs_fs+0x9/0x3a [btrfs]
[   36.688083]  [<ffffffff810c324e>] sys_delete_module+0x16e/0x2f0
[   36.688090]  [<ffffffff8128cf29>] ? lockdep_sys_exit_thunk+0x35/0x67
[   36.688097]  [<ffffffff8161eba6>] system_call_fastpath+0x1a/0x1f
[   36.688111] Pid: 3455, comm: rmmod Not tainted 3.5.0-rc1-00130-g48d212a-dirty #40
[   36.688114] Call Trace:
[   36.688119]  [<ffffffff811365ee>] kmem_cache_destroy+0x34e/0x420
[   36.688143]  [<ffffffffa00f0f25>] btrfs_destroy_cachep+0x15/0x60 [btrfs]
[   36.688162]  [<ffffffffa013cac3>] exit_btrfs_fs+0x9/0x3a [btrfs]
[   36.688168]  [<ffffffff810c324e>] sys_delete_module+0x16e/0x2f0
[   36.688174]  [<ffffffff8128cf29>] ? lockdep_sys_exit_thunk+0x35/0x67
[   36.688179]  [<ffffffff8161eba6>] system_call_fastpath+0x1a/0x1f

IIUC, moving rcu_barrier() up should help, but I can't say that I fully
understand SLAB_DESTROY_BY_RCU semantics.

-- 
 Kirill A. Shutemov

Attachment: signature.asc
Description: Digital signature