Re: [ 004/120] slab: fix the DEADLOCK issue on l3 alien lock

[Steven Rostedt]

On Thu, Oct 11, 2012 at 09:59:16AM +0900, Greg Kroah-Hartman wrote:
> 3.4-stable review patch.  If anyone has any objections, please let me know.
> 
> ------------------
> 
> From: Michael Wang <wangyun@xxxxxxxxxxxxxxxxxx>
> 
> commit 947ca1856a7e60aa6d20536785e6a42dff25aa6e upstream.
> 
> DEADLOCK will be report while running a kernel with NUMA and LOCKDEP enabled,
> the process of this fake report is:
> 
> 	   kmem_cache_free()	//free obj in cachep
> 	-> cache_free_alien()	//acquire cachep's l3 alien lock
> 	-> __drain_alien_cache()
> 	-> free_block()
> 	-> slab_destroy()
> 	-> kmem_cache_free()	//free slab in cachep->slabp_cache
> 	-> cache_free_alien()	//acquire cachep->slabp_cache's l3 alien lock
> 
> Since the cachep and cachep->slabp_cache's l3 alien are in the same lock class,
> fake report generated.
> 
> This should not happen since we already have init_lock_keys() which will
> reassign the lock class for both l3 list and l3 alien.
> 
> However, init_lock_keys() was invoked at a wrong position which is before we
> invoke enable_cpucache() on each cache.
> 
> Since until set slab_state to be FULL, we won't invoke enable_cpucache()
> on caches to build their l3 alien while creating them, so although we invoked
> init_lock_keys(), the l3 alien lock class won't change since we don't have
> them until invoked enable_cpucache() later.
> 
> This patch will invoke init_lock_keys() after we done enable_cpucache()
> instead of before to avoid the fake DEADLOCK report.
> 
> Michael traced the problem back to a commit in release 3.0.0:

I don't see this fix in the last 3.0 stable release. Shouldn't it go
there too?

-- Steve

> 
> commit 30765b92ada267c5395fc788623cb15233276f5c
> Author: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
> Date:   Thu Jul 28 23:22:56 2011 +0200
> 
>     slab, lockdep: Annotate the locks before using them
> 
>     Fernando found we hit the regular OFF_SLAB 'recursion' before we
>     annotate the locks, cure this.
> 
>     The relevant portion of the stack-trace:
> 
>     > [    0.000000]  [<c085e24f>] rt_spin_lock+0x50/0x56
>     > [    0.000000]  [<c04fb406>] __cache_free+0x43/0xc3
>     > [    0.000000]  [<c04fb23f>] kmem_cache_free+0x6c/0xdc
>     > [    0.000000]  [<c04fb2fe>] slab_destroy+0x4f/0x53
>     > [    0.000000]  [<c04fb396>] free_block+0x94/0xc1
>     > [    0.000000]  [<c04fc551>] do_tune_cpucache+0x10b/0x2bb
>     > [    0.000000]  [<c04fc8dc>] enable_cpucache+0x7b/0xa7
>     > [    0.000000]  [<c0bd9d3c>] kmem_cache_init_late+0x1f/0x61
>     > [    0.000000]  [<c0bba687>] start_kernel+0x24c/0x363
>     > [    0.000000]  [<c0bba0ba>] i386_start_kernel+0xa9/0xaf
> 
>     Reported-by: Fernando Lopez-Lezcano <nando@xxxxxxxxxxxxxxxxxx>
>     Acked-by: Pekka Enberg <penberg@xxxxxxxxxx>
>     Signed-off-by: Peter Zijlstra <a.p.zijlstra@xxxxxxxxx>
>     Link: http://lkml.kernel.org/r/1311888176.2617.379.camel@laptop
>     Signed-off-by: Ingo Molnar <mingo@xxxxxxx>
> 
> The commit moved init_lock_keys() before we build up the alien, so we
> failed to reclass it.
> 
> Acked-by: Christoph Lameter <cl@xxxxxxxxx>
> Tested-by: Paul E. McKenney <paulmck@xxxxxxxxxxxxxxxxxx>
> Signed-off-by: Michael Wang <wangyun@xxxxxxxxxxxxxxxxxx>
> Signed-off-by: Pekka Enberg <penberg@xxxxxxxxxx>
> Signed-off-by: Greg Kroah-Hartman <gregkh@xxxxxxxxxxxxxxxxxxx>
> 
> ---
>  mm/slab.c |    6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> --- a/mm/slab.c
> +++ b/mm/slab.c
> @@ -1685,9 +1685,6 @@ void __init kmem_cache_init_late(void)
>  
>  	g_cpucache_up = LATE;
>  
> -	/* Annotate slab for lockdep -- annotate the malloc caches */
> -	init_lock_keys();
> -
>  	/* 6) resize the head arrays to their final sizes */
>  	mutex_lock(&cache_chain_mutex);
>  	list_for_each_entry(cachep, &cache_chain, next)
> @@ -1695,6 +1692,9 @@ void __init kmem_cache_init_late(void)
>  			BUG();
>  	mutex_unlock(&cache_chain_mutex);
>  
> +	/* Annotate slab for lockdep -- annotate the malloc caches */
> +	init_lock_keys();
> +
>  	/* Done! */
>  	g_cpucache_up = FULL;
>  
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/