Re: [PATCH] kernel/rcutree.c: deem to be lazy if there are nocallbacks.

[Paul E. McKenney]

On Tue, Sep 03, 2013 at 01:41:03PM +0800, Chen Gang wrote:
> Hello Maintainers:
> 
> Is this issue finished ?
> 
> If need additional help from me (e.g. some test things, or others, if
> you have no time, can let me try), please let me know, I should try.

Ah, sorry, here is the patch.

							Thanx, Paul

------------------------------------------------------------------------

    rcu: Micro-optimize rcu_cpu_has_callbacks()
    
    The for_each_rcu_flavor() loop unconditionally scans all flavors, even
    when the first flavor might have some non-lazy callbacks.  Once the
    loop has seen a non-lazy callback, further passes through the loop
    cannot change the state.  This is not a huge problem, given that there
    can be at most three RCU flavors (RCU-bh, RCU-preempt, and RCU-sched),
    but this code is on the path to idle, so speeding it up even a small
    amount would have some benefit.
    
    This commit therefore does two things:
    
    1.	Rearranges the order of the list of RCU flavors in order to
    	place the most active flavor first in the list.  The most active
    	RCU flavor is RCU-preempt, or, if there is no RCU-preempt,
    	RCU-sched.
    
    2.	Reworks the for_each_rcu_flavor() to exit early when the first
    	non-lazy callback is seen, or, in the case where the caller
    	does not care about non-lazy callbacks (RCU_FAST_NO_HZ=n),
    	when the first callback is seen.
    
    Reported-by: Chen Gang <gang.chen@xxxxxxxxxxx>
    Signed-off-by: Paul E. McKenney <paulmck@xxxxxxxxxxxxxxxxxx>

diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index b1b959d..38596be 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -2756,10 +2756,13 @@ static int rcu_cpu_has_callbacks(int cpu, bool *all_lazy)
 
 	for_each_rcu_flavor(rsp) {
 		rdp = per_cpu_ptr(rsp->rda, cpu);
-		if (rdp->qlen != rdp->qlen_lazy)
+		if (!rdp->nxtlist)
+			continue;
+		hc = true;
+		if (rdp->qlen != rdp->qlen_lazy || !all_lazy) {
 			al = false;
-		if (rdp->nxtlist)
-			hc = true;
+			break;
+		}
 	}
 	if (all_lazy)
 		*all_lazy = al;
@@ -3326,8 +3329,8 @@ void __init rcu_init(void)
 
 	rcu_bootup_announce();
 	rcu_init_geometry();
-	rcu_init_one(&rcu_sched_state, &rcu_sched_data);
 	rcu_init_one(&rcu_bh_state, &rcu_bh_data);
+	rcu_init_one(&rcu_sched_state, &rcu_sched_data);
 	__rcu_init_preempt();
 	open_softirq(RCU_SOFTIRQ, rcu_process_callbacks);
 

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/