[patch v3.18+ regression fix] sched: Further improve spurious CPU_IDLE active migrations

[Mike Galbraith]

43f4d666 partially cured spurious migrations, but when there are
completely idle groups on a lightly loaded processor, and there is
a buddy pair occupying the busiest group, we will not attempt to
migrate due to select_idle_sibling() buddy placement, leaving the
busiest queue with one task.  We skip balancing, but increment
nr_balance_failed until we kick active balancing, and bounce a
buddy pair endlessly, demolishing throughput.

Regression detected on X5472 box, which has 4 MC groups of 2 cores.

netperf -l 60 -H 127.0.0.1 -t UDP_STREAM -i5,1 -I 95,5
pre:
!!! WARNING
!!! Desired confidence was not achieved within the specified iterations.
!!! This implies that there was variability in the test environment that
!!! must be investigated before going further.
!!! Confidence intervals: Throughput      : 66.421%
!!!                       Local CPU util  : 0.000%
!!!                       Remote CPU util : 0.000%

Socket  Message  Elapsed      Messages                
Size    Size     Time         Okay Errors   Throughput
bytes   bytes    secs            #      #   10^6bits/sec

212992   65507   60.00     1779143      0    15539.49
212992           60.00     1773551           15490.65

post:
Socket  Message  Elapsed      Messages                
Size    Size     Time         Okay Errors   Throughput
bytes   bytes    secs            #      #   10^6bits/sec

212992   65507   60.00     3719377      0    32486.01
212992           60.00     3717492           32469.54

Signed-off-by: Mike Galbraith <mgalbraith@xxxxxxx>
Fixes: caeb178c sched/fair: Make update_sd_pick_busiest() return 'true' on a busier sd
Cc: <stable@xxxxxxxxxxxxxxx> # v3.18+
---
 kernel/sched/fair.c |    7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -7249,11 +7249,12 @@ static struct sched_group *find_busiest_
 		 * This cpu is idle. If the busiest group is not overloaded
 		 * and there is no imbalance between this and busiest group
 		 * wrt idle cpus, it is balanced. The imbalance becomes
-		 * significant if the diff is greater than 1 otherwise we
-		 * might end up to just move the imbalance on another group
+		 * significant if the diff is greater than 2 otherwise we
+		 * may end up merely moving the imbalance to another group,
+		 * or bouncing a buddy pair needlessly.
 		 */
 		if ((busiest->group_type != group_overloaded) &&
-				(local->idle_cpus <= (busiest->idle_cpus + 1)))
+				(local->idle_cpus <= (busiest->idle_cpus + 2)))
 			goto out_balanced;
 	} else {
 		/*