Re: [PATCH] sched/fair: Make sched-idle cpu selection consistent throughout

[Mel Gorman]

On Thu, Oct 24, 2019 at 12:15:27PM +0530, Viresh Kumar wrote:
> There are instances where we keep searching for an idle CPU despite
> having a sched-idle cpu already (in find_idlest_group_cpu(),
> select_idle_smt() and select_idle_cpu() and then there are places where
> we don't necessarily do that and return a sched-idle cpu as soon as we
> find one (in select_idle_sibling()). This looks a bit inconsistent and
> it may be worth having the same policy everywhere.
> 

This needs supporting data. find_idlest_group_cpu is generally from
a fork() context where it's not particularly performance critical.
select_idle_sibling and the helpers it uses is wakeup context where is
is often much more critical to wake quickly than find the best CPU. The
biggest challenge of select_idle_sibling is making a "good enough decision"
quickly without disrupting cache but a fork-intensive workload making quick
decision can overload local domains requiring fixing by the load balancer.

> On the other hand, choosing a sched-idle cpu over a idle one shall be
> beneficial from performance point of view as well, as we don't need to
> get the cpu online from a deep idle state which is quite a time
> consuming process and delays the scheduling of the newly wakeup task.
> 
> This patch tries to simplify code around sched-idle cpu selection and
> make it consistent throughout.
> 
> FWIW, tests were done with the help of rt-app (8 SCHED_OTHER and 5
> SCHED_IDLE tasks, not bound to any cpu) on ARM platform (octa-core), and
> no significant difference in scheduling latency of SCHED_OTHER tasks was
> found.
> 

As the patch stands, I think a fork-intensive workload where each
process is doing small amounts of work will suffer from overloading
domains and have variable performance depending on how quickly the load
balancer reacts.

-- 
Mel Gorman
SUSE Labs