Re: [RFC][PATCH 00/18] Increase resolution of load weights

[Nikunj A. Dadhania]

On Wed, 20 Apr 2011 13:51:19 -0700, Nikhil Rao <ncrao@xxxxxxxxxx> wrote:
> Hi All,
> 
> I have attached an early version of a RFC patchset to increase resolution of
> sched entity load weights. This RFC introduces SCHED_LOAD_RESOLUTION which
> scales NICE_0_LOAD by a factor of 1024. The scaling is done internally and should
> be completely invisible to the user.
> 
> Why do we need this?
> This extra resolution allows us to scale on two dimensions - number of cpus and
> the depth of hierarchies. It also allows for proper load balancing of low weight
> task groups (for eg., nice+19 on autogroup).
> 
> One of the big roadblocks for increasing resolution is the use of unsigned long
> for load.weight, which on 32-bit architectures can overflow with ~48 max-weight
> sched entities. In this RFC we convert all uses of load.weight to u64. This is
> still a work-in-progress and I have listed some of the issues I am still
> investigating below.
> 
> I would like to get some feedback on the direction of this patchset. Please let
> me know if there are alternative ways of doing this, and I'll be happy to
> explore them as well.
> 
> The patchset applies cleanly to v2.6.39-rc4. It compiles for i386 and boots on
> x86_64. Beyond the basic checks, it has not been well tested yet.
> 
> Major TODOs:
> - Detect overflow in update shares calculations (time * load), and set load_avg
>   to maximum possible value (~0ULL).
> - tg->task_weight uses an atomic which needs to be updates to 64-bit on 32-bit
>   machines. Might need to add a lock to protect this instead of atomic ops.
> - Check wake-affine math and effective load calculations for overflows.
> - Needs more testing and need to ensure fairness/balancing is not broken.
> 
Hi Nikhil,

I did a quick test for creating 600 cpu hog tasks with and without this
patches on a 16cpu machine(x86_64) and I am seeing some mis-behaviour:

Base kernel - 2.6.39-rc4

[root@krm1 ~]# time -p ./test
real 43.54
user 0.12
sys 1.05
[root@krm1 ~]# 

Base + patches

[root@krm1 ~]# time -p ./test

Takes almost infinity, after 2 minutes I see only 16 tasks created
viewed from another ssh session to the machine:

[root@krm1 ~]# ps ax | grep dowhile$ | wc  -l
16
[root@krm1 ~]# 

I am attaching the script.

Regards,
Nikunj

Attachment: create_loop_tasks.sh
Description: Bourne shell script