Re: tip: demise of tsk_cpus_allowed() and tsk_nr_cpus_allowed()

From: Ingo Molnar
Date: Thu Feb 09 2017 - 02:10:00 EST



* Peter Zijlstra <peterz@xxxxxxxxxxxxx> wrote:

> On Wed, Feb 08, 2017 at 11:20:19AM +0100, Thomas Gleixner wrote:
> > On Mon, 6 Feb 2017, Ingo Molnar wrote:
> > > * Peter Zijlstra <peterz@xxxxxxxxxxxxx> wrote:
> > > >
> > > > cpumasks are a pain, the above avoids allocating more of them.
> >
> > Indeed.
> >
> > > Yeah, so this could then be done by pointerifying ->cpus_allowed - more robust
> > > than the wrappery,
> >
> > You mean:
> >
> > struct task_struct {
> > cpumask_t cpus_allowed;
> > cpumask_t *effective_cpus_allowed;
> > };

Yeah. I'd name it a bit differently and constify the pointer to get type
safety and to make sure the mask is never modified through the pointer:

struct task_struct {
const cpumask_t *cpus_ptr;
cpumask_t cpus_mask;
};

( I'd drop the 'allowed' part, it's obvious enough what task->cpus_mask does,
right? )

and upstream would essentially just do:

t->cpus_allowed_ptr = &t->cpus_allowed;

And -rt, when it wants to pin a task, would do:

t->cpus_allowed_ptr = &cpumask_of(task_cpu(p));

The rules are:

- Code that 'uses' ->cpus_allowed would use the pointer.

- Code that 'modifies' ->cpus_allowed would use the direct mask.

The upstream advantages are:

- The type separation of modifications from usage.

- Removal of wrappery.

- Maybe sometime in the future upstream would want to disable migration too ...

In fact -rt gains something too:

- With this scheme we would AFAICS get slightly more optimal code on -rt.
(Because there's no __migration_disabled() branching anymore.)

- Plus all new code is automatically -rt ready - no need to maintain the wrappery
space. Much less code path forking.

So as I see it it's win-win for both upstream and for -rt!

> > and make the scheduler use effective_cpus_allowed instead of cpus_allowed? Or
> > what do you have in mind?
>
> That scheme is weird for nr_cpus_allowed. Not to mention that the
> pointer to the integer is larger than the integer itself.

So in the new scheme I don't think nr_cpus_allowed would have to be wrapped
at all: whenever the pointer (or mask) is changed in set_cpus_allowed_common()
nr_cpus_allowed is recalculated as well - like today.

It should be self-maintaining. Am I missing something?

> I really prefer the current wrappers, they're trivial and consistent
> with one another.

I think it's ugly wrappery and we can do better! ;-)

But of course if I cannot suggest a better alternative then it stands.

Thanks,

Ingo