[regression] cpuset: offlined CPUs removed from affinity masks

[Mathieu Desnoyers]

Hi,

I noticed the following regression with CONFIG_CPUSET=y. Note that
I am not using cpusets at all (only using the root cpuset I'm given
at boot), it's just configured in. I am currently working on a 5.2.5
kernel. I am simply combining use of taskset(1) (setting the affinity
mask of a process) and cpu hotplug. The result is that with
CONFIG_CPUSET=y, setting the affinity mask including an offline CPU number
don't keep that CPU in the affinity mask, and it is never put back when the
CPU comes back online. CONFIG_CPUSET=n behaves as expected, and puts back
the CPU into the affinity mask reported to user-space when it comes back
online.


* With CONFIG_CPUSET=y (unexpected behavior):

# echo 0 > /sys/devices/system/cpu/cpu1/online

% taskset 0x7 ./loop &
% taskset -p $!
pid 1341's current affinity mask: 5

# echo 1 > /sys/devices/system/cpu/cpu1/online

taskset -p $!
pid 1341's current affinity mask: 5

kill $!


* With CONFIG_CPUSET=n (expected behavior):

(Offlining CPU, then start task)

# echo 0 > /sys/devices/system/cpu/cpu1/online

% taskset 0x7 ./loop &
% taskset -p $!
pid 1358's current affinity mask: 5

# echo 1 > /sys/devices/system/cpu/cpu1/online

taskset -p $!
pid 1358's current affinity mask: 7

kill $!


Test system lscpu output:

Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              32
On-line CPU(s) list: 0-31
Thread(s) per core:  2
Core(s) per socket:  8
Socket(s):           2
NUMA node(s):        2
Vendor ID:           GenuineIntel
CPU family:          6
Model:               60
Model name:          Intel Core Processor (Haswell, no TSX, IBRS)
Stepping:            1
CPU MHz:             2399.996
BogoMIPS:            4799.99
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           32K
L1i cache:           32K
L2 cache:            4096K
NUMA node0 CPU(s):   0-7,16-23
NUMA node1 CPU(s):   8-15,24-31
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm cpuid_fault invpcid_single pti ibrs ibpb fsgsbase bmi1 avx2 smep bmi2 erms invpcid xsaveopt



-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com