Re: [PATCH 0/2] intel_pstate: fix turbo not being used after a processor is rebooted

[Pratyush Yadav]

Hi Srinivas,

On Wed, Dec 21 2022, srinivas pandruvada wrote:
> On Wed, 2022-12-21 at 16:52 +0100, Pratyush Yadav wrote:
>> When a processor is brought offline and online again, it is unable to
>> use Turbo mode because the _PSS table does not contain the whole
>> turbo
>> frequency range, but only +1 MHz above the max non-turbo frequency.
>> This
>> causes problems when ACPI processor driver tries to set frequency
>> constraints. See patch 2 for more details.
>>
> Are you using some _PPC constraint to force to limit frequency?
> I did a offline/online with PPC=0 with no HWP, I can get to full turbo
> range.
>
> [  121.237752] smpboot: CPU 1 is now offline
> [  125.734886] x86: Booting SMP configuration:
> [  125.734892] smpboot: Booting Node 0 Processor 1 APIC 0x2
> [  125.741007] intel_pstate: CPU 1 going online
> [  125.741692] intel_pstate: CPU1 - ACPI _PSS perf data
> [  125.741698] intel_pstate:      *P0: 2301 MHz, 28000 mW, 0x2a00
> [  125.741703] intel_pstate:       P1: 2300 MHz, 28000 mW, 0x1700
> [  125.741705] intel_pstate:       P2: 2200 MHz, 26297 mW, 0x1600
> [  125.741707] intel_pstate:       P3: 2000 MHz, 23263 mW, 0x1400
> [  125.741710] intel_pstate:       P4: 1900 MHz, 21924 mW, 0x1300
> [  125.741712] intel_pstate:       P5: 1800 MHz, 20612 mW, 0x1200
> [  125.741714] intel_pstate:       P6: 1600 MHz, 17812 mW, 0x1000
> [  125.741716] intel_pstate:       P7: 1500 MHz, 16581 mW, 0xf00
> [  125.741718] intel_pstate:       P8: 1300 MHz, 13946 mW, 0xd00
> [  125.741720] intel_pstate:       P9: 1200 MHz, 12796 mW, 0xc00
> [  125.741722] intel_pstate:       P10: 1100 MHz, 11426 mW, 0xb00
> [  125.741724] intel_pstate:       P11: 900 MHz, 9250 mW, 0x900
> [  125.741726] intel_pstate:       P12: 800 MHz, 7965 mW, 0x800
> [  125.741729] intel_pstate:       P13: 700 MHz, 6940 mW, 0x700
> [  125.741731] intel_pstate:       P14: 500 MHz, 4738 mW, 0x500
> [  125.741733] intel_pstate:       P15: 400 MHz, 3787 mW, 0x400
> [  125.741735] intel_pstate: _PPC limits will be enforced
> [  125.741740] intel_pstate: policy->max > max non turbo frequency
> [  125.741742] intel_pstate: cpu:1 min_policy_perf:4 max_policy_perf:42
> [  125.741745] intel_pstate: cpu:1 global_min:4 global_max:42
> [  125.741747] intel_pstate: cpu:1 max_perf_ratio:42 min_perf_ratio:4
> [  125.742243] intel_pstate: policy->max > max non turbo frequency
> [  125.742247] intel_pstate: cpu:1 min_policy_perf:4 max_policy_perf:42
> [  125.742251] intel_pstate: cpu:1 global_min:4 global_max:42
> [  125.742255] intel_pstate: cpu:1 max_perf_ratio:42 min_perf_ratio:4
>
>
> It is not clear how to get to this non turbo situation.

Look at the scaling_max_freq before and after rebooting the CPU. Before
you do it, it should be the max turbo frequency (say 2500 MHz). After
rebooting the CPU, it should now be 2301 MHz. So the kernel will now not
ask for anything above 2301 MHz, so you will never get to 2500 MHz.

Another interesting thing I observed is that if I reboot only 1 CPU, its
scaling_max_freq goes down to 2301, but it still keeps working at 2500
MHz. This might be something to do with how turbo works, I don't
understand that very well. But if you reboot say 20 CPUs, then you see
the frequency drop.

I use the below steps to reproduce this bug on my system, which has 40
CPUs with a base frequency of 2500 MHz and turbo frequency of 3300 MHz:

$ grep 'cpu MHz' /proc/cpuinfo
cpu MHz         : 3300.000
cpu MHz         : 1199.652
cpu MHz         : 3300.000
cpu MHz         : 3300.000
cpu MHz         : 3300.000
cpu MHz         : 3300.000
cpu MHz         : 3300.000
cpu MHz         : 3300.000
cpu MHz         : 3300.000
cpu MHz         : 3300.000
[ repeat 30 times ]

$ cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_max_freq | sort -n | uniq -c
     40 3300000
$ for i in `seq 1 20`; do echo 0 | sudo tee /sys/devices/system/cpu/cpu$i/online; done
[...]
$ for i in `seq 1 20`; do echo 1 | sudo tee /sys/devices/system/cpu/cpu$i/online; done
[...]
$ grep 'cpu MHz' /proc/cpuinfo
cpu MHz         : 3300.000
cpu MHz         : 2500.000
cpu MHz         : 2500.000
cpu MHz         : 2500.000
cpu MHz         : 2500.000
cpu MHz         : 2500.000
[ repeat 15 times ]
cpu MHz         : 3300.000
cpu MHz         : 3300.000
[ repeat 17 times ]
$ cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_max_freq | sort -n | uniq -c
     20 2501000
     20 3300000

>
> Thanks,
> Srinivas
>
>> Pratyush Yadav (2):
>>   acpi: processor: allow fixing up the frequency for a performance
>> state
>>   cpufreq: intel_pstate: use acpi perflib to update turbo frequency
>>
>>  drivers/acpi/processor_perflib.c | 40
>> ++++++++++++++++++++++++++++++++
>>  drivers/cpufreq/intel_pstate.c   |  5 ++--
>>  include/acpi/processor.h         |  2 ++
>>  3 files changed, 45 insertions(+), 2 deletions(-)
>>
>> --
>> 2.38.1
>>
>

-- 
Regards,
Pratyush Yadav



Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879