Re: [PATCH v5 00/21] x86/resctrl: Make resctrl_arch_rmid_read() return values in bytes

[Xin Hao]

Hi  james,

I have a review all of the patches, it looks goot to me, but i also test 
them once again, i have a little confusion with my test.

# mkdir p1

# echo "L3:0=001;1=001" > p1/schemata

# [root@iZbp1bu26qv0j3ddyusot3Z p1]# cat schemata
    MB:0=100;1=100
    L3:0=001;1=001

# memhog -r1000000 1000m > /mnt/log &

[1] 53023
[root@iZbp1bu26qv0j3ddyusot3Z p1]# echo 53023 > tasks
[
[root@iZbp1bu26qv0j3ddyusot3Z p1]# cat mon_data/mon_L3_00/llc_occupancy
3833856
[root@iZbp1bu26qv0j3ddyusot3Z p1]# cat mon_data/mon_L3_00/llc_occupancy
3620864
[root@iZbp1bu26qv0j3ddyusot3Z p1]# cat mon_data/mon_L3_00/llc_occupancy
3727360
[root@iZbp1bu26qv0j3ddyusot3Z p1]# cat size
    MB:0=100;1=100
    L3:0=3407872;1=3407872

Obviously, the value has been overflowed,  Can you explain why？

My machine environment is:

Intel(R) Xeon(R) Platinum 8269CY CPU @ 2.50GHz

numactl -H
available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 
23 24 25 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 
73 74 75 76 77
node 0 size: 191813 MB
node 0 free: 189340 MB
node 1 cpus: 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 
46 47 48 49 50 51 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 
96 97 98 99 100 101 102 103
node 1 size: 193522 MB
node 1 free: 192332 MB
node distances:
node   0   1
  0:  10  21
  1:  21  10

On 6/23/22 12:46 AM, James Morse wrote:
> Changes in this version?
>   * Use supports_mba_mbps() in resctrl_{on,off}line_domain()
>   * Remove some error handling for errors that can't happen.
>   * Restored mbps_val[] reset code to set_mba_sc().
>   * Moved mbps_val[] reset in rdtgroup_init_mba() to reduce noise.
>   * Added resctrl_arch_round_mon_val() to fix the user provided value.
>
> ---
> The aim of this series is to insert a split between the parts of the monitor
> code that the architecture must implement, and those that are part of the
> resctrl filesystem. The eventual aim is to move all filesystem parts out
> to live in /fs/resctrl, so that resctrl can be wired up for MPAM.
>
> What's MPAM? See the cover letter of a previous series. [1]
>
> The series adds domain online/offline callbacks to allow the filesystem to
> manage some of its structures itself, then moves all the 'mba_sc' behaviour
> to be part of the filesystem.
> This means another architecture doesn't need to provide an mbps_val array.
> As its all software, the resctrl filesystem should be able to do this without
> any help from the architecture code.
>
> Finally __rmid_read() is refactored to be the API call that the architecture
> provides to read a counter value. All the hardware specific overflow detection,
> scaling and value correction should occur behind this helper.
>
>
> This series is based on v5.19-rc1, and can be retrieved from:
> git://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/resctrl_monitors_in_bytes/v5
>
> [0] git://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/resctrl_merge_cdp/v7
> [1] https://lore.kernel.org/lkml/20210728170637.25610-1-james.morse@xxxxxxx/
>
> [v1] https://lore.kernel.org/lkml/20210729223610.29373-1-james.morse@xxxxxxx/
> [v2] https://lore.kernel.org/lkml/20211001160302.31189-1-james.morse@xxxxxxx/
> [v3] https://lore.kernel.org/lkml/20220217182110.7176-1-james.morse@xxxxxxx/
> [v4] https://lore.kernel.org/lkml/20220412124419.30689-1-james.morse@xxxxxxx/
>
> James Morse (21):
>    x86/resctrl: Kill off alloc_enabled
>    x86/resctrl: Merge mon_capable and mon_enabled
>    x86/resctrl: Add domain online callback for resctrl work
>    x86/resctrl: Group struct rdt_hw_domain cleanup
>    x86/resctrl: Add domain offline callback for resctrl work
>    x86/resctrl: Remove set_mba_sc()s control array re-initialisation
>    x86/resctrl: Abstract and use supports_mba_mbps()
>    x86/resctrl: Create mba_sc configuration in the rdt_domain
>    x86/resctrl: Switch over to the resctrl mbps_val list
>    x86/resctrl: Remove architecture copy of mbps_val
>    x86/resctrl: Allow update_mba_bw() to update controls directly
>    x86/resctrl: Calculate bandwidth from the previous __mon_event_count()
>      chunks
>    x86/resctrl: Add per-rmid arch private storage for overflow and chunks
>    x86/resctrl: Allow per-rmid arch private storage to be reset
>    x86/resctrl: Abstract __rmid_read()
>    x86/resctrl: Pass the required parameters into
>      resctrl_arch_rmid_read()
>    x86/resctrl: Move mbm_overflow_count() into resctrl_arch_rmid_read()
>    x86/resctrl: Move get_corrected_mbm_count() into
>      resctrl_arch_rmid_read()
>    x86/resctrl: Rename and change the units of resctrl_cqm_threshold
>    x86/resctrl: Add resctrl_rmid_realloc_limit to abstract x86's
>      boot_cpu_data
>    x86/resctrl: Make resctrl_arch_rmid_read() return values in bytes
>
>   arch/x86/include/asm/resctrl.h            |   9 +
>   arch/x86/kernel/cpu/resctrl/core.c        | 117 ++++-------
>   arch/x86/kernel/cpu/resctrl/ctrlmondata.c |  75 ++++---
>   arch/x86/kernel/cpu/resctrl/internal.h    |  61 +++---
>   arch/x86/kernel/cpu/resctrl/monitor.c     | 232 ++++++++++++++--------
>   arch/x86/kernel/cpu/resctrl/pseudo_lock.c |   2 +-
>   arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 216 ++++++++++++++++----
>   include/linux/resctrl.h                   |  64 +++++-
>   8 files changed, 514 insertions(+), 262 deletions(-)
--
Best Regards!
Xin Hao