Re: [PATCH v3 0/7] Support L3 Smart Data Cache Injection Allocation Enforcement (SDCIAE)

[Reinette Chatre]

Hi Babu,

On 4/8/25 5:41 PM, Moger, Babu wrote:
> Hi Reinette,
> 
> On 4/8/2025 4:44 PM, Reinette Chatre wrote:
>> Hi Babu,
>>
>> On 4/7/25 1:12 PM, Moger, Babu wrote:
>>> On 3/21/25 17:50, Reinette Chatre wrote:
>>>> On 1/30/25 1:20 PM, Babu Moger wrote:
>>
>>
>>>>>
>>>>
>>>> AMD also supports what is exposed to user space as "shareable_bits". According
>>>> to APM:
>>>>     Depending on the implementation, some portions of the L3 Cache may be
>>>>     shared by other system functions or used for some other purpose not
>>>>     under the control of the PQOS feature set. The L3 Cache Allocation
>>>>     Sharing Mask returned by CPUID Fn0000_0010_EBX_x1[L3ShareAllocMask] is a
>>>>     bitmask that represents portions of the L3 that may be shared by those
>>>>     functions.
>>>
>>> Here is the complete text.
>>>
>>> The L3 Cache allocation sharing mask (L3ShareAllocMask) returned in EBX by
>>> CPUID Fn0000_0010 with ECX=1 is a bit vector which represents portions of
>>> the cache which may be shared with other system entities or used for some
>>> other purpose not under the control of the QOS feature set. When software
>>> sets a bit in one of the L3_MASK_n registers at the same bit positions a
>>> bit in the L3ShareAllocMask, processors executing with the corresponding
>>> COS will competitively share that portion of the cache with the other
>>> function. If this mask is all 0’s, then there is no other entity in the
>>> system competing with the processors for use of the L3 cache.
>>>
>>> The "L3ShareAllocMask" is always reported as 0 on AMD systems.
>>>
>>>> Could you please include what (if any) the relationship is between the CBM
>>>> discoverable via Fn0000_0010_EBX_x1[L3ShareAllocMask] and the CBM of
>>>> "highest-supported L3_MASK_n register" when SDCIAE is enabled?
>>>
>>> No. There is no relationship in here.
>>>
>>>>
>>>> On the resctrl interface side the documentation currently states:
>>>>
>>>>     "shareable_bits":
>>>>         Bitmask of shareable resource with other executing
>>>>         entities (e.g. I/O). User can use this when
>>>>         setting up exclusive cache partitions. Note that
>>>>         some platforms support devices that have their
>>>>         own settings for cache use which can over-ride
>>>>         these bits.
>>>>
>>>> Even though this was originally used to expose the content of
>>>> Fn0000_0010_EBX_x1[L3ShareAllocMask] the intent of the content does
>>>> seem to also apply to the "io_alloc" CBM also.
>>>
>>> It says "shared by other system functions or used for some other purpose
>>> not under the control of the PQOS feature set".
>>
>> This is a quote from the AMD spec, not the resctrl user interface documentation.
>>
>> Please consider this from resctrl user interface perspective.
>>
>>>
>>> "io_alloc" is PQOS feature set. I feel it should not affect "shareable_bits".
>>
>> When I read the resctrl user interface documentation for "shareable_bits" it
>> sounds relevant to io_alloc. The "shareable_bits" contains a bitmask "of
>> shareable resource with other executing entities (e.g. I/O)" ... is this
>> not exactly io_alloc?
> 
> I agree the text is pretty generic. Actually, the whole bit mask (0xfffF) is shareable with io_alloc.

I think the value of "shareable_bits" presented to user space could be the
actual io_alloc_cbm value. Thus, not the "possible IO bitmask" but the actual
value. This seems to be the best match of what "shareable_bits" represents, which
is the region currently used by IO devices. This partners well with the "bit_usage"
output, for example, "X" can be used to show which portions of cache are currently
used by both software (via schemata of resource groups) and hardware (via io_alloc_cbm).

> 
> The 'shareable_bits' is coming from CPUID Fn0000_0010_EBX_x1[L3ShareAllocMask] which is always 0 on AMD systems.
> It will be bit odd to manipulate these value. Not sure if we have to do it.

It is not clear to me what you mean with "manipulate". "shareable_bits" does currently
come from the existing register but AMD now provides more interfaces with which this data
can be obtained and it seems appropriate to use it.

Reinette