Re: [PATCH net-next v2 0/8] net/mlx5: HWS, Optimize matchers ICM usage

[Mark Bloch]

On 23/06/2025 1:39, Zhu Yanjun wrote:
> 在 2025/6/22 10:22, Mark Bloch 写道:
>> This series optimizes ICM usage for unidirectional rules and
>> empty matchers and with the last patch we make hardware steering
>> the default FDB steering provider for NICs that don't support software
>> steering.
> 
> In this patchset, ICM is not explained. I googled this ICM. And I got the following
> 
> "
> ICM stands for Internal Context Memory, a specialized memory region used by Mellanox/NVIDIA network devices (e.g., ConnectX series NICs) to store hardware context and rule tables for offloaded operations like flow steering, filtering, and traffic redirection.
> 
> ICM is crucial when using hardware steering (HWS), where the NIC itself performs packet matching and forwarding without involving the host CPU.
> "

Broadly speaking, yes. You can also check its consumption via devlink health reporter.
https://docs.kernel.org/networking/devlink/mlx5.html
check out icm_consumption on the above page.

Mark

> If I am missing something, please correct me.
> 
> Zhu Yanjun
> 
>>
>> Hardware steering (HWS) uses a type of rule table container (RTC) that
>> is unidirectional, so matchers consist of two RTCs to accommodate
>> bidirectional rules.
>>
>> This small series enables resizing the two RTCs independently by
>> tracking the number of rules separately. For extreme cases where all
>> rules are unidirectional, this results in saving close to half the
>> memory footprint.
>>
>> Results for inserting 1M unidirectional rules using a simple module:
>>
>>             Pages        Memory
>> Before this patch:    300k        1.5GiB
>> After this patch:    160k        900MiB
>>
>> The 'Pages' column measures the number of 4KiB pages the device requests
>> for itself (the ICM).
>>
>> The 'Memory' column is the difference between peak usage and baseline
>> usage (before starting the test) as reported by `free -h`.
>>
>> In addition, second to last patch of the series handles a case where all
>> the matcher's rules were deleted: the large RTCs of the matcher are no
>> longer required, and we can save some more ICM by shrinking the matcher
>> to its initial size.
>>
>> Finally the last patch makes hardware steering the default mode
>> when in swichdev for NICs that don't have software steering support.
>>
>> Changelog
>> =========
>> Changes from v1 [0]:
>> - Fixed author on patches 5 and 6.
>>
>> References
>> ==========
>> [0] v1: https://lore.kernel.org/all/20250619115522.68469-1-mbloch@xxxxxxxxxx/
>>
>> Moshe Shemesh (1):
>>    net/mlx5: Add HWS as secondary steering mode
>>
>> Vlad Dogaru (5):
>>    net/mlx5: HWS, remove unused create_dest_array parameter
>>    net/mlx5: HWS, Refactor and export rule skip logic
>>    net/mlx5: HWS, Create STEs directly from matcher
>>    net/mlx5: HWS, Decouple matcher RX and TX sizes
>>    net/mlx5: HWS, Track matcher sizes individually
>>
>> Yevgeny Kliteynik (2):
>>    net/mlx5: HWS, remove incorrect comment
>>    net/mlx5: HWS, Shrink empty matchers
>>
>>   .../net/ethernet/mellanox/mlx5/core/fs_core.c |   2 +
>>   .../mellanox/mlx5/core/steering/hws/action.c  |   7 +-
>>   .../mellanox/mlx5/core/steering/hws/bwc.c     | 284 ++++++++++++++----
>>   .../mellanox/mlx5/core/steering/hws/bwc.h     |  14 +-
>>   .../mellanox/mlx5/core/steering/hws/debug.c   |  20 +-
>>   .../mellanox/mlx5/core/steering/hws/fs_hws.c  |  15 +-
>>   .../mellanox/mlx5/core/steering/hws/matcher.c | 166 ++++++----
>>   .../mellanox/mlx5/core/steering/hws/matcher.h |   3 +-
>>   .../mellanox/mlx5/core/steering/hws/mlx5hws.h |  36 ++-
>>   .../mellanox/mlx5/core/steering/hws/rule.c    |  35 +--
>>   .../mellanox/mlx5/core/steering/hws/rule.h    |   3 +
>>   11 files changed, 403 insertions(+), 182 deletions(-)
>>
>>
>> base-commit: 091d019adce033118776ef93b50a268f715ae8f6
> 
>