Re: [PATCH 2/2] mm/damon/sysfs-schemes: add use_nodes_of_tier on sysfs-schemes

[Honggyu Kim]

Hi Simon and SeongJae,

Sorry for the late response.

On 5/28/2025 8:10 PM, wangchuanguo wrote:
> This patch adds use_nodes_of_tier under
>    /sys/kernel/mm/damon/admin/kdamonds/<N>/contexts/<N>/schemes/<N>/
>
> The 'use_nodes_of_tier' can be used to select nodes within the same memory
> tier of target_nid for DAMOS actions such as DAMOS_MIGRATE_{HOT,COLD}.
>
> Signed-off-by: wangchuanguo <wangchuanguo@xxxxxxxxxx>
> ---
>   include/linux/damon.h        |  9 ++++++++-
>   include/linux/memory-tiers.h |  5 +++++
>   mm/damon/core.c              |  6 ++++--
>   mm/damon/lru_sort.c          |  3 ++-
>   mm/damon/paddr.c             | 19 ++++++++++++-------
>   mm/damon/reclaim.c           |  3 ++-
>   mm/damon/sysfs-schemes.c     | 31 ++++++++++++++++++++++++++++++-
>   mm/memory-tiers.c            | 13 +++++++++++++
>   samples/damon/mtier.c        |  3 ++-
>   samples/damon/prcl.c         |  3 ++-
>   10 files changed, 80 insertions(+), 15 deletions(-)

[...snip...]

> diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c
> index e8464f7e0014..e13321cff38f 100644
> --- a/mm/damon/paddr.c
> +++ b/mm/damon/paddr.c
> @@ -383,7 +383,7 @@ static unsigned long damon_pa_deactivate_pages(struct damon_region *r,
>   
>   static unsigned int __damon_pa_migrate_folio_list(
>   		struct list_head *migrate_folios, struct pglist_data *pgdat,
> -		int target_nid)
> +		int target_nid, bool use_nodes_of_tier)
>   {
>   	unsigned int nr_succeeded = 0;
>   	nodemask_t allowed_mask = NODE_MASK_NONE;
> @@ -405,6 +405,9 @@ static unsigned int __damon_pa_migrate_folio_list(
>   	if (list_empty(migrate_folios))
>   		return 0;
>   
> +	if (use_nodes_of_tier)
> +		allowed_mask = get_tier_nodemask(target_nid);

I have a concern about this part.  This might work but, IMHO, the current memory
tier doesn't provide a concept of cross socket bridge, which is UPI in Intel.

For example, please see the following topology.

       node0           node1
     +-------+  UPI  +-------+
     | CPU 0 |-------| CPU 1 |
     +-------+       +-------+
     | DRAM0 |       | DRAM1 |  <-- memory tier 0
     +---+---+       +---+---+
         |               |
     +---+---+       +---+---+
     | CXL 0 |       | CXL 1 |  <-- memory tier 1
     +---+---+       +---+---+
       node2           node3

Even if some nodes are in the same memory tier, but if those are in the
different socket side, then the migratio makes the situation worse unexpectedly.

For example, if the page at node0 is tried to be demoted to node2, but if node2
is full then the current behavior is to cancel the demotion, but this change
makes it to be demoted to node3, which is on the other side of socket.

Since the cross socket access is a lot worse, I worry about this change.

Please let me know if you have a different thought.

Thanks,
Honggyu