Re: [PATCH 1/3] arm64/numa: set numa_off to false when numa node is fake

[David Hildenbrand]

On 06.07.20 14:36, Justin He wrote:
> Hi David, thanks for the comments. See my answer please:
> 
>> -----Original Message-----
>> From: David Hildenbrand <david@xxxxxxxxxx>
>> Sent: Monday, July 6, 2020 4:03 PM
>> To: Justin He <Justin.He@xxxxxxx>; Catalin Marinas
>> <Catalin.Marinas@xxxxxxx>; Will Deacon <will@xxxxxxxxxx>
>> Cc: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>; Mike Rapoport
>> <rppt@xxxxxxxxxxxxx>; Baoquan He <bhe@xxxxxxxxxx>; Chuhong Yuan
>> <hslester96@xxxxxxxxx>; linux-arm-kernel@xxxxxxxxxxxxxxxxxxx; linux-
>> kernel@xxxxxxxxxxxxxxx; linux-mm@xxxxxxxxx; Kaly Xin <Kaly.Xin@xxxxxxx>
>> Subject: Re: [PATCH 1/3] arm64/numa: set numa_off to false when numa node
>> is fake
>>
>> On 06.07.20 03:19, Jia He wrote:
>>> Previously, numa_off is set to true unconditionally in dummy_numa_init(),
>>> even if there is a fake numa node.
>>>
>>> But acpi will translate node id to NUMA_NO_NODE(-1) in
>> acpi_map_pxm_to_node()
>>> because it regards numa_off as turning off the numa node.
>>>
>>> Without this patch, pmem can't be probed as a RAM device on arm64 if
>> SRAT table
>>> isn't present.
>>>
>>> $ndctl create-namespace -fe namespace0.0 --mode=devdax --map=dev -s 1g -
>> a 64K
>>> kmem dax0.0: rejecting DAX region [mem 0x240400000-0x2bfffffff] with
>> invalid node: -1
>>> kmem: probe of dax0.0 failed with error -22
>>>
>>> This fixes it by setting numa_off to false.
>>>
>>> Signed-off-by: Jia He <justin.he@xxxxxxx>
>>> ---
>>>  arch/arm64/mm/numa.c | 3 ++-
>>>  1 file changed, 2 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
>>> index aafcee3e3f7e..7689986020d9 100644
>>> --- a/arch/arm64/mm/numa.c
>>> +++ b/arch/arm64/mm/numa.c
>>> @@ -440,7 +440,8 @@ static int __init dummy_numa_init(void)
>>>  		return ret;
>>>  	}
>>>
>>> -	numa_off = true;
>>> +	/* force numa_off to be false since we have a fake numa node here
>> */
>>> +	numa_off = false;
>>>  	return 0;
>>>  }
>>>
>>>
>>
>> What would happen if we use something like this in drivers/dax/kmem.c
>> instead:
>>
>> numa_node = dev_dax->target_node;
>> if (numa_node == NUMA_NO_NODE)
>> 	numa_node = memory_add_physaddr_to_nid(kmem_start);
>>
>> and eventually dropping the pr_warn in
>> arm64/memory_add_physaddr_to_nid() ? Would that work?
> 
> Yes, it works. I sent a similar patch [1] before. But seems pmem
> maintainer didn't satisfy it. Do you think memory_add_physaddr_to_nid()
> is better than numa_mem_id()? 

Well, it's the somewhat-common way to get a NID for memory hotadd.

E.g.,
- drivers/acpi/acpi_memhotplug.c
- drivers/base/memory.c
- drivers/hv/hv_balloon.c
- drivers/virtio/virtio_mem.c
- drivers/xen/balloon.c

use it in combination with add_memory_*()

Especially, ACPI and virtio-mem use it in case NUMA_NO_NID is detected.

-- 
Thanks,

David / dhildenb