Re: sched: CPU #1's llc-sibling CPU #0 is not on the same node!

From: Yasuaki Ishimatsu
Date: Wed Feb 27 2013 - 00:51:02 EST


2013/02/27 14:11, Yinghai Lu wrote:
On Tue, Feb 26, 2013 at 8:43 PM, Yasuaki Ishimatsu
<isimatu.yasuaki@xxxxxxxxxxxxxx> wrote:
2013/02/27 13:04, Yinghai Lu wrote:

On Tue, Feb 26, 2013 at 7:38 PM, Yasuaki Ishimatsu
<isimatu.yasuaki@xxxxxxxxxxxxxx> wrote:

2013/02/27 11:30, Yinghai Lu wrote:

Do you mean you can not boot one socket system with 1G ram ?
Assume socket 0 does not support hotplug, other 31 sockets support hot
plug.

So we could boot system only with socket0, and later one by one hot
add other cpus.



In this case, system can boot. But other cpus with bunch of ram hot
plug may fails, since system does not have enough memory for cover
hot added memory. When hot adding memory device, kernel object for the
memory is allocated from 1G ram since hot added memory has not been
enabled.


yes, it may fail, if the one node memory need page table and vmemmap
is more than 1g ...



for hot add memory we need to
1. add another wrapper for init_memory_mapping, just like
init_mem_mapping() for booting path.
2. we need make memblock more generic, so we can use it with hot add
memory during runtime.
3. with that we can initialize page table for hot added node with ram.
a. initial page table for 2M near node top is from node0 ( that does
not support hot plug).
b. then will use 2M for memory below node top...
c. with that we will make sure page table stay on local node.
alloc_low_pages need to be updated to support that.
4. need to make sure vmemmap on local node too.


I think so too. By this, memory hot plug becomes more useful.

I agree with your idea. But I think above ideas is future work.
So at first we should use movable memory for memory hot plug.
After that, we will implement above ideas.



so hot-remove node will work too later.

In the long run, we should make booting path and hot adding more
similar and share at most code.
That will make code get more test coverage.

Tang, Yasuaki, Andrew,

Please check if you are ok with attached reverting patch.

We will fix this problem with no objection. So please wait a while.

And the problem occurs by "movablemem_map=srat" not "movablemem_map=nn[KMG]@ss[KMG]"
At least, if you want to revert it, you should revert only "movablemem_map=srat" part.

Thanks,
Yasuaki Ishimatsu


Tim, Don,
Can you try if attached reverting patch fix all the problems for you ?

Thanks

Yinghai



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/