Re: [PATCH v5 1/5] mm,memory_hotplug: Allocate memmap from the added memory range

[David Hildenbrand]

On 25.03.21 13:35, Michal Hocko wrote:
> On Thu 25-03-21 12:08:43, David Hildenbrand wrote:
> > On 25.03.21 11:55, Oscar Salvador wrote:
> > > On Thu, Mar 25, 2021 at 10:17:33AM +0100, Michal Hocko wrote:
> > > > Why do you think it is wrong to initialize/account pages when they are
> > > > used? Keep in mind that offline pages are not used until they are
> > > > onlined. But vmemmap pages are used since the vmemmap is established
> > > > which happens in the hotadd stage.
> > >
> > > Yes, that is true.
> > > vmemmap pages are used right when we populate the vmemmap space.
> >
> > Note: I once herd of a corner-case use case where people offline memory
> > blocks to then use the "free" memory via /dev/mem for other purposes ("large
> > physical memory"). Not that I encourage such use cases, but they would be
> > fundamentally broken if the vmemmap ends up on offline memory and is
> > supposed to keep its state ...
>
> I am not aware of such a use case, it surely sounds, ehm creative, but
> nothing really new. But such a usecase sounds quite incompatible with
> this feature whether we account vmemmap at hotadd or hotremove because
> they would need to understand that part of the memory they have hotadded
> is not useable.

I think they can use it just fine via /dev/mem, which explicitly avoids 
any kind of "struct page" references IIRC. They would be overwriting the 
vmemmap, but that part scan happily read/write until onlining, where the 
vmemmap would get reinitialized and set online - from which point on 
pfn_to_online_page() would succeed also on the vmemmap itself.

> [...]
> > > - When moving the initialization/accounting to hot-add/hot-remove,
> > >     the section containing the vmemmap pages will remain offline.
> > >     It might get onlined once the pages get online in online_pages(),
> > >     or not if vmemmap pages span a whole section.
> > >     I remember (but maybe David rmemeber better) that that was a problem
> > >     wrt. pfn_to_online_page() and hybernation/kdump.
> > >     So, if that is really a problem, we would have to care of ot setting
> > >     the section to the right state.
> >
> > Good memory. Indeed, hibernation/kdump won't save the state of the vmemmap,
> > because the memory is marked as offline and, thus, logically without any
> > valuable content.
>
> Could you point me to the respective hibernation code please? I always
> get lost in that area. Anyway, we do have the same problem even if the
> whole accounting is handled during {on,off}lining, no?

kernel/power/snapshot.c:saveable_page().

> I am not really worried about kdump though. As the memory is offline
> then losing itse vmemmap is a mere annoyance.

Yep, kdump was relevant in our previous discussions when we were talking 
about memory blocks having their altmap located on other memory blocks.

> > > - AFAICS, doing all the above brings us to former times were some
> > >     initialization/accounting was done in a previous stage, and I remember
> > >     it was pushed hard to move those in online/offline_pages().
> > >     Are we ok with that?
> > >     As I said, we might have to set the right zone in hot-add stage, as
> > >     otherwise problems might come up.
> > >     Being that case, would not that also be conflating different concepts
> > >     at a wrong phases?
> >
> > I expressed my opinion already, no need to repeat. Sub-section online maps
> > would make it cleaner, but I am still not convinced we want/need that.
>
> Nah, subsections are more tricky than necessary and if we can live
> without them and have it just as pmem weirdness is more than enough ;)

Yes, absolutely :)

--
Thanks,

David / dhildenb