Re: [RFC PATCH v3 0/7] device backed vmemmap crash dump support
From: Baoquan He
Date: Thu Mar 21 2024 - 02:17:25 EST
On 03/21/24 at 05:40am, Zhijian Li (Fujitsu) wrote:
> ping
>
>
> Any comment is welcome.
I will have a look at this from kdump side. How do you test your code?
By the way, there's issue reported by test robot.
Thanks
Baoquan
>
>
> On 06/03/2024 18:28, Li Zhijian wrote:
> > Hello folks,
> >
> > Compared with the V2[1] I posted a long time ago, this time it is a
> > completely new proposal design.
> >
> > ### Background and motivate overview ###
> > ---
> > Crash dump is an important feature for troubleshooting the kernel. It is the
> > final way to chase what happened at the kernel panic, slowdown, and so on. It
> > is one of the most important tools for customer support.
> >
> > Currently, there are 2 syscalls(kexec_file_load(2) and kexec_load(2)) to
> > configure the dumpable regions. Generally, (A)iomem resources registered with
> > flags (IORESOURCE_SYSTEM_RAM | IORESOUCE_BUSY) for kexec_file_load(2) or
> > (B)iomem resources registered with "System RAM" name prefix for kexec_load(2)
> > are dumpable.
> >
> > The pmem use cases including fsdax and devdax, could map their vmemmap to
> > their own devices. In this case, these part of vmemmap will not be dumped when
> > crash happened since these regions are satisfied with neither the above (A)
> > nor (B).
> >
> > In fsdax, the vmemmap(struct page array) becomes very important, it is one of
> > the key data to find status of reverse map. Lacking of the information may
> > cause difficulty to analyze trouble around pmem (especially Filesystem-DAX).
> > That means troubleshooters are unable to check more details about pmem from
> > the dumpfile.
> >
> > ### Proposal ###
> > ---
> > In this proposal, register the device backed vmemmap as a separate resource.
> > This resource has its own new flag and name, and then teaches kexec_file_load(2)
> > and kexec_load(2) to mark it as dumpable.
> >
> > Proposed flag: IORESOURCE_DEVICE_BACKED_VMEMMAP
> > Proposed name: "Device Backed Vmemmap"
> >
> > NOTE: crash-utils also needs to adapt to this new name for kexec_load()
> >
> > With current proposal, the /proc/iomem should show as following for device
> > backed vmemmap
> > # cat /proc/iomem
> > ...
> > fffc0000-ffffffff : Reserved
> > 100000000-13fffffff : Persistent Memory
> > 100000000-10fffffff : namespace0.0
> > 100000000-1005fffff : Device Backed Vmemmap # fsdax
> > a80000000-b7fffffff : CXL Window 0
> > a80000000-affffffff : Persistent Memory
> > a80000000-affffffff : region1
> > a80000000-a811fffff : namespace1.0
> > a80000000-a811fffff : Device Backed Vmemmap # devdax
> > a81200000-abfffffff : dax1.0
> > b80000000-c7fffffff : CXL Window 1
> > c80000000-147fffffff : PCI Bus 0000:00
> > c80000000-c801fffff : PCI Bus 0000:01
> > ...
> >
> > ### Kdump service reloading ###
> > ---
> > Once the kdump service is loaded, if changes to CPUs or memory occur,
> > either by hot un/plug or off/onlining, the crash elfcorehdr should also
> > be updated. There are 2 approaches to make the reloading more efficient.
> > 1) Use udev rules to watch CPU and memory events, then reload kdump
> > 2) Enable kernel crash hotplug to automatically reload elfcorehdr (>= 6.5)
> >
> > This reloading also needed when device backed vmemmap layouts change, Similar
> > to what 1) does now, one could add the following as the first lines to the
> > RHEL udev rule file /usr/lib/udev/rules.d/98-kexec.rules:
> >
> > # namespace updated: watch daxX.Y(devdax) and pfnX.Y(fsdax) of nd
> > SUBSYSTEM=="nd", KERNEL=="[dp][af][xn][0-9].*", ACTION=="bind", GOTO="kdump_reload"
> > SUBSYSTEM=="nd", KERNEL=="[dp][af][xn][0-9].*", ACTION=="unbind", GOTO="kdump_reload"
> > # devdax <-> system-ram updated: watch daxX.Y of dax
> > SUBSYSTEM=="dax", KERNEL=="dax[0-9].*", ACTION=="bind", GOTO="kdump_reload"
> > SUBSYSTEM=="dax", KERNEL=="dax[0-9].*", ACTION=="unbind", GOTO="kdump_reload"
> >
> > Regarding 2), my idea is that it would need to call the memory_notify() in
> > devm_memremap_pages_release() and devm_memremap_pages() to trigger the crash
> > hotplug. This part is not yet mature, but it does not affect the whole feature
> > because we can still use method 1) alternatively.
> >
> > [1] https://lore.kernel.org/lkml/02066f0f-dbc0-0388-4233-8e24b6f8435b@xxxxxxxxxxx/T/
> > --------------------------------------------
> > changes from V2[1]
> > - new proposal design
> >
> > CC: Alison Schofield <alison.schofield@xxxxxxxxx>
> > CC: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
> > CC: Baoquan He <bhe@xxxxxxxxxx>
> > CC: Borislav Petkov <bp@xxxxxxxxx>
> > CC: Dan Williams <dan.j.williams@xxxxxxxxx>
> > CC: Dave Hansen <dave.hansen@xxxxxxxxxxxxxxx>
> > CC: Dave Jiang <dave.jiang@xxxxxxxxx>
> > CC: Greg Kroah-Hartman <gregkh@xxxxxxxxxxxxxxxxxxx>
> > CC: "H. Peter Anvin" <hpa@xxxxxxxxx>
> > CC: Ingo Molnar <mingo@xxxxxxxxxx>
> > CC: Ira Weiny <ira.weiny@xxxxxxxxx>
> > CC: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
> > CC: Vishal Verma <vishal.l.verma@xxxxxxxxx>
> > CC: linux-cxl@xxxxxxxxxxxxxxx
> > CC: linux-mm@xxxxxxxxx
> > CC: nvdimm@xxxxxxxxxxxxxxx
> > CC: x86@xxxxxxxxxx
> >
> > Li Zhijian (7):
> > mm: memremap: register/unregister altmap region to a separate resource
> > mm: memremap: add pgmap_parent_resource() helper
> > nvdimm: pmem: assign a parent resource for vmemmap region for the
> > fsdax
> > dax: pmem: assign a parent resource for vmemmap region for the devdax
> > resource: Introduce walk device_backed_vmemmap res() helper
> > x86/crash: make device backed vmemmap dumpable for kexec_file_load
> > nvdimm: set force_raw=1 in kdump kernel
> >
> > arch/x86/kernel/crash.c | 5 +++++
> > drivers/dax/pmem.c | 8 ++++++--
> > drivers/nvdimm/namespace_devs.c | 3 +++
> > drivers/nvdimm/pmem.c | 9 ++++++---
> > include/linux/ioport.h | 4 ++++
> > include/linux/memremap.h | 4 ++++
> > kernel/resource.c | 13 +++++++++++++
> > mm/memremap.c | 30 +++++++++++++++++++++++++++++-
> > 8 files changed, 70 insertions(+), 6 deletions(-)
> >