Re: [PATCH 0/7] RISC-V: Sparsmem, Memory Hotplug and pte_devmap for P2P

From: Palmer Dabbelt
Date: Wed Apr 24 2019 - 19:23:07 EST


On Wed, 27 Mar 2019 14:36:36 PDT (-0700), logang@xxxxxxxxxxxx wrote:
Hi,

This patchset enables P2P on the RISC-V architecture. To do this on the
current kernel, we only need to be able to back IO memory with struct
pages using devm_memremap_pages(). This requires ARCH_HAS_ZONE_DEVICE,
ARCH_ENABLE_MEMORY_HOTPLUG, and ARCH_ENABLE_MEMORY_HOTREMOVE; which in
turn requires ARCH_SPARSEMEM_ENABLE. We also need to ensure that the
IO memory regions in hardware can be covered by the linear region
so that there is a linear relation ship between the virtual address and
the struct page address in the vmemmap region.

While our reason to do this work is for P2P, these features are all
useful, more generally, and also enable other kernel features.

The first patch in the series implements sparse mem. It was already
submitted and reviewed last cycle, only forgotten. It has been rebased
onto v5.1-rc2.

Patches 2 through 4 rework the architecture's virtual address space
mapping trying to get as much of the IO regions covered by the linear
mapping. With Sv39, we do not have enough address space to cover all the
typical hardware regions but we can get the majority of it.

Patch 5 and 6 implement memory hotplug and remove. These are relatively
straight forward additions similar to other arches.

Patch 7 implements pte_devmap which allows us to set
ARCH_HAS_ZONE_DEVICE.

The patchset was tested in QEMU and on a HiFive Unleashed board.
However, we were unable to actually test P2P transactions with this
exact set because we have been unable to get PCI working with v5.1-rc2.
We were able to get it running on a 4.19 era kernel (with a bunch of
out-of-tree patches for PCI on a Microsemi PolarFire board).

This series is based on v5.1-rc2 and a git tree is available here:

https://github.com/sbates130272/linux-p2pmem riscv-p2p-v1

Looks like these don't build on rv32 when applied on top of 5.1-rc6. We now
have rv32_defconfig, which should make it easier to tests these sorts of
things.