RE: [PATCH v6 31/31] selftests/vfio: Add CXL Type-2 passthrough tests

From: Manish Honap

Date: Wed Nov 18 2026 - 14:38:51 EST




> -----Original Message-----
> From: David Matlack <dmatlack@xxxxxxxxxx>
> Sent: Monday, November 16, 2026 11:23 PM
> To: Manish Honap <mhonap@xxxxxxxxxx>; alex@xxxxxxxxxxx; jgg@xxxxxxxx;
> Ankit Agrawal <ankita@xxxxxxxxxx>; jic23@xxxxxxxxxx; dave.jiang@xxxxxxxxx;
> alejandro.lucero-palau@xxxxxxx; Srirangan Madhavan
> <smadhavan@xxxxxxxxxx>; corbet@xxxxxxx; skhan@xxxxxxxxxxxxxxxxxxx;
> dave@xxxxxxxxxxxx; alison.schofield@xxxxxxxxx; vishal.l.verma@xxxxxxxxx;
> iweiny@xxxxxxxxxx; ming.li@xxxxxxxxxxxx; Yishai Hadas
> <yishaih@xxxxxxxxxx>; Shameer Kolothum Thodi
> <skolothumtho@xxxxxxxxxx>; kevin.tian@xxxxxxxxx; bhelgaas@xxxxxxxxxx;
> dmatlack@xxxxxxxxxx; kees@xxxxxxxxxx; gustavoars@xxxxxxxxxx
> Cc: Neo Jia <cjia@xxxxxxxxxx>; Krishnakant Jaju <kjaju@xxxxxxxxxx>; Vikram
> Sethi <vsethi@xxxxxxxxxx>; Zhi Wang <zhiw@xxxxxxxxxx>; linux-
> doc@xxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx; kvm@xxxxxxxxxxxxxxx;
> linux-cxl@xxxxxxxxxxxxxxx; linux-pci@xxxxxxxxxxxxxxx; linux-
> kselftest@xxxxxxxxxxxxxxx; linux-hardening@xxxxxxxxxxxxxxx
> Subject: Re: [PATCH v6 31/31] selftests/vfio: Add CXL Type-2 passthrough tests
>
> External email: Use caution opening links or attachments
>
>
> On Fri Oct 16, 2026 at 9:18 AM PDT, mhonap wrote:
> > Exercise the vfio-cxl contract on a bound Type-2 device: the CXL
> > device-info flag and the two vendor-type regions, the HDM memory mmap
> > (including a 2 MB huge fault), the read-only trapped decoder region
> > against the live host-committed decoder, and the guest's DVSEC reset
> > doorbell and manual HDM discovery.
>
> I tried running this test in a VM with the emulated cxl-type2 device from your
> QEMU RFC [1], on an x86 q35 machine with intel-iommu. Out of the box the
> device binds as plain vfio-pci and the fixture skips everything. v6 now requires
> that the host can do CPU cache invalidation for CXL Reset, and
> cpu_cache_has_invalidate_memregion() returns false in a VM (x86 refuses it
> under a hypervisor, and arm64 has no cache maintenance provider in a guest).
>
> The QEMU RFC reports 19/19 passing in an Arm guest, but that was against v5,
> which didn't have this requirement. How did you run the v6 version of this test
> without hardware? Did you use something like
> CONFIG_CXL_REGION_INVALIDATION_TEST? That option only covers region
> management in drivers/cxl/core/region.c, and depends on CXL_REGION, so it
> doesn't help here.

Thanks for running it. I did not: the 27 results in the v6 cover
letter are from hardware only, an arm64 host with the SMCCC cache
provider and firmware that implements the cache operations. You are
right that the 19/19 in the QEMU RFC was the v5 test.

>
> I added a local test-only bypass (a CONFIG_CXL_RESET_INVALIDATION_TEST
> that skips the invalidation in cxl_reset_check() and
> cxl_hdm_quiesce_prepare()/cxl_hdm_ranges_invalidate()), I get 26 passes and
> 1 skip (pm_d3hot_restores_decoder, since cxl-type2 has no PM capability in
> QEMU). I think something like that belongs in the cxl_reset series. Otherwise
> this test can only be run by people with real Type-2 hardware.
>

Good to see 26 pass on x86. cxl_reset v17 already has this: its patch
13 lets CONFIG_CXL_REGION_INVALIDATION_TEST skip the missing CPU cache
invalidation for CXL Reset too, and its cover letter runs the
cxl-type2 device in a q35 guest with that option set:

https://lore.kernel.org/r/20261105185917.2667656-14-smadhavan@xxxxxxxxxx

v7 of this series will be based on v17 (or possibly v18), so a guest kernel
with CXL_REGION and that option should run the test without a local bypass.
I will check that before posting and put it in the test documentation.

> [1] https://lore.kernel.org/linux-cxl/20261106035911.1280066-1-
> mhonap@xxxxxxxxxx/
>
> > Check that the decoder register region reads fail with EIO while
> > Memory Space is disabled, and that the decoder reads back committed
> > and HDM access works after low power entry and exit.
>
> The series says a Type-2 device loses its committed HDM decoder in D3hot and
> D3cold (patches 17 and 30), but the emulated cxl-type2 device in your QEMU
> RFC has no PM capability, so on it these tests never take the decoder restore
> path in vfio_cxl_power_up(). Please make the emulated device behave the
> same way as the real hardware across D3hot (PM capability, No_Soft_Reset,
> and whether the decoder is lost), so that the selftest exercises the same vfio-cxl
> code paths in a VM as it does on hardware.

Okay, I will handle this in next cxl-type2 version to have a more faithful hardware
validation.

>
> To help get that right: what does lspci -vv show for the PM capability of the
> device you tested with (NoSoftRst+ or NoSoftRst-), and did you observe the
> decoder actually being lost?
>

NoSoftRst+, and no. The statement in 17 and 30 is wrong for the device
I tested, and I will correct this in v7.

lspci shows "Status: D0 NoSoftRst+ PME-Enable-".
On hardware the test only took the path where the decoder is still
committed; the restore path in vfio_cxl_power_up() has not run there either.

> > Add a partial-mmap path to the vfio selftest library so the trapped
> > component BAR maps its sparse ranges, and an __iommu_map_file() helper
> > for the dma-buf mapping.
>
> Please split the library changes into separate commits ahead of the
> test: __iommu_map_file(), the sparse mmap support in vfio_pci_bar_map(),
> and each of the new libvfio helpers suggested below. That way each one can be
> reviewed on its own, and the final commit only adds the test, its Makefile
> entry, and the MAINTAINERS line.

Okay, for v7, the library change will be split out ahead of the test, one
per helper.

>
> This series is already quite long (31 patches), and with the library changes split
> out the selftest changes will grow to several patches.
> Consider sending all the VFIO selftests patches as a separate series that
> depends on this one.

I would like to keep the library and the test at the end of this series for v7,
so the test arrives with the code it covers and each version can be checked
against it.

>
> > +/* Return the sparse-mmap capability in @info, or NULL if the region
> > +has none. */ static struct vfio_region_info_cap_sparse_mmap *
> > +vfio_pci_sparse_mmap_cap(struct vfio_region_info *info) {
> > + struct vfio_info_cap_header *hdr;
> > + u32 offset;
> > +
> > + if (!(info->flags & VFIO_REGION_INFO_FLAG_CAPS))
> > + return NULL;
> > +
> > + for (offset = info->cap_offset; offset; offset = hdr->next) {
> > + hdr = (void *)info + offset;
> > + if (hdr->id == VFIO_REGION_INFO_CAP_SPARSE_MMAP)
> > + return (struct vfio_region_info_cap_sparse_mmap *)hdr;
> > + }
> > +
> > + return NULL;
> > +}
>
> This walks the region info capability chain looking for one ID, which is the same
> thing find_region_cap() does in the test. Please add one generic libvfio helper
> that looks up a region info capability by ID, and use it in both places.
>

Okay, I will update this.

> > + * Usage: ./vfio_cxl_type2_test <BDF> (or export
> VFIO_SELFTESTS_BDF=<BDF>).
> > + * The device must be bound to vfio-pci with the vfio-cxl module available.
>
> Please document how to run this test in more detail: the hardware
> requirements, how to set up the emulated QEMU device, and the kernel config
> needed (VFIO_CXL, IOMMUFD, any test-only bypass), so others can reproduce
> your results.
>

Okay, I will update this with the qemu command line and guest kernel config.

> > +/* vfio-pci region-offset packing is kernel-internal, not UAPI;
> > +define locally. */ #ifndef VFIO_PCI_OFFSET_SHIFT
> > +#define VFIO_PCI_OFFSET_SHIFT 40
> > +#endif
> > +#ifndef VFIO_PCI_INDEX_TO_OFFSET
> > +#define VFIO_PCI_INDEX_TO_OFFSET(i) ((uint64_t)(i) <<
> > +VFIO_PCI_OFFSET_SHIFT) #endif
>
> As the comment says, this is kernel-internal and not uAPI, so the test shouldn't
> depend on it. Please use vfio_region_info.offset from
> VFIO_DEVICE_GET_REGION_INFO instead, as comp_bar_sparse_mmap already
> does for the BAR. Better yet, use region access helpers in libvfio that take a
> region index and apply the offset internally (see below), so the test never
> computes region offsets itself.

Okay, I will use the region helper for this.

>
> > +/* Find a CXL region by subtype; returns the region index or -1, @buf
> > +left holding its info. */ static int find_cxl_region(int fd, uint32_t nregions,
> uint32_t subtype,
> > + void *buf, size_t bufsz) {
> > + uint32_t i;
> > +
> > + for (i = 0; i < nregions; i++) {
> > + struct vfio_region_info *ri = buf;
> > + const struct vfio_region_info_cap_type *t;
> > + const struct vfio_info_cap_header *hdr;
> > +
> > + memset(buf, 0, bufsz);
> > + ri->argsz = bufsz;
> > + ri->index = i;
> > + if (ioctl(fd, VFIO_DEVICE_GET_REGION_INFO, ri))
> > + continue;
>
> This is a generic lookup of a region by type and subtype. Only the CXL vendor ID
> is specific. Please make it a libvfio helper (e.g.
> vfio_pci_find_region(device, type, subtype)) built on the existing region info
> code in libvfio.
>
> More generally, please replace the raw ioctl() calls throughout the test with
> libvfio helpers: VFIO_DEVICE_GET_INFO, VFIO_DEVICE_GET_REGION_INFO,
> and VFIO_DEVICE_FEATURE for DMA_BUF and LOW_POWER_ENTRY/EXIT.
> Please follow the existing convention, e.g. __vfio_pci_device_reset() and
> vfio_pci_device_reset(): a double-underscore variant that returns -errno for
> the tests that expect failures or skip when something isn't supported, and a
> plain variant that asserts success.

Ack. I will update this.

>
> > +/* Walk the PCI extended capability list for the CXL Device DVSEC. */
> > +static uint16_t find_cxl_dvsec(struct vfio_pci_device *dev) {
>
> Same here, the extended capability walk and the DVSEC match aren't CXL-
> specific. Please consider vfio_pci_find_ext_capability() and
> vfio_pci_find_dvsec(device, vendor, id) helpers in libvfio, with the test passing
> in the CXL IDs.

Okay, will do.

>
> > +/* Read config space from sysfs, which is the hardware, not the vfio
> > +view. */ static int sysfs_config_read(uint32_t off, void *buf, size_t
> > +len) {
> > + char path[128];
> > + ssize_t n;
> > + int fd;
> > +
> > + snprintf(path, sizeof(path), "/sys/bus/pci/devices/%s/config",
> > + device_bdf);
> > + fd = open(path, O_RDONLY);
> > + if (fd < 0)
> > + return -errno;
> > + n = pread(fd, buf, len, off);
> > + close(fd);
> > + return n == (ssize_t)len ? 0 : -EIO; }
>
> This and sysfs_config() further down do the same thing. Please replace both
> with one sysfs config access helper in lib/sysfs.c, modeled after the existing vfio
> config helpers: a sysfs_config_access(bdf, write, offset, size, data) core with
> sysfs_config_read{b,w,l}() and
> sysfs_config_write{b,w,l}() wrappers that assert on failure, like
> vfio_pci_config_access() and vfio_pci_config_read{b,w,l}().
>
> Taking the BDF as a parameter also gets helpers off the device_bdf global.
> Please only use device_bdf where the device is opened
> (FIXTURE_SETUP() and close_restores_open_time_command). Everywhere
> else can use self->dev->bdf.

Agree for both. I will update this.

>
> > +/* The trapped block starts at the HDM decoder registers; CTRL 0
> > +reads back. */ TEST_F(vfio_cxl, comp_regs_hdm_read) {
> > + uint64_t ctrl = self->comp_off + CXL_HDM_DECODER0_CTRL_OFFSET(0);
> > + uint32_t val = 0;
> > +
> > + ASSERT_GE(self->comp_size, CXL_HDM_DECODER0_CTRL_OFFSET(0) +
> 4);
> > + ASSERT_EQ((ssize_t)sizeof(val),
> > + pread(self->dev->fd, &val, sizeof(val), ctrl)); }
>
> Please add region access helpers to libvfio instead of open-coding
> pread()/pwrite()/mmap() on the device fd, which this test does in about
> 50 places. Something modeled on the config space helpers:
> __vfio_pci_region_access(device, index, write, offset, size, data) returning the
> number of bytes or -errno, vfio_pci_region_access() asserting success, and
> typed vfio_pci_region_read{b,w,l,q}() and
> vfio_pci_region_write{b,w,l,q}() wrappers. Then this becomes:
>
> val = vfio_pci_region_readl(self->dev, self->comp_idx,
> CXL_HDM_DECODER0_CTRL_OFFSET(0));
>
> The double-underscore variant covers the expected failures, like
> comp_regs_read_needs_memory and the unaligned and out of range tests.
> vfio_pci_config_access() could become a wrapper around the same code using
> VFIO_PCI_CONFIG_REGION_INDEX. libvfio would need to cache the region info
> for all regions, not just config space and the BARs, so it can apply the offset
> reported by VFIO_DEVICE_GET_REGION_INFO.
>
> Similarly, __vfio_pci_region_mmap() and vfio_pci_region_mmap() could cover
> the HDM region mappings and the expected mmap failures over the decoder
> block, and vfio_pci_bar_map() could be built on them.
>
> > +/* Find a standard capability through the vfio config view. */ static
> > +uint8_t find_std_cap(struct vfio_pci_device *dev, uint8_t id)
>
> Please make this a libvfio helper too, e.g.
> vfio_pci_find_capability(device, cap_id).

Okay; I will update this.
Region access and mmap helpers with double-underscore variants for the
expected failures, region info cached for every region, and
vfio_pci_config_access() and vfio_pci_bar_map() built on them.

Thanks,
Manish