Re: [PATCH v3 0/8] vfio/mdev: IOMMU aware mediated device

From: Xu Zaibo
Date: Tue Oct 16 2018 - 22:02:32 EST


Hi,

On 2018/10/16 9:21, Lu Baolu wrote:
Hi,

On 10/15/2018 04:50 PM, Xu Zaibo wrote:
Hi,

On 2018/10/15 10:48, Lu Baolu wrote:
Hi,

On 10/13/2018 04:25 PM, Xu Zaibo wrote:
Hi,

On 2018/10/12 13:16, Lu Baolu wrote:
Hi,

The Mediate Device is a framework for fine-grained physical device
sharing across the isolated domains. Currently the mdev framework
is designed to be independent of the platform IOMMU support. As the
result, the DMA isolation relies on the mdev parent device in a
vendor specific way.

There are several cases where a mediated device could be protected
and isolated by the platform IOMMU. For example, Intel vt-d rev3.0
[1] introduces a new translation mode called 'scalable mode', which
enables PASID-granular translations. The vt-d scalable mode is the
key ingredient for Scalable I/O Virtualization [2] [3] which allows
sharing a device in minimal possible granularity (ADI - Assignable
Device Interface).

A mediated device backed by an ADI could be protected and isolated
by the IOMMU since 1) the parent device supports tagging an unique
PASID to all DMA traffic out of the mediated device; and 2) the DMA
translation unit (IOMMU) supports the PASID granular translation.
We can apply IOMMU protection and isolation to this kind of devices
just as what we are doing with an assignable PCI device.

In order to distinguish the IOMMU-capable mediated devices from those
which still need to rely on parent devices, this patch set adds two
new members in struct mdev_device.

* iommu_device
- This, if set, indicates that the mediated device could
be fully isolated and protected by IOMMU via attaching
an iommu domain to this device. If empty, it indicates
using vendor defined isolation.

* iommu_domain
- This is a place holder for an iommu domain. A domain
could be store here for later use once it has been
attached to the iommu_device of this mdev.

Below helpers are added to set and get above iommu device
and iommu domain pointers in mdev core implementation.

* mdev_set/get_iommu_device(dev, iommu_device)
- Set or get the iommu device which represents this mdev
in IOMMU's device scope. Drivers don't need to set the
iommu device if it uses vendor defined isolation.

* mdev_set/get_iommu_domain(domain)
- A iommu domain which has been attached to the iommu
device in order to protect and isolate the mediated
device will be kept in the mdev data structure and
could be retrieved later.

The mdev parent device driver could opt-in that the mdev could be
fully isolated and protected by the IOMMU when the mdev is being
created by invoking mdev_set_iommu_device() in its @create().
I just cannot understand here, how to get an iommu_device while I create mediated
device in my parent device driver?

When you are creating an mdev in your parent driver, you should know
which PCI device this mdev belonging to.


So, generally, I can set the parent device as mdev's iommu_device?
If that, however, Mdev already holds its parent device. So, I just figure what
differences between Mdev's parent device and iommu_device are.

And why not reuse the device of MDEV instread of adding a new device here?

iommu_device in the mdev_device structure represents the PCI device
that represents this mdev in iommu's device scope. IOMMU is only aware
of pci devices, it's not aware of mdev device.

Could I understand like that: IOMMU can be aware of the parent device of Mdev?
And more, I am doubting the necessary of iommu_device in Mdev.


The "mdev parent device" and "mdev iommu device" are different although
they might be the same in practice. "mdev parent device" represents the
device who created the mdev. "mdev iommu device" represents the device
who shares the device context entry in iommu tables.

"mdev iommu device" is always a PCI/PCIe device since IOMMU always use
source id (bus:dev:func) to walk the device context table. But there is
no limitation on who can create an mdev, right?

Actually, I am not sure.

My understanding:
The DMA address will be issued by the parent device with PASID or something like that to IOMMU
facilities. However, the translation units such as iommu ïPASID/page .etxïtables are from another
device node. I cannot figure out how to control this in hardware level, or whether there will be
conflicts between the DMA transation of iommu_device and parent device.

Thanks,
Zaibo

.