Re: [PATCH RFC v2 03/11] iommu/sva: Add iommu_domain type for SVA

[Lu Baolu]

Hi Jason and Kevin,

On 2022/4/3 7:32, Jason Gunthorpe wrote:
> On Sat, Apr 02, 2022 at 08:43:16AM +0000, Tian, Kevin wrote:
>
> > > This assumes any domain is interchangeable with any device, which is
> > > not the iommu model. We need a domain op to check if a device is
> > > compatiable with the domain for vfio an iommufd, this should do the
> > > same.
> >
> > This suggests that mm_struct needs to include the format information
> > of the CPU page table so the format can be checked by the domain op?
>
> No, Linux does not support multiple formats for CPU page tables,
> AFAICT, and creating the SVA domain in the first place should check
> this.
>
> > > It means each mm can have a list of domains associated with it and a
> > > new domain is auto-created if the device doesn't work with any of the
> > > existing domains.
> >
> > mm has only one page table and one format. If a device is incompatible
> > with an existing domain wrapping that page table, how come creating
> > another domain could make it compatible?
>
> Because domains wrap more than just the IOPTE format, they have
> additional data related to the IOMMU HW block itself. Imagine a SOC
> with two IOMMU HW blocks that can both process the CPU IOPTE format,
> but have different configuration.
>
> So if device A users IOMMU A it needs an iommu_domain from driver A and
> same for another device B, even if both iommu_domains are thin
> wrappers around the same mm_struct.

How about below data structure design?

- [New]struct iommu_sva_ioas
 Represent the I/O address space shared with an application CPU address
 space. This structure has a 1:1 relationship with an mm_struct. It
 graps a "mm->mm_count" refcount during creation and drop it on release.

struct iommu_sva_ioas {
        struct mm_struct *mm;
        ioasid_t pasid;

        /* Counter of domains attached to this ioas. */
        refcount_t users;

        /* All bindings are linked here. */
        struct list_head bonds;
};

- [Enhance existing] struct iommu_domain (IOMMU_DOMAIN_SVA type)
 Represent a hardware pagetable that the IOMMU hardware could use for
 SVA translation. Multiple iommu domains could be bound with an SVA ioas
 and each graps a refcount from ioas in order to make sure ioas could
 only be freed after all domains have been unbound.

@@ -95,6 +101,7 @@ struct iommu_domain {
        void *handler_token;
        struct iommu_domain_geometry geometry;
        struct iommu_dma_cookie *iova_cookie;
+       struct iommu_sva_ioas *sva_ioas;
 };


- [Enhance existing] struct iommu_sva
  Represent a bond relationship between an SVA ioas and an iommu domain.
  If a bond already exists, it's reused and a reference is taken.

/**
 * struct iommu_sva - handle to a device-mm bond
 */
struct iommu_sva {
        struct device           *dev;
        struct iommu_sva_ioas   *sva_ioas;
        struct iommu_domain     *domain;
        /* Link to sva ioas's bonds list */
        struct list_head        node;
        refcount_t              users;
};

Best regards,
baolu