Re: [patch 21/32] NTB/msi: Convert to msi_on_each_desc()

[Dave Jiang]

On 12/1/2021 11:41 AM, Thomas Gleixner wrote:
> Dave,
>
> please trim your replies.
>
> On Wed, Dec 01 2021 at 09:28, Dave Jiang wrote:
>
> > On 12/1/2021 3:16 AM, Thomas Gleixner wrote:
> > > Jason,
> > >
> > > CC+ IOMMU folks
> > >
> > > On Tue, Nov 30 2021 at 20:17, Jason Gunthorpe wrote:
> > > > On Tue, Nov 30, 2021 at 10:23:16PM +0100, Thomas Gleixner wrote:
> > > Though I fear there is also a use case for MSI-X and IMS tied to the
> > > same device. That network card you are talking about might end up using
> > > MSI-X for a control block and then IMS for the actual network queues
> > > when it is used as physical function device as a whole, but that's
> > > conceptually a different case.
> > Hi Thomas. This is actually the IDXD usage for a mediated device passed
> > to a guest kernel when we plumb the pass through of IMS to the guest
> > rather than doing previous implementation of having a MSIX vector on
> > guest backed by IMS.
> Which makes a lot of sense.
>
> > The control block for the mediated device is emulated and therefore an
> > emulated MSIX vector will be surfaced as vector 0. However the queues
> > will backed by IMS vectors. So we end up needing MSIX and IMS coexist
> > running on the guest kernel for the same device.
> Why? What's wrong with using straight MSI-X for all of them?

The hardware implementation does not have enough MSIX vectors for 
guests. There are only 9 MSIX vectors total (8 for queues) and 2048 IMS 
vectors. So if we are to do MSI-X for all of them, then we need to do 
the IMS backed MSIX scheme rather than passthrough IMS to guests.


> Thanks,
>
>          tglx