RE: [patch 21/32] NTB/msi: Convert to msi_on_each_desc()

[Tian, Kevin]

> From: Jason Gunthorpe <jgg@xxxxxxxxxx>
> Sent: Saturday, December 4, 2021 12:41 AM
> 
> > Or has each queue and controlblock and whatever access to a shared large
> > array where the messages are stored and the indices are handed out to
> > the queues and controlblocks?
> 
> > If each of them have their own small array, then queue relative indexing
> > makes a ton of sense, no?
> 
> Okay, I see.
> 
> I don't know of any use case for more than one interrupt on a queue,
> and if it did come up I'd probably approach it by making the queue
> handle above also specify the 'queue relative HW index'
> 

We have such use case with IDXD.

Basically the IDXD queue allows software to put an interrupt handle
(the index of MSI-X or IMS entry) in the submitted descriptor. Upon
completion of the descriptor the hardware finds the specified entry 
and then generate interrupt to notify software.

Conceptually descriptors submitted to a same queue can use different
handles, implying one queue can be associated to multiple interrupts.

One example is the shared work queue usage which allows multiple 
clients directly and simultaneously submitting descriptors to the 
same queue, by using ENQCMD(pasid, descriptor) instruction. In this 
case each client can be allocated with an interrupt entry (including the
information about the client's pasid for permission check when the
HW generates completion interrupt) and then use this entry for 
all descriptors submitted by that client.

Haven't completed reading of this thread, but would like to point 
out this usage so it is not ignored in the final rework. It basically
means one queue might be associated to multiple interrupt entries
and multiple pasids. 😊

Thanks
Kevin