Re: [PATCH 1/2] cxl/pci: Add generic MSI-X/MSI irq support

[Jonathan Cameron]

On Wed, 2 Nov 2022 10:15:24 -0700
Davidlohr Bueso <dave@xxxxxxxxxxxx> wrote:

> On Tue, 25 Oct 2022, Bjorn Helgaas wrote:
> 
> >> In short that calls:
> >> /* Allocate the maximum possible number of MSI/MSI-X vectors */
> >> nr_entries = pci_alloc_irq_vectors(dev, 1, PCIE_PORT_MAX_MSI_ENTRIES,
> >>			PCI_IRQ_MSIX | PCI_IRQ_MSI);
> >>
> >> /* See how many and which Interrupt Message Numbers we actually use */
> >> nvec = pcie_message_numbers(dev, mask, &pme, &aer, &dpc);
> >>
> >> if (nvec != nr_entries) {
> >>	pci_free_irq_vectors(dev);
> >>
> >>	nr_entries = pci_alloc_irq_vectors(dev, nvec, nvec,
> >>			PCI_IRQ_MSIX | PCI_IRQ_MSI);
> >> }
> >>
> >> My worry here is that the implicit assumption is that the vectors
> >> won't move if we reduce the overall number of vectors we are asking
> >> for...  
> 
> This would also apply to what is currently in portdrv machinery, no?
> 
> >>
> >> However, imagine the case that we have a feature the driver doesn't
> >> know about that was previously at a higher vector.  After reducing
> >> the vectors allocated the hardware might decide that feature needs
> >> its own vector whereas some others can be combined.  Hence we'd end
> >> up with a less than ideal packing for the features we actually
> >> support.
> >>
> >> Could do something iterative to solve this if it actually matters
> >> (increase number of vectors until the layout matches what we get
> >> with max possible vectors).  
> 
> Maybe do a bounded retry loop until we get stable value?
> 
> retry = 1;
> do {
> 	pci_alloc_irq_vectors(1, 32);
> 	nvecs = get_max_msgnum(); // max(pmu, events, mbox, isolation)
> 	pci_free_irq_vectors();
> 
> 	pci_alloc_irq_vectors(nvecs, nvecs);
> 	new_nvecs = get_max_msgnum();
> 
> 	if (likely(new_nvecs == nvecs))
> 		return 0;
> 
> 	pci_free_irq_vectors();
> }  while (retry--);
> 
> return -1; // no irq support

Yup. That's pretty much what I was thinking - if we care :)

> 
> But yeah I'm not sure how much we actually care about this. 

That was my feeling. This might be worth a comment to say that
it's not guaranteed to be optimal (in portdrv), but probably 
a won't fix.

Jonathan