PCI/kernel msi code vs GIC ITS driver conflict?

[John Garry]

Hi Marc, Bjorn, Thomas,

We've come across a conflict with the kernel/pci msi code and GIC ITS 
driver on our arm64 system, whereby we can't unbind and re-bind a PCI 
device driver under special conditions. I'll explain...

Our PCI device support 32 MSIs. The driver attempts to allocate msi 
vectors with min msi=17, max msi = 32, and affd.pre vectors = 16. For 
our test we make nr_cpus = 1 (just anything less than 16).

We find that the pci/kernel msi code gives us 17 vectors, but the GIC 
ITS code reserves 32 lpi maps in its_irq_domain_alloc(). The problem 
then occurs when unbinding the driver in its_irq_domain_free() call, 
where we only clear bits for 17 vectors. So if we unbind the driver and 
then attempt to bind again, it fails.

Where the fault lies, I can't say. Maybe the kernel msi code should 
always give power of 2 vectors - as I understand, the PCI spec mandates 
this. Or maybe the GIC ITS driver has a problem in the free path, as 
above. Or maybe the PCI driver should not be allowed to request !power 
of 2 min/max vectors.

Opinion?

Thanks in advance,
John