Re: [PATCH v6 16/21] vfio-pci/zdev: add open/close device hooks

[Matthew Rosato]

On 4/27/22 11:01 AM, Jason Gunthorpe wrote:
> On Wed, Apr 27, 2022 at 10:42:07AM -0400, Matthew Rosato wrote:
> > On 4/27/22 10:04 AM, Jason Gunthorpe wrote:
> > > On Tue, Apr 26, 2022 at 04:08:37PM -0400, Matthew Rosato wrote:
> > >
> > > > +static int vfio_pci_zdev_group_notifier(struct notifier_block *nb,
> > > > +					unsigned long action, void *data)
> > > > +{
> > > > +	struct zpci_dev *zdev = container_of(nb, struct zpci_dev, nb);
> > > > +	int (*fn)(struct zpci_dev *zdev, struct kvm *kvm);
> > > > +	int rc = NOTIFY_OK;
> > > > +
> > > > +	if (action == VFIO_GROUP_NOTIFY_SET_KVM) {
> > > > +		if (!zdev)
> > > > +			return NOTIFY_DONE;
> > > > +
> > > > +		fn = symbol_get(kvm_s390_pci_register_kvm);
> > > > +		if (!fn)
> > > > +			return NOTIFY_DONE;
> > > > +
> > > > +		if (fn(zdev, (struct kvm *)data))
> > > > +			rc = NOTIFY_BAD;
> > > > +
> > > > +		symbol_put(kvm_s390_pci_register_kvm);
> > >
> > > Is it possible this function can be in statically linked arch code?
> > >
> > > Or, actually, is zPCI useful anyhow without kvm ie can you just have a
> > > direct dependency here?
> >
> > zPCI devices (zpci_dev) exist regardless of whether kvm is configured or
> > not, and you can e.g. bind the associated PCI device to vfio-pci when KVM is
> > not configured (or module not loaded) and get the existing vfio-pci-zdev
> > extensions for that device (extra VFIO_DEVICE_INFO response data).  Making a
> > direct dependency on KVM would remove that; this was discussed in a prior
> > version because this extra info is not used today outside of a KVM usecase
> > are not specific to kvm that need vfio-pci-zdev).
>
> I'm a bit confused, what is the drawback of just having a direct
> symbol dependency here? It means vfio loads a little extra kernel
> module code, but is that really a big worry given almost all vfio
> users on s390 will be using it with kvm?

It's about trying to avoid loading unnecessary code (or at least giving 
a way to turn it off).

Previously I did something like....

https://lore.kernel.org/kvm/20220204211536.321475-15-mjrosato@xxxxxxxxxxxxx/

And could do so again; as discussed in the thread there, I can use e.g. 
CONFIG_VFIO_PCI_ZDEV_KVM and make vfio-pci-zdev depend on KVM in this 
series.  You only get the vfio-pci-zdev extensions when you configure KVM.

Then if we find a usecase for a vfio-pci-zdev extension without KVM 
later, we can further split vfio-pci-zdev and go back to building 
vfio-pci-zdev when CONFIG_S390 but only build the kvm pieces (like the 
code above) when you specify CONFIG_VFIO_PCI_ZDEV_KVM.  This would 
eliminate this symbol_get.  Sound OK?

> Or is there some technical blocker? (circular dep or something?)
>
> Jason