Re: [PATCH] s390/vfio-ap: fix memory leak in mdev remove callback

[Tony Krowiak]

On 5/6/21 6:45 AM, Cornelia Huck wrote:
> On Thu, 6 May 2021 12:22:45 +0200
> Cornelia Huck <cohuck@xxxxxxxxxx> wrote:
>
> > On Wed,  5 May 2021 13:28:26 -0400
> > Tony Krowiak <akrowiak@xxxxxxxxxxxxx> wrote:
> >
> > > The mdev remove callback for the vfio_ap device driver bails out with
> > > -EBUSY if the mdev is in use by a KVM guest. The intended purpose was
> > > to prevent the mdev from being removed while in use; however, returning a
> > > non-zero rc does not prevent removal. This could result in a memory leak
> > > of the resources allocated when the mdev was created. In addition, the
> > > KVM guest will still have access to the AP devices assigned to the mdev
> > > even though the mdev no longer exists.
> > >
> > > To prevent this scenario, cleanup will be done - including unplugging the
> > > AP adapters, domains and control domains - regardless of whether the mdev
> > > is in use by a KVM guest or not.
> > >
> > > Fixes: 258287c994de ("s390: vfio-ap: implement mediated device open callback")
> > > Cc: stable@xxxxxxxxxxxxxxx
> > > Signed-off-by: Tony Krowiak <akrowiak@xxxxxxxxxxx>
> > > Signed-off-by: Tony Krowiak <akrowiak@xxxxxxxxxxxxx>
> > > ---
> > >   drivers/s390/crypto/vfio_ap_ops.c | 39 +++++++++++++++++++++++--------
> > >   1 file changed, 29 insertions(+), 10 deletions(-)
> > >
> > > diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c
> > > index b2c7e10dfdcd..757166da947e 100644
> > > --- a/drivers/s390/crypto/vfio_ap_ops.c
> > > +++ b/drivers/s390/crypto/vfio_ap_ops.c
> > > @@ -335,6 +335,32 @@ static void vfio_ap_matrix_init(struct ap_config_info *info,
> > >   	matrix->adm_max = info->apxa ? info->Nd : 15;
> > >   }
> > >   
> > > +static bool vfio_ap_mdev_has_crycb(struct ap_matrix_mdev *matrix_mdev)
> > > +{
> > > +	return (matrix_mdev->kvm && matrix_mdev->kvm->arch.crypto.crycbd);
> > > +}
> > > +
> > > +static void vfio_ap_mdev_clear_apcb(struct ap_matrix_mdev *matrix_mdev)
> > > +{
> > > +	/*
> > > +	 * If the KVM pointer is in the process of being set, wait until the
> > > +	 * process has completed.
> > > +	 */
> > > +	wait_event_cmd(matrix_mdev->wait_for_kvm,
> > > +		       !matrix_mdev->kvm_busy,
> > > +		       mutex_unlock(&matrix_dev->lock),
> > > +		       mutex_lock(&matrix_dev->lock));
> > > +
> > > +	if (vfio_ap_mdev_has_crycb(matrix_mdev)) {
> > > +		matrix_mdev->kvm_busy = true;
> > > +		mutex_unlock(&matrix_dev->lock);
> > > +		kvm_arch_crypto_clear_masks(matrix_mdev->kvm);
> > > +		mutex_lock(&matrix_dev->lock);
> > > +		matrix_mdev->kvm_busy = false;
> > > +		wake_up_all(&matrix_mdev->wait_for_kvm);
> > > +	}
> > > +}
> > Looking at vfio_ap_mdev_unset_kvm(), do you need to unhook the kvm here
> > as well?
> >
> > (Or can you maybe even combine the two functions into one?)
> Staring at the code some more, the rules where you unset the kvm stuff
> seem pretty confusing (at least to me). Does this partial unhooking in
> the remove callback make sense?

If you stare at it too long, you'll go blind:) As I stated in my response
to your previous review comment, I'm going to remove the function
above and call the vfio_ap_mdev_unset_kvm() function from the
remove callback.

> > > +
> > >   static int vfio_ap_mdev_create(struct mdev_device *mdev)
> > >   {
> > >   	struct ap_matrix_mdev *matrix_mdev;
> > > @@ -366,16 +392,9 @@ static int vfio_ap_mdev_remove(struct mdev_device *mdev)
> > >   	struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev);
> > >   
> > >   	mutex_lock(&matrix_dev->lock);
> > > -
> > > -	/*
> > > -	 * If the KVM pointer is in flux or the guest is running, disallow
> > > -	 * un-assignment of control domain.
> > > -	 */
> > > -	if (matrix_mdev->kvm_busy || matrix_mdev->kvm) {
> > > -		mutex_unlock(&matrix_dev->lock);
> > > -		return -EBUSY;
> > > -	}
> > > -
> > > +	WARN(vfio_ap_mdev_has_crycb(matrix_mdev),
> > > +	     "Removing mdev leaves KVM guest without any crypto devices");
> > > +	vfio_ap_mdev_clear_apcb(matrix_mdev);
> > >   	vfio_ap_mdev_reset_queues(mdev);
> > >   	list_del(&matrix_mdev->node);
> > >   	kfree(matrix_mdev);