Re: [PATCH] virtio-balloon spec: provide a version of the "silentdeflate" feature that works

[Michael S. Tsirkin]

On Mon, Sep 10, 2012 at 11:13:12AM +0930, Rusty Russell wrote:
> "Michael S. Tsirkin" <mst@xxxxxxxxxx> writes:
> > On Sat, Sep 08, 2012 at 02:36:00PM +0930, Rusty Russell wrote:
> >> "Michael S. Tsirkin" <mst@xxxxxxxxxx> writes:
> >> > On Fri, Sep 07, 2012 at 04:09:50PM +0930, Rusty Russell wrote:
> >> >> > So it looks like a bug: we should teach driver to tell host first on leak?
> >> >> > Yan, Vadim, can you comment please?
> >> >> >
> >> >> > Also if true, looks like this bit will be useful to detect a fixed driver on
> >> >> > the hypervisor side - to avoid unmapping such pages? Rusty what do you
> >> >> > think?
> >> >> 
> >> >> So, feature is unimplemented in qemu, and broken in drivers.  I starting
> >> >> to share Paolo's dislike of it.
> >> >
> >> > What is broken in drivers?
> >> 
> >> Because supporting the feature is *not* optional for a driver.
> >> 
> >> If the device said MUST_TELL_HOST, it meant that the driver *had* to
> >> tell the host before it touched the page, otherwise Bad Things might
> >> happen.  It was in the original spec precisely to allow devices to
> >> actually *remove* pages.
> >> 
> >> Noone ever noticed the windows driver didn't support it, because qemu
> >> never requires MUST_TELL_HOST.
> >> 
> >> So in practice, it's now an optional feature.  Since no device used it
> >> anyway, we're better off discarding it than trying to fix it.
> >
> > I trust you this was not the intent. But it seems to be
> > the intent in spec, because almost all features are optional.
> >
> > And so windows driver authors interpreted it
> > this way. And it is *useful* like this.  See below.
> 
> ...
> 
> > Suggested use is for device assignment which needs all guest memory
> > locked.  hypervisor can unlock pages in balloon but guest must wait for
> > hypervisor to lock them back before use.
> >
> > when a hypervisor implements this,
> > this will work with linux guests but not windows
> > guests and the existing bit fits exactly the purpose.
> 
> If a hypervisor needs this, and the guest doesn't support it, then
> the hypervisor can only abandon the balloon device.  That's not my
> definition of "optional".
> 
> >> > Do we really know there are no hypervisors implementing it?
> >> 
> >> As much as can be known.  Qemu doesn't, lkvm doesn't.
> >
> > But we can add it in qemu and it will be a useful feature.
> >
> >> > As I said above drivers do have support.
> >> 
> >> Not the windows drivers.  So it's optional, thus removing it will likely
> >> harm noone.
> >> 
> >> Cheers,
> >> Rusty.
> >
> > I think the issue is that kvm always locked all guest memory
> > for assignment. This restriction is removed
> > with vfio which has separate page tables.
> > Now that vfio is upstream and work on qemu integration
> > is being worked on, we might finally see people using this bit
> > to allow memory overcommit with device assignment.
> 
> That was left-field.... so you're saying some guest might pull a page
> from the balloon and DMA to it, and the vfio device needs to know in
> advance that it's going to do it?
> 
> But what will we do if the guest doesn't ack the bit?
> ie. I don't think it can really be optional.
> 
> Cheers,
> Rusty.

We have several options:
1. No memory overcommit feature. This is always the case ATM!
2. Do not hot-plug assigned device.
3. Hot-unplug assigned device.
4. Some assigned devices can be able to handle memory errors
   e.g. using ATS. Limit hotplug to these.

> I don't think it can really be optional.

It is optional *for the device*.

But I don't insist on my patch. I am merely saying that
1. The bit is useful for host to detect guests
which can't combine memory overcommit with device assignment,
and this set of guests is not empty.

2. The fact that this bit is not optional for drivers is not well documented.
The only hint seems the use of words "feature is set" as
opposed to "feature is negoticated" as with other features.
The spec intended
"feature is set in Device Features bits". Drivers interpreted this
as "feature is set in Guest Features bits".

Hard to blame them, let us give them time to address this.

-- 
MST
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/