Re: [PATCH 21/21] vdpasim: control virtqueue support

[Jason Wang]

On 2021/1/23 上午3:43, Eugenio Perez Martin wrote:
> On Tue, Jan 12, 2021 at 4:12 AM Jason Wang <jasowang@xxxxxxxxxx> wrote:
> > On 2021/1/11 下午8:26, Eli Cohen wrote:
> > > On Wed, Dec 16, 2020 at 02:48:18PM +0800, Jason Wang wrote:
> > > > This patch introduces the control virtqueue support for vDPA
> > > > simulator. This is a requirement for supporting advanced features like
> > > > multiqueue.
> > > >
> > > > A requirement for control virtqueue is to isolate its memory access
> > > > from the rx/tx virtqueues. This is because when using vDPA device
> > > > for VM, the control virqueue is not directly assigned to VM. Userspace
> > > > (Qemu) will present a shadow control virtqueue to control for
> > > > recording the device states.
> > > >
> > > > The isolation is done via the virtqueue groups and ASID support in
> > > > vDPA through vhost-vdpa. The simulator is extended to have:
> > > >
> > > > 1) three virtqueues: RXVQ, TXVQ and CVQ (control virtqueue)
> > > > 2) two virtqueue groups: group 0 contains RXVQ and TXVQ; group 1
> > > >      contains CVQ
> > > > 3) two address spaces and the simulator simply implements the address
> > > >      spaces by mapping it 1:1 to IOTLB.
> > > >
> > > > For the VM use cases, userspace(Qemu) may set AS 0 to group 0 and AS 1
> > > > to group 1. So we have:
> > > >
> > > > 1) The IOTLB for virtqueue group 0 contains the mappings of guest, so
> > > >      RX and TX can be assigned to guest directly.
> > > > 2) The IOTLB for virtqueue group 1 contains the mappings of CVQ which
> > > >      is the buffers that allocated and managed by VMM only. So CVQ of
> > > >      vhost-vdpa is visible to VMM only. And Guest can not access the CVQ
> > > >      of vhost-vdpa.
> > > >
> > > > For the other use cases, since AS 0 is associated to all virtqueue
> > > > groups by default. All virtqueues share the same mapping by default.
> > > >
> > > > To demonstrate the function, VIRITO_NET_F_CTRL_MACADDR is
> > > > implemented in the simulator for the driver to set mac address.
> > > Hi Jason,
> > >
> > > is there any version of qemu/libvirt available that I can see the
> > > control virtqueue working in action?
> >
> > Not yet, the qemu part depends on the shadow virtqueue work of Eugenio.
> > But it will work as:
> >
> > 1) qemu will use a separated address space for the control virtqueue
> > (shadow) exposed through vhost-vDPA
> > 2) the commands sent through control virtqueue by guest driver will
> > intercept by qemu
> > 3) Qemu will send those commands to the shadow control virtqueue
> >
> > Eugenio, any ETA for the new version of shadow virtqueue support in Qemu?
> Hi Jason. Sorry for the late response.
>
> For the notification part I have addressed all the issues of the RFC
> [1], except the potential race conditions Stefan pointed, and tested
> with vdpa devices. You can find at
> https://github.com/eugpermar/qemu/tree/vdpa_sw_live_migration.d/notifications.rfc
> . Since the shadow path is activated only through QMP and does not
> interfere with regular operation, I could post to the qemu list if you
> prefer. The series will be smaller if merged in steps.


Sure. Please post them.


> Adding the buffer forwarding on top should not take long.
>
> [1] https://lkml.org/lkml/2020/9/23/1243
>
> Thanks!


Thanks