Re: [ANNOUNCE] Native Linux KVM tool v2

From: Ingo Molnar
Date: Thu Jun 16 2011 - 03:25:22 EST

* Pekka Enberg <penberg@xxxxxxxxxx> wrote:

> On Thu, Jun 16, 2011 at 1:07 AM, Alexander Graf <agraf@xxxxxxx> wrote:
> >> qemu-system-x86_64 -drive file=/dev/shm/test.qcow2,cache=writeback,if=virtio
> >
> > Wouldn't this still be using threaded AIO mode? I thought KVM tools used native AIO?
> We don't use AIO at all. It's just normal read()/write() with a
> thread pool. I actually looked at AIO but didn't really see why
> we'd want to use it.

We could certainly try kernel AIO, it would allow us to do all the
virtio-blk logic from the vcpu thread, without single threading it -
turning the QCOW2 logic into an AIO driven state machine in essence.


- we wouldnt do context-switching between the vcpu thread and the
helper threads

- it would potentially provide tighter caching and potentially would
allow higher scalability.


- the kaio codepaths are actually *more* complex than the regular
read()/write() IO codepaths - they keep track of an 'IO context',
so part of the efficiency advantages are spent on AIO tracking.

- executing AIO in the vcpu thread eats up precious vcpu execution
time: combined QCOW2 throughput would be limited by a single
core's performance, and any time spent on QCOW2 processing would
not be spent running the guest CPU. (In such a model we certainly
couldnt do more intelligent, CPU-intense storage solutions like on
the fly compress/decompress of QCOW2 data.)

- state machines are also fragile in the sense that any
unintentional blocking of the vcpu context will kill the
performance and latencies of *all* processing in certain
circumstances. So we generally strive to keep the vcpu demux path
obvious, simple and atomic.

- more advanced security models go out the window as well: we
couldnt isolate drivers from each other if all of them execute in
the same vcpu context ...

- state machines are also notoriously difficult to develop,
debug and maintain.

So careful performance, scalability, IO delay and maintainability
measurements have to accompany such a model switch, because the
disadvantages are numerous.

I'd only consider KAIO it if it provides some *real* measurable
performance advantage of at least 10% in some important usecase.
A few percent probably wouldnt be worth it.


To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at
Please read the FAQ at