Re: [PATCH RFC 1/2] kvm: Handle undercommitted guest case in PLEhandler

[Raghavendra K T]

On 10/10/2012 08:29 AM, Andrew Theurer wrote:
> On Wed, 2012-10-10 at 00:21 +0530, Raghavendra K T wrote:
> > * Avi Kivity <avi@xxxxxxxxxx> [2012-10-04 17:00:28]:
> >
> > > On 10/04/2012 03:07 PM, Peter Zijlstra wrote:
> > > > On Thu, 2012-10-04 at 14:41 +0200, Avi Kivity wrote:
> > > > > Again the numbers are ridiculously high for arch_local_irq_restore.
> > > > > Maybe there's a bad perf/kvm interaction when we're injecting an
> > > > > interrupt, I can't believe we're spending 84% of the time running the
> > > > > popf instruction.
> > > >
> > > > Smells like a software fallback that doesn't do NMI, hrtimer based
> > > > sampling typically hits popf where we re-enable interrupts.
> > >
> > > Good nose, that's probably it.  Raghavendra, can you ensure that the PMU
> > > is properly exposed?  'dmesg' in the guest will tell.  If it isn't, -cpu
> > > host will expose it (and a good idea anyway to get best performance).
> >
> > Hi Avi, you are right. SandyBridge machine result was not proper.
> > I cleaned up the services, enabled PMU, re-ran all the test again.
> >
> > Here is the summary:
> > We do get good benefit by increasing ple window. Though we don't
> > see good benefit for kernbench and sysbench, for ebizzy, we get huge
> > improvement for 1x scenario. (almost 2/3rd of ple disabled case).
> >
> > Let me know if you think we can increase the default ple_window
> > itself to 16k.
> >
> > I am experimenting with V2 version of undercommit improvement(this) patch
> > series, But I think if you wish  to go for increase of
> > default ple_window, then we would have to measure the benefit of patches
> > when ple_window = 16k.
> >
> > I can respin the whole series including this default ple_window change.
> >
> > I also have the perf kvm top result for both ebizzy and kernbench.
> > I think they are in expected lines now.
> >
> > Improvements
> > ================
> >
> > 16 core PLE machine with 16 vcpu guest
> >
> > base = 3.6.0-rc5 + ple handler optimization patches
> > base_pleopt_16k = base + ple_window = 16k
> > base_pleopt_32k = base + ple_window = 32k
> > base_pleopt_nople = base + ple_gap = 0
> > kernbench, hackbench, sysbench (time in sec lower is better)
> > ebizzy (rec/sec higher is better)
> >
> > % improvements w.r.t base (ple_window = 4k)
> > ---------------+---------------+-----------------+-------------------+
> >                 |base_pleopt_16k| base_pleopt_32k | base_pleopt_nople |
> > ---------------+---------------+-----------------+-------------------+
> > kernbench_1x   |  0.42371      |  1.15164        |   0.09320         |
> > kernbench_2x   | -1.40981      | -17.48282       |  -570.77053       |
> > ---------------+---------------+-----------------+-------------------+
> > sysbench_1x    | -0.92367      | 0.24241         | -0.27027          |
> > sysbench_2x    | -2.22706      |-0.30896         | -1.27573          |
> > sysbench_3x    | -0.75509      | 0.09444         | -2.97756          |
> > ---------------+---------------+-----------------+-------------------+
> > ebizzy_1x      | 54.99976      | 67.29460        |  74.14076         |
> > ebizzy_2x      | -8.83386      |-27.38403        | -96.22066         |
> > ---------------+---------------+-----------------+-------------------+
> >
> > perf kvm top observation for kernbench and ebizzy (nople, 4k, 32k window)
> > ========================================================================
>
> Is the perf data for 1x overcommit?

Yes, 16vcpu guest on 16 core

> > pleopt   ple_gap=0
> > --------------------
> > ebizzy : 18131 records/s
> > 63.78%  [guest.kernel]  [g] _raw_spin_lock_irqsave
> >      5.65%  [guest.kernel]  [g] smp_call_function_many
> >      3.12%  [guest.kernel]  [g] clear_page
> >      3.02%  [guest.kernel]  [g] down_read_trylock
> >      1.85%  [guest.kernel]  [g] async_page_fault
> >      1.81%  [guest.kernel]  [g] up_read
> >      1.76%  [guest.kernel]  [g] native_apic_mem_write
> >      1.70%  [guest.kernel]  [g] find_vma
>
> Does 'perf kvm top' not give host samples at the same time?  Would be
> nice to see the host overhead as a function of varying ple window.  I
> would expect that to be the major difference between 4/16/32k window
> sizes.

No, I did something like this
perf kvm  --guestvmlinux ./vmlinux.guest top -g  -U -d 3. Yes that is a
good idea.

(I am getting some segfaults with perf top, I think it is already fixed
but yet to see the patch that fixes)



> A big concern I have (if this is 1x overcommit) for ebizzy is that it
> has just terrible scalability to begin with.  I do not think we should
> try to optimize such a bad workload.

I think my way of running dbench has some flaw, so I went to ebizzy.
Could you let me know how you generally run dbench?

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/