Re: [RFC PATCH v3 00/16] Core scheduling v3

From: Dario Faggioli
Date: Tue Oct 29 2019 - 05:17:04 EST

On Tue, 2019-10-29 at 10:11 +0100, Dario Faggioli wrote:
> On Sun, 2019-09-15 at 22:14 +0800, Aaron Lu wrote:
> > I'm using the following branch as base which is v5.1.5 based:
> > https://github.com/digitalocean/linux-coresched coresched-v3-
> > v5.1.5-
> > test
> >
> > And I have pushed Tim's branch to:
> > https://github.com/aaronlu/linux coresched-v3-v5.1.5-test-tim
> >
> > Mine:
> > https://github.com/aaronlu/linux coresched-v3-v5.1.5-test-
> > core_vruntime
> >
> Hello,
>
> As anticipated, I've been trying to follow the development of this
> feature and, in the meantime, I have done some benchmarks.
>
> I actually have a lot of data (and am planning for more), so I am
> sending a few emails, each one with a subset of the numbers in it,
> instead than just one which would be beyond giant! :-)
>
KERNBENCH
=========

http://xenbits.xen.org/people/dariof/benchmarks/results/linux/core-sched/mmtests/boxes/wayrath/coresched-email-3_kernbench.txt

v v BM BM BM BM BM BM
BM-HT BM-noHT HT noHT csc-HT csc_stallfix-HT csc_tim-HT csc_vruntime-HT
Amean elsp-2 200.07 ( 0.00%) 196.88 * 1.60%* 199.93 ( 0.07%) 196.89 * 1.59%* 200.86 * -0.39%* 200.83 * -0.38%* 199.47 * 0.30%* 200.45 * -0.19%*
Amean elsp-4 115.56 ( 0.00%) 108.64 * 5.99%* 115.12 ( 0.39%) 108.72 * 5.92%* 118.17 * -2.25%* 116.92 * -1.18%* 115.92 ( -0.31%) 115.86 ( -0.25%)
Amean elsp-8 84.72 ( 0.00%) 110.77 * -30.75%* 84.19 ( 0.62%) 111.03 * -31.06%* 84.78 ( -0.07%) 84.63 ( 0.11%) 89.09 * -5.16%* 90.21 * -6.48%*
Amean elsp-16 85.06 ( 0.00%) 113.63 * -33.59%* 85.33 ( -0.32%) 113.83 * -33.81%* 85.95 * -1.04%* 85.73 * -0.78%* 90.20 * -6.04%* 90.46 * -6.35%*
Stddev elsp-2 0.11 ( 0.00%) 0.05 ( 59.33%) 0.43 (-278.63%) 0.05 ( 60.30%) 0.20 ( -75.43%) 0.15 ( -28.90%) 0.16 ( -40.87%) 0.08 ( 26.69%)
Stddev elsp-4 0.54 ( 0.00%) 0.37 ( 30.80%) 0.02 ( 96.11%) 0.09 ( 83.05%) 1.10 (-105.52%) 0.24 ( 55.54%) 0.10 ( 81.29%) 0.26 ( 50.67%)
Stddev elsp-8 0.82 ( 0.00%) 0.25 ( 69.66%) 0.28 ( 65.58%) 0.27 ( 66.75%) 0.30 ( 63.64%) 0.07 ( 92.05%) 0.09 ( 88.92%) 0.19 ( 77.18%)
Stddev elsp-16 0.07 ( 0.00%) 0.32 (-375.21%) 0.41 (-502.93%) 0.19 (-176.54%) 0.22 (-219.28%) 0.21 (-208.51%) 0.31 (-358.10%) 0.09 ( -32.49%)
v v VM VM VM VM VM VM
VM-HT VM-noHT HT noHT csc-HT csc_stallfix-HT csc_tim-HT csc_vruntime-HT
Amean elsp-2 229.61 ( 0.00%) 202.26 * 11.91%* 205.30 * 10.59%* 201.14 * 12.40%* 207.43 * 9.66%* 207.30 * 9.72%* 205.92 * 10.32%* 206.69 * 9.98%*
Amean elsp-4 128.32 ( 0.00%) 124.33 * 3.11%* 116.84 * 8.95%* 124.66 * 2.85%* 121.86 * 5.04%* 140.87 * -9.78%* 118.28 * 7.83%* 122.58 * 4.48%*
Amean elsp-8 87.33 ( 0.00%) 118.45 * -35.63%* 85.52 ( 2.07%) 118.61 * -35.81%* 96.92 * -10.98%* 110.31 * -26.31%* 88.69 ( -1.55%) 88.49 ( -1.32%)
Amean elsp-16 87.00 ( 0.00%) 116.17 * -33.52%* 86.41 ( 0.68%) 116.43 * -33.82%* 103.24 * -18.67%* 90.77 * -4.33%* 88.89 * -2.17%* 89.35 * -2.70%*
Stddev elsp-2 0.24 ( 0.00%) 1.90 (-702.44%) 0.39 ( -63.41%) 0.45 ( -91.41%) 1.78 (-650.49%) 1.31 (-454.67%) 0.22 ( 8.97%) 1.09 (-362.56%)
Stddev elsp-4 0.10 ( 0.00%) 0.56 (-484.51%) 0.16 ( -63.75%) 0.60 (-524.50%) 0.47 (-392.73%) 2.53 (-2556.69%) 0.37 (-288.05%) 0.19 ( -96.77%)
Stddev elsp-8 1.48 ( 0.00%) 0.28 ( 81.02%) 0.08 ( 94.57%) 0.19 ( 87.46%) 1.25 ( 15.23%) 2.84 ( -92.18%) 1.23 ( 16.49%) 0.58 ( 60.60%)
Stddev elsp-16 0.62 ( 0.00%) 0.33 ( 46.43%) 0.07 ( 89.43%) 0.50 ( 20.24%) 0.54 ( 12.75%) 0.93 ( -49.52%) 0.44 ( 29.54%) 0.29 ( 53.95%)
v v VM VM VM VM VM VM
VM-v4-HT VM-v4-noHT v4-HT v4-noHT v4-csc-HT v4-csc_stallfix-HT v4-csc_tim-HT v4-csc_vruntime-HT
Amean elsp-2 227.42 ( 0.00%) 202.75 * 10.85%* 201.44 * 11.42%* 201.05 * 11.60%* 207.88 * 8.59%* 210.35 * 7.51%* 204.07 * 10.27%* 204.16 * 10.23%*
Amean elsp-4 124.46 ( 0.00%) 128.74 * -3.44%* 111.03 * 10.79%* 110.48 * 11.23%* 116.46 * 6.43%* 132.08 * -6.13%* 112.10 * 9.93%* 112.44 * 9.66%*
Amean elsp-8 127.12 ( 0.00%) 114.00 * 10.32%* 113.37 * 10.82%* 112.50 * 11.50%* 118.62 * 6.69%* 135.04 * -6.23%* 114.19 * 10.18%* 114.36 * 10.04%*
Stddev elsp-2 0.16 ( 0.00%) 0.05 ( 68.44%) 0.23 ( -45.93%) 0.32 (-101.53%) 0.89 (-471.21%) 0.86 (-452.61%) 0.13 ( 18.49%) 0.08 ( 51.98%)
Stddev elsp-4 0.09 ( 0.00%) 0.90 (-958.30%) 0.17 ( -95.69%) 0.40 (-364.45%) 1.33 (-1462.83%) 0.61 (-621.84%) 0.20 (-134.86%) 0.06 ( 28.48%)
Stddev elsp-8 0.14 ( 0.00%) 0.10 ( 29.71%) 0.40 (-181.91%) 0.05 ( 67.30%) 1.34 (-857.21%) 0.43 (-206.99%) 0.09 ( 39.30%) 0.12 ( 12.79%)
v v VMx2 VMx2 VMx2 VMx2 VMx2 VMx2
VMx2-HT VMx2-noHT HT noHT csc-HT csc_stallfix-HT csc_tim-HT csc_vruntime-HT
Amean elsp-2 206.25 ( 0.00%) 338.78 * -64.26%* 296.38 * -43.70%* 335.66 * -62.75%* 327.38 * -58.73%* 443.46 *-115.01%* 317.41 * -53.90%* 295.89 * -43.47%*
Amean elsp-4 117.69 ( 0.00%) 267.38 *-127.19%* 174.91 * -48.62%* 263.22 *-123.66%* 201.69 * -71.37%* 317.24 *-169.56%* 193.92 * -64.77%* 193.04 * -64.02%*
Amean elsp-8 85.93 ( 0.00%) 225.41 *-162.31%* 146.21 * -70.14%* 221.96 *-158.29%* 188.12 *-118.91%* 203.79 *-137.15%* 154.66 * -79.98%* 162.26 * -88.82%*
Amean elsp-16 86.82 ( 0.00%) 221.32 *-154.92%* 141.53 * -63.02%* 217.27 *-150.27%* 180.06 *-107.41%* 190.36 *-119.27%* 143.76 * -65.59%* 156.84 * -80.65%*
Stddev elsp-2 1.01 ( 0.00%) 1.11 ( -10.37%) 2.24 (-121.88%) 1.36 ( -35.11%) 22.86 (-2164.27%) 24.78 (-2353.59%) 2.84 (-180.79%) 0.91 ( 10.16%)
Stddev elsp-4 0.18 ( 0.00%) 4.55 (-2398.38%) 0.62 (-239.73%) 3.15 (-1630.91%) 3.00 (-1551.21%) 10.85 (-5864.08%) 5.87 (-3124.85%) 3.79 (-1984.93%)
Stddev elsp-8 0.42 ( 0.00%) 1.13 (-167.05%) 0.44 ( -4.45%) 2.91 (-590.61%) 4.16 (-885.58%) 6.85 (-1524.29%) 0.78 ( -85.34%) 1.48 (-252.23%)
Stddev elsp-16 0.41 ( 0.00%) 2.40 (-478.92%) 2.21 (-432.16%) 3.27 (-688.15%) 9.50 (-2189.54%) 3.39 (-717.66%) 2.00 (-383.08%) 0.48 ( -15.60%)


So, if only building kernels were the only thing that people do with
computers, we could say we're done, and go have beers! :-)

In fact, here, core scheduling is doing good on baremetal. E.g., look
at what happens, in the BM- group of measurements, when number of jobs
is higher than 8, which is how many CPUs we have.

It's actually doing quite fine in Virt too. Among the various variants
(sorry), plain v3, even with stallfix on, is the one performing worse.
Tim's patches seems to me to be the better looking set of numbers for
this workload.

Furthermore, there's basically no overhead --actually, there are
speedups-- until there is not virt-overcommitment. In fact, in the VMx2
group of measurements, even just applying the core scheduling v3
patches makes the VM

--
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)

Attachment: signature.asc
Description: This is a digitally signed message part