RE: [RFC PATCH v4 00/19] Core scheduling v4

From: Gruza, Agata
Date: Wed Jan 22 2020 - 13:04:28 EST

-----Original Message-----
From: linux-kernel-owner@xxxxxxxxxxxxxxx <linux-kernel-owner@xxxxxxxxxxxxxxx> On Behalf Of Vineeth Remanan Pillai
Sent: Friday, January 17, 2020 8:01 AM
To: Aubrey Li <aubrey.intel@xxxxxxxxx>
Cc: Tim Chen <tim.c.chen@xxxxxxxxxxxxxxx>; Aubrey Li <aubrey.li@xxxxxxxxxxxxxxx>; Nishanth Aravamudan <naravamudan@xxxxxxxxxxxxxxxx>; Julien Desfossez <jdesfossez@xxxxxxxxxxxxxxxx>; Peter Zijlstra <peterz@xxxxxxxxxxxxx>; Ingo Molnar <mingo@xxxxxxxxxx>; Thomas Gleixner <tglx@xxxxxxxxxxxxx>; Paul Turner <pjt@xxxxxxxxxx>; Linus Torvalds <torvalds@xxxxxxxxxxxxxxxxxxxx>; Linux List Kernel Mailing <linux-kernel@xxxxxxxxxxxxxxx>; Dario Faggioli <dfaggioli@xxxxxxxx>; FrÃdÃric Weisbecker <fweisbec@xxxxxxxxx>; Kees Cook <keescook@xxxxxxxxxxxx>; Greg Kerr <kerrnel@xxxxxxxxxx>; Phil Auld <pauld@xxxxxxxxxx>; Aaron Lu <aaron.lwe@xxxxxxxxx>; Valentin Schneider <valentin.schneider@xxxxxxx>; Mel Gorman <mgorman@xxxxxxxxxxxxxxxxxxx>; Pawan Gupta <pawan.kumar.gupta@xxxxxxxxxxxxxxx>; Paolo Bonzini <pbonzini@xxxxxxxxxx>
Subject: Re: [RFC PATCH v4 00/19] Core scheduling v4

> >
> > Aubrey's attached patch should replace his previous patch
> > sched/fair: don't migrate task if cookie not match
> >
> > I've also added a fix below for Aubrey's patch
> > sched/fair: find cookie matched idlest CPU.
> >
> > Aubrey, can you merge this fix into that patch when you update your
> > patches?
>
> Thanks Tim, I'll include your fix when I update my patches.
>
We have included both these changes in our branch rebased to 5.4.y:
https://github.com/digitalocean/linux-coresched/tree/coresched/v4-v5.4.y

This is our testing branch.



----------------------------------------------------------------------
ABOUT:
----------------------------------------------------------------------
Hello,

Core scheduling is required to protect against leakage of sensitive
data allocated on a sibling thread. Our goal is to measure performance
impact of core scheduling across different workloads and show how it
evolved over time. Below you will find data for core scheduling. Data
presented here are based on previous version of core-sched (v3) plus
additional kernel patches added by tim.c.chen@xxxxxxxxx and load
balancer made by aubrey.li@xxxxxxxxx that are now in (v4) core
scheduler kernel
https://github.com/digitalocean/linux-coresched/commits/coresched/v4-v5.4.y
Detailed specifics about each system set up in attached PDF.

----------------------------------------------------------------------
BENCHMARKS:
----------------------------------------------------------------------
- SPECreate2017int : industry standard for measure compute intensive
CPU performance.
- CASSANDRA : NoSQL distributed DB and benchmark. Provides
high availability with no single point of failure.
- HammerDB : database benchmarking application
- SPECvirt : industry standard for measure server performance
in virtual environment.

----------------------------------------------------------------------
PERFORMANCE IMPACT:
----------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------------------------------------
Kernel & Benchmark |# of cgroups|Overcommit | Performance Change %| Performance Change % |
| | (HT ON, core-sched) | (HT OFF, No core-sched) |
---------------------------------------------------------------------------------------------------------------------------------------------------
| 2cgroups | 0.5x | -0.48 | 0.26 |
SPECint_rate_base 2017 | 1 per VM | 1x | -0.54 | 0.66 |
| | 2x | -0.18 | -0.07 |
---------------------------------------------------------------------------------------------------------------------------------------------------
| 2cgroups | 0.5x | -2.56 | -44.61 |
CASSANDRA | 1 per VM | 1x | -4.71 | -47.47 |
| | 2x | -7.63 | -28.47 |
---------------------------------------------------------------------------------------------------------------------------------------------------
| 2cgroups | 0.5x | -19.99 | -23.52 |
HammerDB | 1 per VM | 1x | -14.90 | -24.63 |
| | 2x | -13.31 | -25.71 |
---------------------------------------------------------------------------------------------------------------------------------------------------
| attachment |attachment | -18 | -31 |
SPECvirt | | | | |
---------------------------------------------------------------------------------------------------------------------------------------------------
| 2cgroups | 0.5x | -5.18 | -44.61 |
CASSANDRA | 1 per VM | 1x | -7.90 | -47.47 |
| | 2x | -6.52 | -28.47 |
---------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------
TAKE AWAYS:
----------------------------------------------------------------------
1. Core scheduling performs better than turning off HT in all
overcommitting scenarios. However we observed in certain cases
up to 20% performance drop.
2. Impact of core scheduling depends on the workload and thread
scheduling intensity.
3. Core scheduling requires cgroups. Single cgroup per VM. Each
VM is running on itâs own independent cgroup.

Many thanks,
--Agata

Attachment: LKML_core_scheduling.pdf
Description: LKML_core_scheduling.pdf