Re: [PATCH] Add busy loop polling for idle SMT

[Peng Wang]

On 17/11/2021 18:58, , Peter Zijlstra wrote:
> On Tue, Nov 16, 2021 at 07:51:35PM +0800, Peng Wang wrote:
> > Now we have cpu_idle_force_poll which uses cpu_relax() waiting for
> > an arriving IPI, while sometimes busy loop on idle cpu is also
> > useful to provide consistent pipeline interference for hardware SMT.
> >
> > When hardware SMT is enabled, the switching between idle and
> > busy state of one cpu will cause performance fluctuation of
> > other sibling cpus on the same core.
> >
> > In pay-for-execution-time scenario, cloud service providers prefer
> > stable performance data to set stabel price for same workload.
> > Different execution time of the same workload caused by different
> > idle or busy state of sibling SMT cpus will make different bills, which
> > is confused for customers.
> >
> > Since there is no dynamic CPU time scaling based on SMT pipeline interference,
> > to coordinate sibling SMT noise no matter whether they are idle or not,
> > busy loop in idle state can provide approximately consistent pipeline interference.
> >
> > For example, a workload computing tangent and cotangent will finish in 9071ms when
> > sibling SMT cpus are idle, and 13299ms when sibling SMT cpus are computiing other workload.
> > This generate 32% performance fluctuation.
> >
> > SMT idle polling makes things slower, but we can set bigger cpu quota to make up
> > a deficiency. This also increase power consumption by 2.2%, which is acceptable.
> >
> > There may be some other possible solutions, while each has its own problem:
> > a) disbale hardware SMT, which means half of SMT is unused and more hardware cost.
> > b) busy loop in a userspace thread, but the cpu usage is confusing.
> >
> > We propose this patch to discuss the performance fluctuation problem related to SMT
> > pipeline interference, and any comments are welcome.
>
> I think you missed April Fools' Day by a wide margin.
>
> Lowering performance and increasing power usage is a direct

Siblings' noise depends on workloads, when persuing performance 
stability, we have to consider what performance data to keep:

a) the worst with all-time noise
b) the best with monopolizing a whole core by disabling SMT or using 
core scheduling, while wasting some logic CPUs
c) A number between the worst and the best which is hard to decide

That's where lowering performance comes from.

> contradiction to sanity. It also doesn't really work as advertised,
> if the siblings are competing for AVX resources the performance is a
> *lot* lower than when an AVX task is competing against a spinner like
> this.

Yes, idle SMT busy loop polling can only provide approximately pipeline 
interference for normal instructions.

When it comes to AVX works, we notice an idea modifing CPU time 
accounting[1], do you think the combination can lead to a feasible
solution, or any other better ideas?

[1] https://www.usenix.org/conference/atc21/presentation/gottschlag