Re: [PATCH v3 1/1] iommu/arm-smmu-v3: Simplify useless instructions in arm_smmu_cmdq_build_cmd()

[John Garry]

> > Did you notice any performance change with this change?
>
> Hi John:
>    Thanks for the tip. I wrote a test case today, and I found that the
> performance did not go up but down.

I very quickly tested on a DMA mapping benchmark very similar to the 
kernel DMA benchmark module - I got mixed results. For fewer CPUs (<8), 
a small improvement, like 0.7%. For more CPUs, a dis-improvement - 
that's surprising, I did expect just no change as any improvement would 
get dwarfed from the slower unmap rates for more CPUs. I can check this
more tomorrow.

> It's so weird. So I decided not to
> change it, because it's also poorly readable. So I plan to make only
> the following modifications:
> @@ -237,7 +237,7 @@ static int queue_remove_raw(struct arm_smmu_queue *q, u64 *ent)
>   static int arm_smmu_cmdq_build_cmd(u64 *cmd, struct arm_smmu_cmdq_ent *ent)
>   {
>          memset(cmd, 0, 1 << CMDQ_ENT_SZ_SHIFT);
> -       cmd[0] |= FIELD_PREP(CMDQ_0_OP, ent->opcode);
> +       cmd[0] = FIELD_PREP(CMDQ_0_OP, ent->opcode);
>
>          switch (ent->opcode) {
>          case CMDQ_OP_TLBI_EL2_ALL:
>
> This prevents the compiler from generating the following two inefficient
> instructions:
>       394:       f9400002        ldr     x2, [x0]	//x2 = cmd[0]
>       398:       aa020062        orr     x2, x3, x2	//x3 = FIELD_PREP(CMDQ_0_OP, ent->opcode)
>
> Maybe it's not worth changing because I've only seen a 0.x nanosecond reduction
> in performance. But one thing is, it only comes with benefits, no side effects.

I just think that with the original code that cmd[] is on the stack and 
cached, so if we have write-back attribute (which I think we do) then 
there would not necessarily a write to external memory per write to cmd[].

So, apart from this approach, I think that if we can just reduce the 
instructions through other efficiencies in the function then that would 
be good.

Thanks,
John