Re: arch/x86/entry/entry: RFC on recent kernels building error with llvm 11.0.2 internal assembler

[Nathan Chancellor]

Hi Mauro,

[+ llvm@xxxxxxxxxxxxxxx and Nick]

On Fri, Apr 29, 2022 at 07:40:07PM +0200, Mauro Rossi wrote:
> Hi Andy,
> 
> I am an hobbyist contributing to android-x86 FOSS project lead by
> Chih-Huwei Huang (in Cc: for information/alignement)
> 
> I am performing periodic tests to build kernel for Android 11 based iso image
> which relies on aosp shipped prebuild clang toolchain (clang version 11.0.2)
> 
> When building linux 5.18rc4 and also with linux 5.17 x86_64 64bit kernel targets
> there is a building error in arch/x86/entry
> 
>   AS      arch/x86/entry/entry_64.o
> <instantiation>:2:2: error: unknown use of instruction mnemonic
> without a size suffix
>  lsl %rax, %rax
>  ^
> <instantiation>:1:1: note: while in macro instantiation
> LOAD_CPU_AND_NODE_SEG_LIMIT %rax
> ^
> <instantiation>:2:2: note: while in macro instantiation
>  GET_PERCPU_BASE %rax
>  ^
> /home/utente/r-x86_kernel/kernel/arch/x86/entry/entry_64.S:890:2:
> note: while in macro instantiation
>  SAVE_AND_SET_GSBASE scratch_reg=%rax save_reg=%rbx
>  ^
> make[3]: *** [/home/utente/r-x86_kernel/kernel/scripts/Makefile.build:389:
> arch/x86/entry/entry_64.o] Error 1
> make[2]: *** [/home/utente/r-x86_kernel/kernel/scripts/Makefile.build:550:
> arch/x86/entry] Error 2
> make[1]: *** [/home/utente/r-x86_kernel/kernel/Makefile:1887: arch/x86] Error 2
> make[1]: *** Waiting for unfinished jobs....
> 
> As other interesting info, the building error does not happen when
> building x86 32bit kernel target and i can build 86_64 64bit kernel
> target only by setting the LLVM_IAS=0 parameter to disable the
> internal llvm assembler

This error was fixed in LLVM 11.0.0 final, which was released after
Android's LLVM 11.0.2:

https://github.com/ClangBuiltLinux/linux/issues/1079

> I wanted to ask you if you could help us, if there could be a way to
> improve arch/x86/entry/entry_64.S code to be able to complete the
> build without disabling the llvm internal assembler.
> 
> I don't know if this building error may be caused by the clang version
> 11.0.2, but at some point the aosp and android version may hit this
> same issue,
> so I wanted to highlight this issue to you to have a competent person feedback,
> as I am more a "trial and error" guy than a kernel expert

I am open to other opinions but I am not inclined to suggest working
around this in the kernel for two reasons:

1. This issue was resolved in the toolchain almost two years ago, so it
   is not a recent failure.

2. Android's LLVM 11.0.2 is technically older than the minimum version
   that the kernel supports (11.0.0), which I would argue means it is
   unsupported. 11.0.0 final was released on October 12th, 2020 and
   Android's LLVM 11.0.2 was committed on June 11th, 2020, so you are
   potentially missing four months worth of fixes:

   https://android.googlesource.com/platform/prebuilts/clang/host/linux-x86/+/431c74471920f3f9b0517692fb69515c023bde41

   Unfortunately, due to the way that LLVM versions work, it is not so
   easy to check for this but perhaps we should consider trying to
   handle this, so that others don't continue to trip over old bugs.

Moving to LLVM_IAS=0 is the solution that we went with for clang-10 when
it was supported after the switch to the integrated assembler by
default, which I do not think is an unreasonable solution for this
issue.

Alternatively, you could apply the hack that Nick inserted into Android
for this issue if upgrading your toolchain or turning off the integrated
assembler is not possible:

https://android.googlesource.com/kernel/common/+/e58f084735b8abf744d61083b92172ee23d45aae

I really do not mean to sound dismissive or rude, I apologize if it
comes off that way, but we have worked quite hard to avoid inserting
unnecessary workarounds, as they are ultimately technical debt that can
be hard to manage over the long term.

Cheers,
Nathan