Re: Bug#1033301: linux: arm64 kernel size increased from 31 to 39 MB, causing u-boot-rpi to fail

[Salvatore Bonaccorso]

Hi Aurelien,

Thanks for tracking this down. I would like to loop in Masahiro and
upstream to see if something can/should be done on upstream side.

Masahiro, in short, upstream change 994b7ac1697b ("arm64: remove
special treatment for the link order of head.o") (which got backported
as well to 6.1.14) caused the vmlinuz size to icrease significantly,
causing boot failures on Raspberry Pi 3 Model B Plus with u-boot
parameters previously working. Full quoting the Debian report below

On Tue, Mar 21, 2023 at 11:11:13PM +0100, Aurelien Jarno wrote:
> Source: linux
> Version: 6.1.15-1
> Severity: important
> Tags: upstream
> X-Debbugs-Cc: vagrant@xxxxxxxxxx
> Control: affects -1 + u-boot-rpi
> 
> Hi,
> 
> Following the upgrade of the kernel from 6.1.12-1 to 6.1.15-1 on a
> Raspberry Pi 3 Model B Plus, u-boot (from the u-boot-rpi package) failed
> to boot with:
> 
> | 40175552 bytes read in 1695 ms (23 MiB/s)
> | 43794863 bytes read in 1817 ms (23 MiB/s)
> | Moving Image from 0x80000 to 0x200000, end=2990000
> | ERROR: RD image overlaps OS image (OS=0x200000..0x2990000)
> 
> I tracked the issue to a significant increase of the kernel size between
> version 6.1.12-1 and 6.15-1:
> 
> | 31492   /boot/vmlinuz-6.1.0-5-arm64
> | 39236   /boot/vmlinuz-6.1.0-6-arm64
> 
> This is more than the 36MB that is allowed by u-boot with the default
> load addresses. A workaround is to shift the load addresses at the
> u-boot level as in the attached patch.
> 
> I have tracked issue on the upstream kernel side to the following commit
> on the stable tree:
> 
> | commit 3e3e4d234d46e48480a7c7c35399fa811182e8ef
> | Author: Masahiro Yamada <masahiroy@xxxxxxxxxx>
> | Date:   Thu Oct 13 08:35:00 2022 +0900
> | 
> |     arm64: remove special treatment for the link order of head.o
> |     
> |     commit 994b7ac1697b4581b7726d2ac64321e3c840229b upstream.
> |     
> |     In the previous discussion (see the Link tag), Ard pointed out that
> |     arm/arm64/kernel/head.o does not need any special treatment - the only
> |     piece that must appear right at the start of the binary image is the
> |     image header which is emitted into .head.text.
> |     
> |     The linker script does the right thing to do. The build system does
> |     not need to manipulate the link order of head.o.
> |     
> |     Link: https://lore.kernel.org/lkml/CAMj1kXH77Ja8bSsq2Qj8Ck9iSZKw=1F8Uy-uAWGVDm4-CG=EuA@xxxxxxxxxxxxxx/
> |     Suggested-by: Ard Biesheuvel <ardb@xxxxxxxxxx>
> |     Signed-off-by: Masahiro Yamada <masahiroy@xxxxxxxxxx>
> |     Reviewed-by: Nicolas Schier <nicolas@xxxxxxxxx>
> |     Link: https://lore.kernel.org/r/20221012233500.156764-1-masahiroy@xxxxxxxxxx
> |     Signed-off-by: Will Deacon <will@xxxxxxxxxx>
> |     Signed-off-by: Tom Saeger <tom.saeger@xxxxxxxxxx>
> |     Signed-off-by: Greg Kroah-Hartman <gregkh@xxxxxxxxxxxxxxxxxxx>
> 
> The problem is still reproducible on Linus' master.
> 
> I am reporting the bug to the linux package as I believed there is no
> real reason for such an increase in the kernel size. In case I missed
> something and this is actually wanted, the bug can be reassigned to the
> u-boot package.
> 
> Regards
> Aurelien

> --- u-boot-2023.01+dfsg.orig/include/configs/rpi.h
> +++ u-boot-2023.01+dfsg/include/configs/rpi.h
> @@ -95,32 +95,32 @@
>   *   text_offset bytes (specified in the header of the Image) into a 2MB
>   *   boundary. The 'booti' command relocates the image if necessary. Linux uses
>   *   a default text_offset of 0x80000.  In summary, loading at 0x80000
> - *   satisfies all these constraints and reserving memory up to 0x02400000
> - *   permits fairly large (roughly 36M) kernels.
> + *   satisfies all these constraints and reserving memory up to 0x02a00000
> + *   permits fairly large (roughly 42M) kernels.
>   *
>   * scriptaddr and pxefile_addr_r can be pretty much anywhere that doesn't
>   * conflict with something else. Reserving 1M for each of them at
> - * 0x02400000-0x02500000 and 0x02500000-0x02600000 should be plenty.
> + * 0x02a00000-0x02b00000 and 0x02c00000-0x02d00000 should be plenty.
>   *
>   * On ARM, both the DTB and any possible initrd must be loaded such that they
>   * fit inside the lowmem mapping in Linux. In practice, this usually means not
>   * more than ~700M away from the start of the kernel image but this number can
>   * be larger OR smaller depending on e.g. the 'vmalloc=xxxM' command line
>   * parameter given to the kernel. So reserving memory from low to high
> - * satisfies this constraint again. Reserving 1M at 0x02600000-0x02700000 for
> - * the DTB leaves rest of the free RAM to the initrd starting at 0x02700000.
> + * satisfies this constraint again. Reserving 1M at 0x02c00000-0x02d00000 for
> + * the DTB leaves rest of the free RAM to the initrd starting at 0x02d00000.
>   * Even with the smallest possible CPU-GPU memory split of the CPU getting
> - * only 64M, the remaining 25M starting at 0x02700000 should allow quite
> + * only 64M, the remaining 19M starting at 0x02d00000 should allow quite
>   * large initrds before they start colliding with U-Boot.
>   */
>  #define ENV_MEM_LAYOUT_SETTINGS \
>  	"fdt_high=" FDT_HIGH "\0" \
>  	"initrd_high=" INITRD_HIGH "\0" \
>  	"kernel_addr_r=0x00080000\0" \
> -	"scriptaddr=0x02400000\0" \
> -	"pxefile_addr_r=0x02500000\0" \
> -	"fdt_addr_r=0x02600000\0" \
> -	"ramdisk_addr_r=0x02700000\0"
> +	"scriptaddr=0x02a00000\0" \
> +	"pxefile_addr_r=0x02b00000\0" \
> +	"fdt_addr_r=0x02c00000\0" \
> +	"ramdisk_addr_r=0x02d00000\0"
>  
>  #if CONFIG_IS_ENABLED(CMD_MMC)
>  	#define BOOT_TARGET_MMC(func) \

Any ideas?

Regards,
Salvatore