Re: [PATCH 2/2] x86/kstack_erase: Use REP STOSQ for large stack fills

From: Nick Desaulniers

Date: Thu Nov 19 2026 - 17:28:25 EST


On Thu, Nov 19, 2026 at 1:32 PM Usama Arif <usama.arif@xxxxxxxxx> wrote:
>
> With KSTACK_ERASE, every return to user space poisons the part of the
> task stack that was used since the previous return. The generic
> __stackleak_poison() does that one word per loop iteration, so a syscall
> that went 1-2 KiB deep pays for 128-256 separate stores. On Meta hosts
> running many small VMs, stackleak_erase() accounts for about 1.5% of
> non-idle host kernel cycles.
>
> Override __stackleak_poison() on x86-64, as s390 already does, through
> the hook added by commit 491a78663e03 ("stackleak: allow to specify arch
> specific stackleak poison function"). Fill spans larger than 512 bytes
> with REP STOSQ and keep the word loop for shorter ones, where the fixed
> startup cost of REP STOSQ does not pay off. REP STOSQ starts to win at
> about 300-400 bytes on Zen 5c and Cooper Lake, so on both no span gets
> slower.
>
> The fill is inline asm, so the noinstr function still makes no calls.
> memset64() would be the same instruction, but it is only static inline,
> and with KMSAN it is an out-of-line function.
>
> In a Zen 5c KVM guest, stackleak_erase() got about 35% cheaper per return
> to user space (71 ns to 46 ns, scan included) over a futex, epoll, procfs
> and Unix socket syscall mix. Same-vCPU futex wake/wait and Unix socket
> sendmsg()/recvmsg() ping-pong round trips got 5-8% faster.
>
> Signed-off-by: Usama Arif <usama.arif@xxxxxxxxx>
> ---
> arch/x86/include/asm/processor.h | 25 +++++++++++++++++++++++++
> 1 file changed, 25 insertions(+)
>
> diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
> index 77488ff5eb8c7c..049231d484ad0e 100644
> --- a/arch/x86/include/asm/processor.h
> +++ b/arch/x86/include/asm/processor.h
> @@ -580,6 +580,31 @@ static __always_inline bool on_thread_stack(void)
> current_stack_pointer) < THREAD_SIZE;
> }
>
> +#ifdef CONFIG_X86_64
> +#define __stackleak_poison __stackleak_poison
> +static __always_inline void __stackleak_poison(unsigned long erase_low,
> + unsigned long erase_high,
> + unsigned long poison)
> +{
> + unsigned long count;
> +
> + /* Short spans do not amortize the startup cost of REP STOSQ. */
> + if (erase_high <= erase_low + 512) {
> + while (erase_low < erase_high) {
> + *(unsigned long *)erase_low = poison;
> + erase_low += sizeof(unsigned long);
> + }
> + return;
> + }
> +
> + count = (erase_high - erase_low) / sizeof(unsigned long);
> + asm volatile("rep stosq"
> + : "+D" (erase_low), "+c" (count)
> + : "a" (poison)
> + : "memory");

Surely this isn't the only place using `rep stosq` in the tree, right?
It's not, see memset64 in arch/x86/include/asm/string_64.h.

> +}
> +#endif
> +
> #ifdef CONFIG_PARAVIRT_XXL
> #include <asm/paravirt.h>
> #else
> --
> 2.53.1-Meta
>


--
Thanks,
~Nick Desaulniers