Re: [patch 1/3] net: dst: Prevent false sharing vs. dst_entry::__refcnt

[Eric Dumazet]

On Tue, Feb 28, 2023 at 3:33 PM Thomas Gleixner <tglx@xxxxxxxxxxxxx> wrote:
>
> From: Wangyang Guo <wangyang.guo@xxxxxxxxx>
>
> dst_entry::__refcnt is highly contended in scenarios where many connections
> happen from and to the same IP. The reference count is an atomic_t, so the
> reference count operations have to take the cache-line exclusive.
>
> Aside of the unavoidable reference count contention there is another
> significant problem which is caused by that: False sharing.
>
> perf top identified two affected read accesses. dst_entry::lwtstate and
> rtable::rt_genid.
>
> dst_entry:__refcnt is located at offset 64 of dst_entry, which puts it into
> a seperate cacheline vs. the read mostly members located at the beginning
> of the struct.

This will probably increase struct rt6_info past the 4 cache line size, right ?

It would be nice to allow sharing the 'hot' cache line with seldom used fields.

Instead of mere pads, add some unions, and let rt6i_uncached/rt6i_uncached_list
use them.