Re: [PATCH net] selftests: bonding: fix failover race in bond_options.sh prio test
From: Pin-yen Lin
Date: Tue Nov 17 2026 - 21:33:19 EST
Hi Hangbin,
On Mon, Nov 16, 2026 at 11:26 PM Hangbin Liu <hangbin.liu@xxxxxxxxx> wrote:
>
> Hi Pin-yen,
> On Mon, Nov 16, 2026 at 05:30:28PM -0800, Pin-yen Lin wrote:
> > Hi Hangbin,
> >
> > On Thu, Nov 12, 2026 at 10:00 PM Hangbin Liu <hangbin.liu@xxxxxxxxx> wrote:
> > >
> > > On Fri, Nov 13, 2026 at 01:23:40PM +0800, Hangbin Liu wrote:
> > > > Hi Pin-yen,
> > > > On Wed, Nov 11, 2026 at 05:46:18PM -0800, Pin-yen Lin wrote:
> > > > > The prio test intermittently fails in the arp_ip_target and
> > > > > ns_ip6_target variants:
> > > > >
> > > > > TEST: prio (active-backup ns_ip6_target primary_reselect 1) [FAIL]
> > > > > Current active slave is eth0 but not eth2
> > > > >
> > > > > The ARP/NS monitor marks a slave down when it has received nothing for
> > > > > too long. With arp_interval 100 and the default arp_missed_max, that is
> > > > > about 250ms for the active slave (eth1) and 350ms for a backup slave
> > > > > (eth0, eth2). Every 100ms, eth1 sends a probe: eth1 receives the reply,
> > > > > and eth0/eth2 receive a copy forwarded by the gateway bridge. Once eth1
> > > > > is down, its probes reach no one and no slave receives anything, so
> > > > > eth1 normally times out first and the new active slave is picked by
> > > > > prio:
> > > > >
> > > > > t=0 probe: eth1 gets the reply, eth0/eth2 get a copy
> > > > > t=10 test sets eth1 down
> > > > > t=300 monitor: eth1 idle 300ms -> down
> > > > > eth0/eth2 idle 300ms -> up, eth2 picked by prio
> > > > >
> > > > > But the test takes eth1 down right after bond_check_connection(), whose
> > > > > ping replies only arrive on eth1:
> > > > >
> > > > > t=0 probe: eth1 gets the reply, eth0/eth2 get a copy
> > > > > t=70 ping reply on eth1 only
> > > > > t=85 test sets eth1 down
> > > > > t=300 monitor: eth1 idle 230ms -> still up
> > > > > t=400 monitor: eth1 idle 330ms -> down
> > > > > eth2 idle 400ms -> down too
> > > > >
> > > > > eth2 can then no longer be picked. eth0 becomes the active slave if it
> > > > > happened to receive some other packet (e.g. an MLD report), otherwise
> > > > > whichever slave comes back first does. With primary_reselect 1 or 2,
> > > > > the bond keeps eth0 even after eth2 comes back.
> > > > >
> > > > > Before taking the active slave down, wait two monitor intervals
> > > > > (200ms), so that at least one more probe is sent after the last ping
> > > > > reply and all slaves last received that probe. The miimon variants do
> > > > > not need the wait, but the extra delay is negligible.
> > > >
> > > > I'm afraid the waiting time is too strict, especially with a debug kernel
> > > > since delays can happen.
> > > >
> > > > Bjoern is also fixing[1] this issue by pinging a non-existent neighbor to
> > > > keep traffic flowing. Would you like to test whether that works for you?
> > > >
> > > > [1] https://lore.kernel.org/netdev/20261104142018.552135-1-doebel@xxxxxxxxx/
> > >
> > > BTW, using ping may not able to triger the arp request in time. You can try
> > > arping instead.
> >
> > Thanks for the pointer, but I believe Bjoern's patch addresses a
> > different problem.
>
> I think they are the same issue, as you comment also said
>
> The ARP/NS monitor marks a slave down when it has received nothing for
> too long.
This is the existing behavior of the bonding driver, and the prio test
uses it to check which backup slave takes over. Slaves being marked
down is not the root cause of the test failure. Instead, the root
cause is the ping replies described below.
>
> >
> > Bjoern's patch keeps the traffic flowing so that idle slaves are kept
> > alive between the test steps. This patch fixes the case where the ping
> > replies from bond_check_connection() only arrive on the active slave
> > right before it is set down. The active slave's failure is then
> > detected one monitor interval late, when the backups have also timed
> > out.
>
> The reason is we set arp interval to 100ms, and slowwait also check every
> 100ms. If the backup slave down at the same check point, e.g.
>
> t=300 monitor: eth1 idle 230ms -> still up
> t=400 monitor: eth1 idle 330ms -> down
> eth2 idle 400ms -> down too
>
> The test failed.
I don't think the slowwait interval makes a difference here.
The ARP monitor decides which slave becomes active, and slowwait only
reads the result afterwards. The test expects the active slave and the
backups to time out in different monitor runs, so that a backup can
take over. In the example above, eth2 is already down when the bond
picks the new active slave, no matter when slowwait polls.
>
> >
> > The ARP requests triggered by Bjoern's ping are sent only about once
> > per second, which isn't frequent enough for this failure. I tested
> > Bjoern's patch and confirmed that it didn't fix the issue in our
> > testing environment. We also can't simply use arping instead because
> > iputils arping doesn't support sub-second intervals.
> >
> > If you're worried that the 200ms sleep is too short, would increasing
> > it to 0.5s or 1s work for you?
>
> It's not about the sleep time before setting active slave down. I'm afraid
> about the slowwait checking time. Even we sleep 1s, with arp interval 100,
> we still only have 100ms checking window. Before the window, the active slave
> is up, after the window, the backup slave also down. With a debug kernel, this
> time is a little tight.
>
> So, either we have some background arp traffic, or let's extend the arp
> interval to a larger one.
>
> What do you think?
I agree that a 100ms ARP interval could be too short in environments
like a debug kernel.
Background ARP traffic doesn't widen that window, though: it is sent
through the active slave, so once the active slave is set down,
nothing reaches the backups. It could only help with the ping reply
race if it were very frequent; Bjoern's patch triggers about one ARP
request per second and didn't fix the failures in our testing.
Increasing the ARP interval alone won't address the ping reply race,
but it should help in environments like a debug kernel. I'd prefer to
keep this patch focused on the ping reply race and send the ARP
interval change as a follow-up, but I can fold it into v2 if you
prefer.
>
> Thanks
> Hangbin
Regards,
Pin-yen