Re: [patch] ipv4: initialize arp_tbl rw lock

[Heiko Carstens]

> > Tried to figure out what is causing the delays I experienced when I replaced
> > module_init() in af_inet.c with fs_initcall(). After all it turned out that
> > synchronize_net() which is basicically nothing else than synchronize_rcu()
> > sometimes takes several seconds to complete?! No idea why that is...
> > 
> > callchain: inet_init() -> inet_register_protosw() -> synchronize_net()
> 
> The problem can't be rcu_init(), that gets done very early
> in init/main.c
> 
> Maybe it's some timer or something else specific to s390?
> 
> It could also be that there's perhaps nothing to context
> switch to, thus the RCU takes forever to "happen".

Yes, it's more or less s390 specific.

What happens is the following: synchronize_rcu() enqueues an RCU callback on
cpu 0. Later on cpu 0 handles a bunch of RCU batches, but without handling
this specific request (it's in rdp->curlist). Since this cpu has nothing else
to do it enters cpu_idle() (it's a nohz idle, therefore it might be quite a
long time in idle state).
While cpu 0 is in idle state cpu 2 calls cpu_quiet() which in turn will call
rcu_start_batch(). If cpu 0 would run now, it would notice rdp->curlist moved
to rdp->donelist and that there is something to do. Unfortunately it doesn't
get notified from rcu_start_batch(). That's why I ended up waiting several
seconds until finally some interrupt arrived at cpu 0 which made things go
on finally.

Avoiding this could be done if we look at rdp->curlist before going into
a nohz idle wait, or we could send an interprocessor interrupt to idle
cpus. Sending an interrupt while looking only on nohz_cpu_mask seems to
be a bit racy, since other cpus might have entered cpu idle after
nohz_cpu_mask has been read...

At least the initcall change for inet_init() can go in, since it just
revealed a problem that we have anyway.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/