Re: why does netfilter make upload very slow? (was: Re: e1000 -> 82540EM on linux 2.6.0-test[45] very slow in one direction)

[Harald Welte]

Hi Florian!

I'm Cc'ing all the mailinglists in order to keep them posted about the
question you've raised there.  All further discussion will move to
netfilter-devel, so for those interested: Please continue there.

On Wed, Oct 15, 2003 at 10:28:50AM +0200, Florian Zwoch wrote:
> >a) CONFIG_NETFILTER disabled.  you won't even have the netfilter hooks
> >   in the network stack (so certainly no netfilter-using modules loaded)
> no problem
> 
> >b) CONFIG_NETFILTER enabled, but _no_ modules (iptable_filter,
> >   ip_conntrack, ...) attached to the netfilter hook
> no problem
> 
> >c) CONFIG_NETFILTER enabled and iptable_filter.o (which pulls ip_tables.o)
> >   loaded, NO RULES in the table
> no problem
> 
> >d) CONFIG_NETFILTER enabled and iptable_filter.o (which pulls ip_tables.o)
> >   loaded, RULES in the table
> no problem (as long as i dont load any rules that require ip_conntrack)
> 
> >e) CONFIG_NETFILTER enabled and ip_conntrack.o loaded, iptable_filter
> >   loaded or not, rules or not
> *boink*

So It's clearly the connection tracking subsystem.  This is on one hand
good (because it means it's neither netfilter nor iptables).  

> whenever i try to load ip_conntrack the nic performance drops from 5mb/s 
> to 200k/s.

On the other hand, this is definitely way worse than you would expect.
Can you please tell me more information about:

- number of connections you have? (cat /proc/net/ip_conntrack | wc -l)
- number of buckets and ip_conntrack_max (printed at ip_conntrack
  loadtime
- your traffic pattern.  Are you spraying udp packets with random
  src/dst? What kind of connections (protocol, application) are you
  testing with?
- what about the hardware (cpu, memory, smp?)

Even the worst tests we've had so far (random UDP packets) 'only'
reduced the througput by about 50%.   Maybe we can do better than 50%
worst case behaviour, but you will always observe a visible impact as
soon as you start connection tracking for every single packet (which is
what 'insmod ip_conntrack' implies).

> still using 2.6.0-test6.

Have you observed this behaviour with other kernel versions?  Was there
a performance change between 2.4 and 2.6?  Or did you always observe
this grave performance loss?

> regards,
> Florian

-- 
- Harald Welte <laforge@xxxxxxxxxxxxx>             http://www.netfilter.org/
============================================================================
  "Fragmentation is like classful addressing -- an interesting early
   architectural error that shows how much experimentation was going
   on while IP was being designed."                    -- Paul Vixie

Attachment: pgp00001.pgp
Description: PGP signature