Re: Network performance - iperf
From: Steve Magnani
Date: Tue Mar 30 2010 - 08:12:04 EST
Looks like a lot of time spent in tcp_collapse(). I think there is assumption that the memcpys
involved in collapsing lead to better throughput than packet retransmit, which on our platforms
may not be the case. If so is there a way to tune when tcp_collapse gets invoked? I haven't
seen one.
Steve
-----Original Message-----
From: Michal Simek <michal.simek@xxxxxxxxxxxxx>
To: Eric Dumazet <eric.dumazet@xxxxxxxxx>
Cc: LKML <linux-kernel@xxxxxxxxxxxxxxx>, John Williams <john.williams@xxxxxxxxxxxxx>,
netdev@xxxxxxxxxxxxxxx, Grant Likely <grant.likely@xxxxxxxxxxxx>, John Linn
<John.Linn@xxxxxxxxxx>, "Steven J. Magnani" <steve@xxxxxxxxxxxxxxx>, Arnd Bergmann
<arnd@xxxxxxxx>, akpm@xxxxxxxxxxxxxxxxxxxx
Date: Tue, 30 Mar 2010 11:34:29 +0200
Subject: Re: Network performance - iperf
> Eric Dumazet wrote:
> > Le lundi 29 mars 2010 à 17:27 +0200, Michal Simek a écrit :
> >> Michal Simek wrote:
> >>> Eric Dumazet wrote:
> >>>> Le lundi 29 mars 2010 à 13:33 +0200, Michal Simek a écrit :
> >>>>
> >>>>> Do you have any idea howto improve TCP/UDP performance in
> general?
> >>>>> Or tests which can point me on weak places.
> >>>> Could you post "netstat -s" on your receiver, after fresh boot and
> your
> >>>> iperf session, for 32 MB and 256 MB ram case ?
> >>>>
> >>> I am not sure if is helpful but look below.
> >>>
> >> Sorry I forget to c&p that second part. :-(
> >>
> >
> > Sorry, your netstat is not up2date.
>
> I am afraid that is up2date.
>
> >
> > If you cannot correct it to last version
> > [ net-tools 1.60 , netstat 1.42 ], please send
> >
> > cat /proc/net/snmp
> > cat /proc/net/netstat
>
> There is small buffer for parsing /proc/net/netstat.
> There is necessary to extend buffer size because one line is greater
> than 1024 chars.
>
> ~ # head -n 1 /proc/net/netstat
> TcpExt: SyncookiesSent SyncookiesRecv SyncookiesFailed EmbryonicRsts
> PruneCalled RcvPruned OfoPruned OutOfWindowIcmps LockDroppedIcmps
> ArpFilter TW TWRecycled TWKilled PAWSPassive PAWSActive PAWSEstab
> DelayedACKs DelayedACKLocked DelayedACKLost ListenOverflows ListenDrops
> TCPPrequeued TCPDirectCopyFromBacklog TCPDirectCopyFromPrequeue
> TCPPrequeueDropped TCPHPHits TCPHPHitsToUser TCPPureAcks TCPHPAcks
> TCPRenoRecovery TCPSackRecovery TCPSACKReneging TCPFACKReorder
> TCPSACKReorder TCPRenoReorder TCPTSReorder TCPFullUndo TCPPartialUndo
> TCPDSACKUndo TCPLossUndo TCPLoss TCPLostRetransmit TCPRenoFailures
> TCPSackFailures TCPLossFailures TCPFastRetrans TCPForwardRetrans
> TCPSlowStartRetrans TCPTimeouts TCPRenoRecoveryFail TCPSackRecoveryFail
> TCPSchedulerFailed TCPRcvCollapsed TCPDSACKOldSent TCPDSACKOfoSent
> TCPDSACKRecv TCPDSACKOfoRecv TCPAbortOnSyn TCPAbortOnData
> TCPAbortOnClose TCPAbortOnMemory TCPAbortOnTimeout TCPAbortOnLinger
> TCPAbortFailed TCPMemoryPressures TCPSACKDiscard TCPDSACKIgnoredOld
> TCPDSACKIgnoredNoUndo TCPSpuriousRTOs TCPMD5NotFound TCPMD5Unexpected
> TCPSackShifted TCPSackMerged TCPSackShiftFallback TCPBacklogDrop
> TCPMinTTLDrop
>
> Look at attached patch.
> And updated results are below.
>
> Thanks,
> Michal
>
>
>
>
>
> 256M
>
> ~ # iperf -s
> ------------------------------------------------------------
> Server listening on TCP port 5001
> TCP window size: 85.3 KByte (default)
> ------------------------------------------------------------
> [ 6] local 192.168.0.10 port 5001 connected with 192.168.0.101 port
> 33261
> [ ID] Interval Transfer Bandwidth
> [ 6] 0.0-50.2 sec 22.9 MBytes 3.83 Mbits/sec
> ~ # ./netstat -s
> Ip:
> 16618 total packets received
> 0 forwarded
> 0 incoming packets discarded
> 16618 incoming packets delivered
> 6490 requests sent out
> Icmp:
> 0 ICMP messages received
> 0 input ICMP message failed.
> ICMP input histogram:
> 0 ICMP messages sent
> 0 ICMP messages failed
> ICMP output histogram:
> Tcp:
> 0 active connections openings
> 1 passive connection openings
> 0 failed connection attempts
> 0 connection resets received
> 0 connections established
> 16618 segments received
> 6490 segments send out
> 0 segments retransmited
> 0 bad segments received.
> 0 resets sent
> Udp:
> 0 packets received
> 0 packets to unknown port received.
> 0 packet receive errors
> 0 packets sent
> RcvbufErrors: 0
> SndbufErrors: 0
> UdpLite:
> InDatagrams: 0
> NoPorts: 0
> InErrors: 0
> OutDatagrams: 0
> RcvbufErrors: 0
> SndbufErrors: 0
> TcpExt:
> 2233 packets pruned from receive queue because of socket buffer
> overrun
> ArpFilter: 0
> 1 delayed acks sent
> 5519 packets header predicted
> TCPPureAcks: 2
> TCPHPAcks: 0
> TCPRenoRecovery: 0
> TCPSackRecovery: 0
> TCPSACKReneging: 0
> TCPFACKReorder: 0
> TCPSACKReorder: 0
> TCPRenoReorder: 0
> TCPTSReorder: 0
> TCPFullUndo: 0
> TCPPartialUndo: 0
> TCPDSACKUndo: 0
> TCPLossUndo: 0
> TCPLoss: 0
> TCPLostRetransmit: 0
> TCPRenoFailures: 0
> TCPSackFailures: 0
> TCPLossFailures: 0
> TCPFastRetrans: 0
> TCPForwardRetrans: 0
> TCPSlowStartRetrans: 0
> TCPTimeouts: 0
> TCPRenoRecoveryFail: 0
> TCPSackRecoveryFail: 0
> TCPSchedulerFailed: 0
> TCPRcvCollapsed: 207654
> TCPDSACKOldSent: 0
> TCPDSACKOfoSent: 0
> TCPDSACKRecv: 0
> TCPDSACKOfoRecv: 0
> TCPAbortOnSyn: 0
> TCPAbortOnData: 0
> TCPAbortOnClose: 0
> TCPAbortOnMemory: 0
> TCPAbortOnTimeout: 0
> TCPAbortOnLinger: 0
> TCPAbortFailed: 0
> TCPMemoryPressures: 0
> TCPSACKDiscard: 0
> TCPDSACKIgnoredOld: 0
> TCPDSACKIgnoredNoUndo: 0
> TCPSpuriousRTOs: 0
> TCPMD5NotFound: 0
> TCPMD5Unexpected: 0
> TCPSackShifted: 0
> TCPSackMerged: 0
> TCPSackShiftFallback: 0
> TCPBacklogDrop: 0
> TCPMinTTLDrop: 0
> IpExt:
> InNoRoutes: 0
> InTruncatedPkts: 0
> InMcastPkts: 0
> OutMcastPkts: 0
> InBcastPkts: 0
> OutBcastPkts: 0
> InOctets: 24915880
> OutOctets: 337488
> InMcastOctets: 0
> OutMcastOctets: 0
> InBcastOctets: 0
> OutBcastOctets: 0
> ~ # ./netstat --version
> net-tools 1.60
> netstat 1.42 (2001-04-15)
> Fred Baumgarten, Alan Cox, Bernd Eckenfels, Phil Blundell, Tuan Hoang
> and others
> +NEW_ADDRT +RTF_IRTT +RTF_REJECT +FW_MASQUERADE -I18N
> AF: (inet) +UNIX +INET -INET6 -IPX -AX25 -NETROM -X25 -ATALK -ECONET
> -ROSE
> HW: +ETHER -ARC +SLIP +PPP -TUNNEL -TR -AX25 -NETROM -X25 -FR -ROSE
> -ASH -SIT -FDDI -HIPPI -HDLC/LAPB
> ~ # head -n 1 /proc/meminfo
> MemTotal: 257108 kB
>
>
>
> 32MB
>
> ~ # head -n 1 /proc/meminfo
> MemTotal: 29920 kB
> ~ # iperf -s
> ------------------------------------------------------------
> Server listening on TCP port 5001
> TCP window size: 85.3 KByte (default)
> ------------------------------------------------------------
> [ 6] local 192.168.0.10 port 5001 connected with 192.168.0.101 port
> 50088
> [ ID] Interval Transfer Bandwidth
> [ 6] 0.0-50.0 sec 109 MBytes 18.3 Mbits/sec
> ~ # ./netstat -s
> Ip:
> 79040 total packets received
> 0 forwarded
> 0 incoming packets discarded
> 79040 incoming packets delivered
> 29655 requests sent out
> Icmp:
> 0 ICMP messages received
> 0 input ICMP message failed.
> ICMP input histogram:
> 0 ICMP messages sent
> 0 ICMP messages failed
> ICMP output histogram:
> Tcp:
> 0 active connections openings
> 1 passive connection openings
> 0 failed connection attempts
> 0 connection resets received
> 0 connections established
> 79040 segments received
> 29655 segments send out
> 0 segments retransmited
> 0 bad segments received.
> 0 resets sent
> Udp:
> 0 packets received
> 0 packets to unknown port received.
> 0 packet receive errors
> 0 packets sent
> RcvbufErrors: 0
> SndbufErrors: 0
> UdpLite:
> InDatagrams: 0
> NoPorts: 0
> InErrors: 0
> OutDatagrams: 0
> RcvbufErrors: 0
> SndbufErrors: 0
> TcpExt:
> 9773 packets pruned from receive queue because of socket buffer
> overrun
> ArpFilter: 0
> 1 delayed acks sent
> 101 packets directly queued to recvmsg prequeue.
> 558928 packets directly received from prequeue
> 33274 packets header predicted
> 378 packets header predicted and directly queued to user
> TCPPureAcks: 2
> TCPHPAcks: 0
> TCPRenoRecovery: 0
> TCPSackRecovery: 0
> TCPSACKReneging: 0
> TCPFACKReorder: 0
> TCPSACKReorder: 0
> TCPRenoReorder: 0
> TCPTSReorder: 0
> TCPFullUndo: 0
> TCPPartialUndo: 0
> TCPDSACKUndo: 0
> TCPLossUndo: 0
> TCPLoss: 0
> TCPLostRetransmit: 0
> TCPRenoFailures: 0
> TCPSackFailures: 0
> TCPLossFailures: 0
> TCPFastRetrans: 0
> TCPForwardRetrans: 0
> TCPSlowStartRetrans: 0
> TCPTimeouts: 0
> TCPRenoRecoveryFail: 0
> TCPSackRecoveryFail: 0
> TCPSchedulerFailed: 0
> TCPRcvCollapsed: 120195
> TCPDSACKOldSent: 0
> TCPDSACKOfoSent: 0
> TCPDSACKRecv: 0
> TCPDSACKOfoRecv: 0
> TCPAbortOnSyn: 0
> TCPAbortOnData: 0
> TCPAbortOnClose: 0
> TCPAbortOnMemory: 0
> TCPAbortOnTimeout: 0
> TCPAbortOnLinger: 0
> TCPAbortFailed: 0
> TCPMemoryPressures: 0
> TCPSACKDiscard: 0
> TCPDSACKIgnoredOld: 0
> TCPDSACKIgnoredNoUndo: 0
> TCPSpuriousRTOs: 0
> TCPMD5NotFound: 0
> TCPMD5Unexpected: 0
> TCPSackShifted: 0
> TCPSackMerged: 0
> TCPSackShiftFallback: 0
> TCPBacklogDrop: 0
> TCPMinTTLDrop: 0
> IpExt:
> InNoRoutes: 0
> InTruncatedPkts: 0
> InMcastPkts: 0
> OutMcastPkts: 0
> InBcastPkts: 0
> OutBcastPkts: 0
> InOctets: 118232864
> OutOctets: 1542068
> InMcastOctets: 0
> OutMcastOctets: 0
> InBcastOctets: 0
> OutBcastOctets: 0
> ~ #
>
>
>
>
>
>
>
>
>
>
> --
> Michal Simek, Ing. (M.Eng)
> PetaLogix - Linux Solutions for a Reconfigurable World
> w: www.petalogix.com p: +61-7-30090663,+42-0-721842854 f:
> +61-7-30090663
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/