Transmit timeouts with forcedeth card in 2.6.30.4
From: Chris Webb
Date: Thu Sep 17 2009 - 11:47:36 EST
Hi. I'm occasionally seeing the following transmit time-out from a forcedeth
ethernet interface with linux 2.6.30.4. We have a cluster of identical machines
with Supermicro H8DMT motherboards on which there are two such interfaces
integrated, and these errors appear on random machines across the cluster,
about once every two or three weeks per machine.
Following this error, it is impossible to ping in or out, but it is restored
back to life by a simple
ip link set eth0 down
ip link set eth0 up
The eth0 is enslaved to a bridge and has the mtu set at the default 1500. A
second forcedeth nic in the same machine with mtu set at 9000 and not joined to
bridge doesn't seem to suffer from this problem, although it may have much
lower traffic in any case.
forcedeth: Reverse Engineered nForce ethernet driver. Version 0.64.
forcedeth 0000:00:08.0: setting latency timer to 64
nv_probe: set workaround bit for reversed mac addr
forcedeth 0000:00:08.0: ifname eth0, PHY OUI 0x5043 @ 2, addr 00:30:48:67:7d:7c
forcedeth 0000:00:08.0: highdma csum vlan pwrctl mgmt gbit lnktim msi desc-v3
[...]
WARNING: at net/sched/sch_generic.c:226 dev_watchdog+0x130/0x1da()
Hardware name: H8DMT
NETDEV WATCHDOG: eth0 (forcedeth): transmit timed out
Modules linked in:
Pid: 0, comm: swapper Tainted: G M 2.6.30.4-elastic-lon-b #2
Call Trace:
<IRQ> [<ffffffff80257fa5>] warn_slowpath_common+0x77/0x8f
[<ffffffff8025800a>] warn_slowpath_fmt+0x3c/0x3e
[<ffffffff80260a27>] ? lock_timer_base+0x27/0x4d
[<ffffffff8058a872>] dev_watchdog+0x130/0x1da
[<ffffffff80260068>] ? ptrace_detach+0x86/0x95
[<ffffffff802607fe>] run_timer_softirq+0x164/0x1dc
[<ffffffff8058a742>] ? dev_watchdog+0x0/0x1da
[<ffffffff8025ca61>] __do_softirq+0xdc/0x1e3
[<ffffffff8022bf6c>] call_softirq+0x1c/0x28
[<ffffffff8022d7c8>] do_softirq+0x34/0x72
[<ffffffff8025c6ed>] irq_exit+0x3f/0x8b
[<ffffffff8023ab59>] smp_apic_timer_interrupt+0x89/0x97
[<ffffffff8022b983>] apic_timer_interrupt+0x13/0x20
<EOI> [<ffffffff80231919>] ? default_idle+0x9d/0x118
[<ffffffff80231c41>] ? c1e_idle+0xe3/0xea
[<ffffffff8026efb6>] ? atomic_notifier_call_chain+0xf/0x11
[<ffffffff8022a25c>] ? cpu_idle+0x52/0xb6
[<ffffffff808ce222>] ? start_secondary+0x1a4/0x1a8
---[ end trace 97f305223326b99b ]---
eth0: Got tx_timeout. irq: 00000036
eth0: Ring at 42596c000
eth0: Dumping tx registers
0: 00002036 00000000 00000003 0006000d 00000000 00000000 00000000 00000000
20: 00000000 00000000 00000000 00000000 00000001 00000000 00000000 00000000
40: 0420e20e 0000a855 00002e20 00000000 00000000 00000000 00000000 00000000
60: 00000000 00000000 00000000 0000ffff 0000ffff 0000ffff 0000ffff 00000000
80: 003b0f3c 00040001 00000000 007f0080 0000061c 00000001 00200000 80007f30
a0: 0016070f 00000016 67483000 00007c7d 00000001 00000000 00000000 00000000
c0: 10000001 00000001 00000001 00000001 00000001 00000001 00000001 00000001
e0: 00000001 00000001 00000001 00000001 00000001 00000001 00000001 00000001
100: 2596e000 2596c000 01ff00ff 00008000 00010064 00000000 0000004f 2596e150
120: 2596cb50 204308c0 a000fff2 00000000 00000000 2596e15c 2596cb50 01e08000
140: 00304120 80c02600 00000004 00000004 00000008 00000000 00000000 00000000
160: 00000004 00000004 00000004 00000004 01ff0080 0000c000 00000000 00000000
180: 00000006 00000008 0294796d 00008103 0000004a 00004000 00000591 0000c183
1a0: 00000006 00000008 0294796d 00008103 0000004a 00004000 0000059e 0000c583
1c0: 00000006 00000008 0294796d 00008103 0000004a 00004000 00000596 0000c583
1e0: 00000006 00000008 0294796d 00008103 0000004a 00004000 0000059b 0000c583
200: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
220: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
240: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
260: 00000000 00000000 fe027001 00000100 00000011 000000a3 fe027011 000001a3
280: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2a0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2c0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2e0: 00000000 00000000 00000000 00000000 00000000 00000001 00000001 00000001
300: 80212000 00000000 00000000 00000000 00000000 00002000 00000000 00000000
320: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
340: 00000000 00000000 00000000 00000000 00000000 00000020 d104438f 00000000
360: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
380: 00000000 00000000 00000000 00000000 00000000 00000000 00000002 00000000
3a0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
3c0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
3e0: 06255300 00701365 00000000 00000000 00000032 00000000 00000000 00000000
400: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
420: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
440: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
460: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
480: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
4a0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
4c0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
4e0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
500: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
520: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
540: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
560: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
580: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
5a0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
5c0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
5e0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
600: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
eth0: Dumping tx ring
000: 00000005 d4ea5802 20000029 // 00000006 92398802 20000029 // 00000004 423a0c02 20000029 // 00000
004: 00000005 dc2bf422 20000040 // 00000005 dc2b8422 20000062 // 00000005 dc2b8c22 2000005c // 00000
008: 00000008 20437c22 20000095 // 00000008 20432022 20000095 // 00000008 20435c22 20000095 // 00000
00c: 00000006 0aa3d822 2000006e // 00000004 fa8d3022 200005c6 // 00000005 7806f022 2000059e // 00000
010: 00000008 20435022 20000095 // 00000008 20437022 20000095 // 00000008 20435822 20000095 // 00000
014: 00000008 20430822 20000095 // 00000006 0aa38822 20000069 // 00000005 61eb3022 200005e9 // 00000
018: 00000005 7806d822 20000599 // 00000008 20436c22 20000090 // 00000008 20432c22 20000090 // 00000
01c: 00000008 20436822 20000090 // 00000005 7806f822 200005b7 // 00000008 206c8c22 2000003c // 00000
020: 00000008 206cc822 20000090 // 00000008 206ce022 20000090 // 00000008 206cc422 20000090 // 00000
024: 00000008 206c9022 20000090 // 00000008 206c8422 20000090 // 00000008 206cb422 2000003d // 00000
028: 00000005 0c077022 200005d5 // 00000005 7806a822 20000599 // 00000008 206ca822 20000090 // 00000
02c: 00000008 206cbc22 20000090 // 00000008 206c9c22 20000090 // 00000006 0aa3ac02 20000029 // 00000
030: 00000006 0aa3dc22 20000090 // 00000006 0aa3cc22 20000090 // 00000006 0aa3d422 20000090 // 00000
034: 00000006 0aa3bc22 20000090 // 00000006 0aa3a022 20000090 // 00000006 0aa39402 20000029 // 00000
038: 00000001 c34fe402 20000029 // 00000004 3683e022 200005e1 // 00000006 0aa3e422 200000a0 // 00000
03c: 00000006 0aa3c422 20000090 // 00000006 0aa3f822 20000090 // 00000006 0aa3b422 20000090 // 00000
040: 00000001 c34fdc22 20000069 // 00000001 c34fd402 20000029 // 00000008 25736022 200000a0 // 00000
044: 00000008 25731022 20000090 // 00000008 25734422 20000090 // 00000008 25735c22 20000090 // 00000
048: 00000008 25732422 20000090 // 00000005 dc2b9402 2c000059 // 00000008 25735002 20000029 // 00000
04c: 00000006 000fbc02 2c000059 // 00000008 25abc002 20000029 // 00000008 25abb002 20000029 // 00000
050: 00000005 dc2bd002 20000029 // 00000008 25ab8c02 20000029 // 00000006 249f5402 20000029 // 00000
054: 00000008 25abfc02 20000029 // 00000006 249f5802 20000029 // 00000005 dc2ba002 20000029 // 00000
058: 00000006 249f0402 20000029 // 00000008 254de402 20000029 // 00000008 254df802 20000029 // 00000
05c: 00000005 dc2ba822 20000041 // 00000006 25091c02 20000029 // 00000006 25096422 20000041 // 00000
060: 00000005 dc2b9002 20000029 // 00000006 25091402 20000029 // 00000006 249f6c22 20000041 // 00000
064: 00000008 2022b002 20000029 // 00000006 25090c02 20000029 // 00000005 e597f802 20000029 // 00000
068: 00000006 25093402 20000029 // 00000006 25093002 20000029 // 00000006 000f9802 20000029 // 00000
06c: 00000006 249f2422 2000003f // 00000006 000fcc02 20000029 // 00000006 249f1802 20000029 // 00000
070: 00000005 dc2ba402 20000029 // 00000005 dc2bc002 20000029 // 00000005 e597f402 20000029 // 00000
074: 00000008 206cd802 20000029 // 00000005 e580d402 20000029 // 00000006 000ff002 20000029 // 00000
078: 00000006 249f2c02 20000029 // 00000006 249f7802 20000029 // 00000008 2022c802 20000029 // 00000
07c: 00000006 000f9c02 20000029 // 00000006 249f3802 20000029 // 00000005 dc2bec02 20000029 // 00000
080: 00000005 e597b802 20000029 // 00000003 9a19cc02 20000029 // 00000003 9a19f802 20000029 // 00000
084: 00000004 25992c02 20000029 // 00000006 249f2802 20000029 // 00000004 25994c02 20000029 // 00000
088: 00000006 000f8402 20000029 // 00000005 dc2b9c02 20000029 // 00000006 25091002 20000029 // 00000
08c: 00000006 000f9422 2000003f // 00000006 000fa822 2000003f // 00000006 9239ac02 20000029 // 00000
090: 00000006 0aa38402 20000029 // 00000001 c34fc002 20000029 // 00000008 2022ac02 20000029 // 00000
094: 00000001 c34f9802 20000029 // 00000008 20433c02 20000029 // 00000005 e580d802 20000029 // 00000
098: 00000005 e5978002 20000029 // 00000001 c34fbc22 2000003f // 00000001 c34fcc02 20000029 // 00000
09c: 00000006 25096802 20000029 // 00000001 c34ff802 20000029 // 00000005 e580f402 20000029 // 00000
0a0: 00000005 e580b802 20000029 // 00000006 000fe802 20000029 // 00000001 c34fa402 20000029 // 00000
0a4: 00000005 e580b402 20000029 // 00000001 c34ffc02 20000029 // 00000001 c34f9002 20000029 // 00000
0a8: 00000001 c34fe002 20000029 // 00000005 dc2b9802 20000029 // 00000001 c34ff002 20000029 // 00000
0ac: 00000006 9239dc02 20000029 // 00000001 c34fe802 20000029 // 00000006 9239c802 20000029 // 00000
0b0: 00000001 c34f9402 20000029 // 00000006 9239d002 20000029 // 00000005 dc2bc402 20000029 // 00000
0b4: 00000006 249f1c02 20000029 // 00000006 000fc402 20000029 // 00000001 c34fd802 20000029 // 00000
0b8: 00000006 249f5c02 20000029 // 00000001 c34f8002 20000029 // 00000006 92398c02 20000029 // 00000
0bc: 00000001 c34f8402 20000029 // 00000001 c34ff402 20000029 // 00000008 206cb802 20000029 // 00000
0c0: 00000005 e5978c02 20000029 // 00000005 e5978802 20000029 // 00000001 c34fc402 20000029 // 00000
0c4: 00000005 e597e002 20000029 // 00000003 9a199c02 20000029 // 00000005 dc2bb402 20000029 // 00000
0c8: 00000005 e597fc02 20000029 // 00000003 9a19d002 20000029 // 00000005 e597d402 20000029 // 00000
0cc: 00000005 e580fc02 20000029 // 00000003 9a198c02 20000029 // 00000005 e580a402 20000029 // 00000
0d0: 00000003 9a19b002 20000029 // 00000004 25652002 20000029 // 00000004 25656002 20000029 // 00000
0d4: 00000005 e580dc02 20000029 // 00000004 25650802 20000029 // 00000005 e580ec02 20000029 // 00000
0d8: 00000006 000fe402 20000029 // 00000004 23942802 20000029 // 00000004 25997c02 20000029 // 00000
0dc: 00000004 25994802 20000029 // 00000004 25995802 20000029 // 00000006 92399802 20000029 // 00000
0e0: 00000006 92398002 20000029 // 00000004 23940802 20000029 // 00000006 9239d402 20000029 // 00000
0e4: 00000004 25993802 20000029 // 00000004 25995c02 20000029 // 00000004 25996002 20000029 // 00000
0e8: 00000004 25997402 20000029 // 00000003 9a19dc02 20000029 // 00000004 25917c02 20000029 // 00000
0ec: 00000004 25914c02 20000029 // 00000004 25914802 20000029 // 00000004 25911402 20000029 // 00000
0f0: 00000006 9239b402 20000029 // 00000001 c34fac02 20000029 // 00000006 92399002 20000029 // 00000
0f4: 00000006 92398402 20000029 // 00000001 c34fb002 20000029 // 00000005 d4ea4802 20000029 // 00000
0f8: 00000003 f5818c02 20000029 // 00000008 206ccc02 20000029 // 00000003 f581ec02 20000029 // 00000
0fc: 00000008 206cd402 20000029 // 00000003 f581c002 20000029 // 00000006 9239f802 20000029 // 00000
Best wishes,
Chris.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/