Screwed up locking?

[Ricky Beam]

Somewhere past 2.6.6 somebody screwed something up big time.  It looks like
*something* is grabbing a kernel lock and never letting go (or being
prempted.)  The system is stable for a while (in this case ~12 hours)
and then:
bad: scheduling while atomic!
 [<c02dbc24>] schedule+0xb24/0xb30
 [<c014190e>] __alloc_pages+0x1ce/0x370
 [<c0272527>] sock_aio_write+0x137/0x150
 [<c0222d6d>] n_tty_chars_in_buffer+0x3d/0x80
 [<c02dc13d>] schedule_timeout+0xbd/0xc0
 [<c0221163>] tty_poll+0x83/0xa0
 [<c01704be>] do_select+0x19e/0x2d0
 [<c0170170>] __pollwait+0x0/0xd0
 [<c0170907>] sys_select+0x2e7/0x540
 [<c010536b>] syscall_call+0x7/0xb
...
bad: scheduling while atomic!
 [<c02dbc24>] schedule+0xb24/0xb30
 [<c02db3a1>] schedule+0x2a1/0xb30
 [<c01306c8>] rcu_check_quiescent_state+0x78/0x90
 [<c011166b>] mark_offset_tsc+0x1fb/0x350
 [<c0224c80>] read_chan+0x0/0x820
 [<c02dbc66>] preempt_schedule+0x36/0x60
 [<c0224e0e>] read_chan+0x18e/0x820
 [<c0107ce4>] handle_IRQ_event+0x34/0x70
 [<c011b570>] default_wake_function+0x0/0x20
 [<c021f860>] tty_read+0x0/0x160
 [<c011b570>] default_wake_function+0x0/0x20
 [<c02dc5ef>] _spin_lock+0x6f/0x90
 [<c0224c80>] read_chan+0x0/0x820
 [<c021f99c>] tty_read+0x13c/0x160
 [<c0170865>] sys_select+0x245/0x540
 [<c015c070>] vfs_read+0xe0/0x150
 [<c012b103>] sigprocmask+0x63/0xd0
 [<c015c371>] sys_read+0x51/0x80
 [<c010536b>] syscall_call+0x7/0xb
bad: scheduling while atomic!
 [<c02dbc24>] schedule+0xb24/0xb30
 [<c011b5d1>] __wake_up_common+0x41/0x60
 [<c011b634>] __wake_up+0x44/0x60
 [<c02dbc66>] preempt_schedule+0x36/0x60
 [<c0226505>] pty_unthrottle+0x45/0x70
 [<c0222c3b>] check_unthrottle+0x3b/0x40
 [<c022518e>] read_chan+0x50e/0x820
 [<c0107ce4>] handle_IRQ_event+0x34/0x70
 [<c011b570>] default_wake_function+0x0/0x20
 [<c021f860>] tty_read+0x0/0x160
 [<c011b570>] default_wake_function+0x0/0x20
 [<c02dc5ef>] _spin_lock+0x6f/0x90
 [<c0224c80>] read_chan+0x0/0x820
 [<c021f99c>] tty_read+0x13c/0x160
 [<c0170865>] sys_select+0x245/0x540
 [<c015c070>] vfs_read+0xe0/0x150
 [<c012b103>] sigprocmask+0x63/0xd0
 [<c015c371>] sys_read+0x51/0x80
 [<c010536b>] syscall_call+0x7/0xb
...

Every keypress in my ssh session leave one of those on the console.  When
the connection closes, the system panics:

(Here we go again... I thought the interleaved panic BS was fixed.)

Warning: kfree_skb on hard IRQ c029e511
Warning: kfree_skb on hard IRQ c029e3cc
bad: scheduling while atomic!
 [<c02dbc 2s4>ys] _rsecahedd+ul0ex5+01x/0b2x840/
xb 3[0<                                         0
0 10[5<c39021>5c]0 8w9or>]k _rvefss_chreedad+0+x05x/f09x/016x
50                                                           1
rn e[<l cp0a1n2ibc10 3-> ]no sti gspyrnocicnmgas:k +A0ixee63,/ k0xidll0i
g  [i<cn0te15rrc3up71t >h]andler!                                       n

Once the locks are messed up, the system is doomed.

Version: 2.6.9-SMP-rc1 SMP BK[20040913014622] (preempt enabled)
System: Dual Athlon MP 1600 (Tyan Tiger MP)

I'm rebuilding with the current tree (BK[20040913174343]) with preempt turned
off.  If that fails, I'll enable lock debugging.

--Ricky


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/