Scenario:
2.1.51 with 16M ram enabled
freepages set to 128 512 1024
overcommit_memory enabled
all ktracer features enabled (freezes with or without)
make -j bzImage
after short (1-2 sec) delay, start Bonnie on another vt
With this scenario, I get a 100% reproducable freeze where nothing
except SysRq works. (E,K and L don't help)
Mem-info:
Free pages: 3540kB
( 453*4kB 186*8kB 15*16kB 0*32kB 0*64kB 0*128kB = 3540kB)
Swap cache: add 46855/46855, delete 4967020/17854, find 125290/29001
Free swap: 240976kB ^^^^^^^ watching while frozen,
4096 pages of RAM this is the only thing
1209 free pages changing at all. (rapidly)
679 reserved pages
21 pages shared
Buffer memory: 4956kB
Buffer heads: 1283
Buffer blocks: 1239
CLEAN: 3 buffers, 3 used (last=3), 0 locked, 0 protected, 0 dirty
LOCKED: 11 buffers, 11 used (last=11), 0 locked, 0 protected, 0 dirty
DIRTY: 1225 buffers, 7 used (last=886), 1 locked, 0 protected, 1225 dirty
Is it possible that this is a hint? -------^
deadlock treshold exceeded, forcing Oops.
Unable to handle kernel NULL pointer dereference at virtual address 00000000
current->tss.cr3 = 00101000, `r3 = 00101000
*pde = 00000000
Oops: 0002
CPU: 0
EIP: 0010:[<c011b669>]
EFLAGS: 00010246
eax: 0000002a ebx: c01283c7 ecx: c01e11a4 edx: c0cee000
esi: 00000001 edi: 00000aa0 ebp: c0095f70 esp: c0095f5c
ds: 0018 es: 0018 ss: 0018
Process kswapd (pid: 3, process nr: 3, stackpage=c0095000)
Stack: c01c4a65 c023ff58 00000300 c01eb998 00009e20 c0095f8c c01283c7 c023ff58
00000300 00000aa0 0000002a c03ad000 c0095fb4 c011db27 c0c01740 c0095fb0
00000006 00000003 00000000 00000003 00001000 c0c01740 c0095fd0 c0122903
Call Trace: [<c01283c7>(0)] [<c011db27>(28)] [<c0122903>(40)] [<c0122b17>(28)] [<c0108a23>(28)] [<00000023>(4294369280)] [<ffffffff>] [<c01283c7>] [<c01283c7>] [<c01283c7>] [<c01283c7>] [<c01283c7>] [<c01283c7>] [<c01283c7>] [<c01283c7>] [<c01283c7>] [<c0
1283c7>] [<c01283c7>] [<c01283c7>] [<c01283c7>] [<c01283c7>] [<c01283c7>] [<c01283c7>] [<c01283c7>] [<c01283c7>] [<c01283c7>] [<c01283c7>] [<c01283c7>] [<c01283c7>] [<c01283c7>] [<c01283c7>] [<c01283c7>] [<c01283c7>] [<c01283c7>] [<c01283c7>] [<c01283c7>]
[<c01283c7>]
Code: c6 05 00 00 00 00 00 83 c4 04 eb 12 8d 74 26 00 8d bc 27 00
EIP (c011b669 = mcount)
try_to_free_buffer
shrink_mmap
try_to_free_page
kswapd
init
<00000023>(4294369280) no idea where this comes from
<ffffffff>
30 x try_to_free_buffer
Ksymoops segfaults on this oops.. ergo the manual trace.
After a freeze, sent both tasks a sigterm and waited for oops.
Did kprofile to see where it spent so much time.
9 0.02% c0131d68 shrink_dcache
12 0.03% c013b2d4 shm_swap
19 0.05% c0122698 swap_out_process
63 0.18% c01228ac try_to_free_page
126 0.36% c01222b0 swap_out_vma
383 1.11% c0121a6c kmem_cache_reap
3981 11.58% c012274c swap_out
4650 13.53% c01283c0 try_to_free_buffer
18120 52.74% c011d9e4 shrink_mmap
34355 100.00% 00000000 total
Bonnie and make -j will finish fine after being thawed by ktracer :)
One strange thing is that, if I start make -j and Bonnie at the same
time, the freeze doesn't happen.. only if I let it get a snoot full
of gcc first and then start Bonnie. A 1-2 second delay is enough.
Running pretty low on things to try (only one more).. ideas welcome.
-Mike