[RFC 2/2] User space tool for dump user space stack

From: Tu, Xiaobing
Date: Wed Apr 11 2012 - 04:23:18 EST

Next message: Mark Brown: "Re: [PATCH] regmap: fix compile errors in regmap-irq.c due to stridechanges"
Previous message: Mark Brown: "[PATCH] trace: Fix build breakage without CONFIG_PERF_EVENTS"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Resend as linux-kernel kicked it back.

From: Tu, Xiaobing
Sent: Wednesday, April 11, 2012 4:07 PM
To: 'akpm@xxxxxxxxxxxxxxxxxxxx'; 'mingo@xxxxxxx'; 'rusty@xxxxxxxxxxxxxxx'; 'a.p.zijlstra@xxxxxxxxx'; 'linux-kernel@xxxxxxxxxxxxxxx'; 'yanmin_zhang@xxxxxxxxxxxxxxx'; 'rostedt@xxxxxxxxxxx'
Cc: Zuo, Jiao
Subject: [RFC 2/2] User space tool for dump user space stack
Importance: High

Here is the a backtrace tool in user space which can dump all the stack for whole processes or a special process, it read kernel space and user space stack call chain from /proc/xxx/stack, and converts the hex format into symbols., we tested it on i386 and x86-64. It works well.

There are two function in this tool:
1. Parse special process we want, using âparse_stack âp 22740â
ããpid : 22740, name: droid.apps.maps
ããppid: 0, name:Â
ããchild count is 22
ãã
ããKernel Stack:
ãã[<c136e3df>] sys_epoll_wait+0x2bf/0x360
ãã[<c18d6ad1>] syscall_call+0x7/0xb
ãã[<ffffffff>] 0xffffffff
ããUser Stack:
ãã[00000000b778ebb7]Â /system/lib/libc.so
ãã[00000000b770cc98]Â /system/lib/libutils.so (_ZN7android6Looper8pollOnceEiPiS1_PPv)
ãã[00000000b75e6f8f]Â /system/lib/libandroid_runtime.so (_ZN7android18NativeMessageQueue8pollOnceEi)
ãã[00000000b75e6fc5]Â /system/lib/libandroid_runtime.so (_ZN7androidL38android_os_MessageQueue_nativePollOnceEP7_JNIEnvP8_jobjectii)
ãã[00000000b632fc40]Â /system/lib/libdvm.so (dvmPlatformInvoke)
ãã[00000000b637cf59]Â /system/lib/libdvm.so (_Z16dvmCallJNIMethodPKjP6JValuePK6MethodP6Thread)
ãã[00000000b6424214]Â /system/lib/libdvm.so
ãã[00000000b6341f86]Â /system/lib/libdvm.so (_Z11dvmMterpStdP6Thread)
ãã[00000000b633e626]Â /system/lib/libdvm.so (_Z12dvmInterpretP6ThreadPK6MethodP6JValue)
ãã[00000000b63aa9ca]Â /system/lib/libdvm.so (_Z15dvmInvokeMethodP6ObjectPK6MethodP11ArrayObjectS5_P11ClassObjectb)
ãã[00000000b63bb46a]Â /system/lib/libdvm.so (_ZL44Dalvik_java_lang_reflect_Method_invokeNativePKjP6JValue)
ãã[00000000b6424214]Â /system/lib/libdvm.so
ãã[00000000b6341f86]Â /system/lib/libdvm.so (_Z11dvmMterpStdP6Thread)
ãã[00000000b633e626]Â /system/lib/libdvm.so (_Z12dvmInterpretP6ThreadPK6MethodP6JValue)
ãã[00000000b63aad32]Â /system/lib/libdvm.so (_Z14dvmCallMethodVP6ThreadPK6MethodP6ObjectbP6JValuePc)
ãã[00000000b637ee7a]Â /system/lib/libdvm.so (_ZL21CallStaticVoidMethodVP7_JNIEnvP7_jclassP10_jmethodIDPc)
ãã[00000000b75b19da]Â /system/lib/libandroid_runtime.so (_ZN7_JNIEnv20CallStaticVoidMethodEP7_jclassP10_jmethodIDz)
ãã[00000000b75b3004]Â /system/lib/libandroid_runtime.so (_ZN7android14AndroidRuntime5startEPKcS2_)
ãã[00000000080491d7]Â /system/bin/app_process (main)
ãã[00000000b778cc0e]Â /system/lib/libc.so (__libc_init)
ãã[0000000008048c0a]Â /system/bin/app_process
ãã
ãã
ãã
ãã=========
ããtid : 22741, name: GC
ããppid: 22740, name: droid.apps.maps
ããKernel Stack:
ãã[<c1290d29>] futex_wait+0x5e9/0x8c0
ãã[<c1292dee>] do_futex+0xee/0xc60
ãã[<c12939fa>] sys_futex+0x9a/0x120
ãã[<c18d6ad1>] syscall_call+0x7/0xb
ãã[<ffffffff>] 0xffffffff
ããUser Stack:
ãã[00000000b778ed1d]Â /system/lib/libc.so
ãã[00000000b7786b49]Â /system/lib/libc.so (__pthread_cond_timedwait_relative)
ãã[00000000b7786c94]Â /system/lib/libc.so (__pthread_cond_timedwait)
ãã[00000000b7786d83]Â /system/lib/libc.so (pthread_cond_timedwait)
ãã[00000000b7786dbc]Â /system/lib/libc.so (pthread_cond_wait)
ãã[00000000b63cd3e0]Â /system/lib/libdvm.so (_ZL14gcDaemonThreadPv)
ãã[00000000b6392bbf]Â /system/lib/libdvm.so (_ZL19internalThreadStartPv)
ãã[00000000b77875a2]Â /system/lib/libc.so (__thread_entry)
ãã[00000000b773f2d0]ÂÂ (__thread_entry)
ãã[00000000b7731f92]Â /system/lib/libcutils.so (mspace_malloc)
ãã
ãã
ãã
ãã=========
ããtid : 22744, name: Signal Catcher
ããppid: 22740, name: droid.apps.maps
ããKernel Stack:
ãã[<c126ad66>] sys_rt_sigtimedwait+0x106/0x430
ãã[<c18d6ad1>] syscall_call+0x7/0xb
ãã[<ffffffff>] 0xffffffff
ããUser Stack:
ãã[00000000b778e45f]Â /system/lib/libc.so
ãã[00000000b638d0d4]Â /system/lib/libdvm.so (_ZL24signalCatcherThreadStartPv)
ãã[00000000b6392bbf]Â /system/lib/libdvm.so (_ZL19internalThreadStartPv)
ãã[00000000b77875a2]Â /system/lib/libc.so (__thread_entry)
ãã[0000000000000009]ÂÂ (__thread_entry)
ãã
ãã
ãã
ãã=========
ããtid : 22746, name: JDWP
ããppid: 22740, name: droid.apps.maps
ããKernel Stack:
ãã[<c184c65e>] unix_stream_recvmsg+0x39e/0x500
ãã[<c179d600>] sock_recvmsg+0xe0/0x110
ãã[<c179f086>] __sys_recvmsg+0xf6/0x1d0
ãã[<c179ff1c>] sys_socketcall+0x14c/0x4f0
ãã[<c18d6ad1>] syscall_call+0x7/0xb
ãã[<ffffffff>] 0xffffffff
ããUser Stack:
ãã[00000000b778e71b]Â /system/lib/libc.so
ãã[00000000b63b313b]Â /system/lib/libdvm.so (_ZL15jdwpThreadStartPv)
ãã[00000000b6392bbf]Â /system/lib/libdvm.so (_ZL19internalThreadStartPv)
ãã[00000000b77875a2]Â /system/lib/libc.so (__thread_entry)
ãã[00000000b773f2d0]ÂÂ (__thread_entry)
ãã[00000000b7731f92]Â /system/lib/libcutils.so (mspace_malloc)
ãã
ãã
ãã
ãã=========
ããtid : 22747, name: Compiler
ããppid: 22740, name: droid.apps.maps
ããKernel Stack:
ãã[<c1290d29>] futex_wait+0x5e9/0x8c0
ãã[<c1292dee>] do_futex+0xee/0xc60
ãã[<c12939fa>] sys_futex+0x9a/0x120
ãã[<c18d6ad1>] syscall_call+0x7/0xb
ãã[<ffffffff>] 0xffffffff
ããUser Stack:
ãã[00000000b778ed1d]Â /system/lib/libc.so
ãã[00000000b7786b49]Â /system/lib/libc.so (__pthread_cond_timedwait_relative)
ãã[00000000b7786c94]Â /system/lib/libc.so (__pthread_cond_timedwait)
ãã[00000000b7786d83]Â /system/lib/libc.so (pthread_cond_timedwait)
ãã[00000000b7786dbc]Â /system/lib/libc.so (pthread_cond_wait)
ãã[00000000b63cf079]Â /system/lib/libdvm.so (_ZL19compilerThreadStartPv)
ãã[00000000b6392bbf]Â /system/lib/libdvm.so (_ZL19internalThreadStartPv)
ãã[00000000b77875a2]Â /system/lib/libc.so (__thread_entry)
ãã[00000000b77326ee]Â /system/lib/libcutils.so (mspace_calloc)
ãã[00000000b63cccc4]Â /system/lib/libdvm.so (_Z18dvmHeapSourceAllocj)
ãã[00000000b2915440]ÂÂ (_Z18dvmHeapSourceAllocj)
ãã
ãã
ãã
ãã=========
ããtid : 22748, name: ReferenceQueueD
ããppid: 22740, name: droid.apps.maps
ããKernel Stack:
ãã[<c1290d29>] futex_wait+0x5e9/0x8c0
ãã[<c1292dee>] do_futex+0xee/0xc60
ãã[<c12939fa>] sys_futex+0x9a/0x120
ãã[<c18d6ad1>] syscall_call+0x7/0xb
ãã[<ffffffff>] 0xffffffff
ããUser Stack:
ãã[00000000b778ed1d]Â /system/lib/libc.so
ãã[00000000b7786b49]Â /system/lib/libc.so (__pthread_cond_timedwait_relative)
ãã[00000000b7786c94]Â /system/lib/libc.so (__pthread_cond_timedwait)
ãã[00000000b7786d83]Â /system/lib/libc.so (pthread_cond_timedwait)
ãã[00000000b7786dbc]Â /system/lib/libc.so (pthread_cond_wait)
ãã[00000000b638eb48]Â /system/lib/libdvm.so (_ZL11waitMonitorP6ThreadP7Monitorxib)
ãã[00000000b63b8eb4]Â /system/lib/libdvm.so (_ZL28Dalvik_java_lang_Object_waitPKjP6JValuePK6MethodP6Thread)
ãã[00000000b6424214]Â /system/lib/libdvm.so
ãã[00000000b6341f86]Â /system/lib/libdvm.so (_Z11dvmMterpStdP6Thread)
ãã[00000000b633e626]Â /system/lib/libdvm.so (_Z12dvmInterpretP6ThreadPK6MethodP6JValue)
ãã[00000000b63aad32]Â /system/lib/libdvm.so (_Z14dvmCallMethodVP6ThreadPK6MethodP6ObjectbP6JValuePc)
ãã[00000000b63aaef2]Â /system/lib/libdvm.so (_Z13dvmCallMethodP6ThreadPK6MethodP6ObjectP6JValuez)
ãã[00000000b639242a]Â /system/lib/libdvm.so (_ZL17interpThreadStartPv)
ãã[00000000b77875a2]Â /system/lib/libc.so (__thread_entry)
ãã[00000000b633dc12]Â /system/lib/libdvm.so (_Z9dvmMallocji)
ãã[00000000b63bb46a]Â /system/lib/libdvm.so (_ZL44Dalvik_java_lang_reflect_Method_invokeNativePKjP6JValue)
ãã[00000000b6424214]Â /system/lib/libdvm.so
ãã[00000000b6341f86]Â /system/lib/libdvm.so (_Z11dvmMterpStdP6Thread)
ãã[00000000b633e626]Â /system/lib/libdvm.so (_Z12dvmInterpretP6ThreadPK6MethodP6JValue)
ãã[00000000b63aad32]Â /system/lib/libdvm.so (_Z14dvmCallMethodVP6ThreadPK6MethodP6ObjectbP6JValuePc)
ãã[00000000b637ee7a]Â /system/lib/libdvm.so (_ZL21CallStaticVoidMethodVP7_JNIEnvP7_jclassP10_jmethodIDPc)
ãã[00000000b75b19da]Â /system/lib/libandroid_runtime.so (_ZN7_JNIEnv20CallStaticVoidMethodEP7_jclassP10_jmethodIDz)
ãã[00000000b75b3004]Â /system/lib/libandroid_runtime.so (_ZN7android14AndroidRuntime5startEPKcS2_)
ãã[00000000080491d7]Â /system/bin/app_process (main)
ãã[00000000b778cc0e]Â /system/lib/libc.so (__libc_init)
ãã[0000000008048c0a]Â /system/bin/app_process
ãã
ãã
ãã
ãã=========
ããtid : 22750, name: FinalizerDaemon
ããppid: 22740, name: droid.apps.maps
ããKernel Stack:
ãã[<c1290d29>] futex_wait+0x5e9/0x8c0
ãã[<c1292dee>] do_futex+0xee/0xc60
ãã[<c12939fa>] sys_futex+0x9a/0x120
ãã[<c18d6ad1>] syscall_call+0x7/0xb
ãã[<ffffffff>] 0xffffffff
ããUser Stack:
ãã[00000000b778ed1d]Â /system/lib/libc.so
ãã[00000000b7786b49]Â /system/lib/libc.so (__pthread_cond_timedwait_relative)
ãã[00000000b7786c94]Â /system/lib/libc.so (__pthread_cond_timedwait)
ãã[00000000b7786d83]Â /system/lib/libc.so (pthread_cond_timedwait)
ãã[00000000b7786dbc]Â /system/lib/libc.so (pthread_cond_wait)
ãã[00000000b638eb48]Â /system/lib/libdvm.so (_ZL11waitMonitorP6ThreadP7Monitorxib)
ãã[00000000b63b8eb4]Â /system/lib/libdvm.so (_ZL28Dalvik_java_lang_Object_waitPKjP6JValuePK6MethodP6Thread)
ãã[00000000b6424214]Â /system/lib/libdvm.so
ãã[00000000b6341f86]Â /system/lib/libdvm.so (_Z11dvmMterpStdP6Thread)
ãã[00000000b633e626]Â /system/lib/libdvm.so (_Z12dvmInterpretP6ThreadPK6MethodP6JValue)
ãã[00000000b63aad32]Â /system/lib/libdvm.so (_Z14dvmCallMethodVP6ThreadPK6MethodP6ObjectbP6JValuePc)
ãã[00000000b63aaef2]Â /system/lib/libdvm.so (_Z13dvmCallMethodP6ThreadPK6MethodP6ObjectP6JValuez)
ãã[00000000b639242a]Â /system/lib/libdvm.so (_ZL17interpThreadStartPv)
ãã[00000000b77875a2]Â /system/lib/libc.so (__thread_entry)
ãã[00000000b633dc12]Â /system/lib/libdvm.so (_Z9dvmMallocji)
ãã[00000000b63bb46a]Â /system/lib/libdvm.so (_ZL44Dalvik_java_lang_reflect_Method_invokeNativePKjP6JValue)
ãã[00000000b6424214]Â /system/lib/libdvm.so
ãã[00000000b6341f86]Â /system/lib/libdvm.so (_Z11dvmMterpStdP6Thread)
ãã[00000000b633e626]Â /system/lib/libdvm.so (_Z12dvmInterpretP6ThreadPK6MethodP6JValue)
ãã[00000000b63aad32]Â /system/lib/libdvm.so (_Z14dvmCallMethodVP6ThreadPK6MethodP6ObjectbP6JValuePc)
ãã[00000000b637ee7a]Â /system/lib/libdvm.so (_ZL21CallStaticVoidMethodVP7_JNIEnvP7_jclassP10_jmethodIDPc)
ãã[00000000b75b19da]Â /system/lib/libandroid_runtime.so (_ZN7_JNIEnv20CallStaticVoidMethodEP7_jclassP10_jmethodIDz)
ãã[00000000b75b3004]Â /system/lib/libandroid_runtime.so (_ZN7android14AndroidRuntime5startEPKcS2_)
ãã[00000000080491d7]Â /system/bin/app_process (main)
ãã[00000000b778cc0e]Â /system/lib/libc.so (__libc_init)
ãã[0000000008048c0a]Â /system/bin/app_process
ãã
ãã
ãã
ãã=========
ããtid : 22751, name: FinalizerWatchd
ããppid: 22740, name: droid.apps.maps
ããKernel Stack:
ãã[<c1290d29>] futex_wait+0x5e9/0x8c0
ãã[<c1292dee>] do_futex+0xee/0xc60
ãã[<c12939fa>] sys_futex+0x9a/0x120
ãã[<c18d6ad1>] syscall_call+0x7/0xb
ãã[<ffffffff>] 0xffffffff
ããUser Stack:
ãã[00000000b778ed1d]Â /system/lib/libc.so
ãã[00000000b7786b49]Â /system/lib/libc.so (__pthread_cond_timedwait_relative)
ãã[00000000b7786c94]Â /system/lib/libc.so (__pthread_cond_timedwait)
ãã[00000000b7786d43]Â /system/lib/libc.so (pthread_cond_timedwait_monotonic)
ãã[00000000b638eacb]Â /system/lib/libdvm.so (_ZL11waitMonitorP6ThreadP7Monitorxib)
ãã[00000000b638ec1f]Â /system/lib/libdvm.so (_Z14dvmThreadSleepyj)
ãã[00000000b63b9d7e]Â /system/lib/libdvm.so (_ZL31Dalvik_java_lang_VMThread_sleepPKjP6JValue)
ãã[00000000b6424214]Â /system/lib/libdvm.so
ãã[00000000b6341f86]Â /system/lib/libdvm.so (_Z11dvmMterpStdP6Thread)
ãã[00000000b633e626]Â /system/lib/libdvm.so (_Z12dvmInterpretP6ThreadPK6MethodP6JValue)
ãã[00000000b63aad32]Â /system/lib/libdvm.so (_Z14dvmCallMethodVP6ThreadPK6MethodP6ObjectbP6JValuePc)
ãã[00000000b63aaef2]Â /system/lib/libdvm.so (_Z13dvmCallMethodP6ThreadPK6MethodP6ObjectP6JValuez)
ãã[00000000b639242a]Â /system/lib/libdvm.so (_ZL17interpThreadStartPv)
ãã[00000000b77875a2]Â /system/lib/libc.so (__thread_entry)
ãã[00000000b633dc12]Â /system/lib/libdvm.so (_Z9dvmMallocji)
ãã[00000000b63bb46a]Â /system/lib/libdvm.so (_ZL44Dalvik_java_lang_reflect_Method_invokeNativePKjP6JValue)
ãã[00000000b6424214]Â /system/lib/libdvm.so
ãã[00000000b6341f86]Â /system/lib/libdvm.so (_Z11dvmMterpStdP6Thread)
ãã[00000000b633e626]Â /system/lib/libdvm.so (_Z12dvmInterpretP6ThreadPK6MethodP6JValue)
ãã[00000000b63aad32]Â /system/lib/libdvm.so (_Z14dvmCallMethodVP6ThreadPK6MethodP6ObjectbP6JValuePc)
ãã[00000000b637ee7a]Â /system/lib/libdvm.so (_ZL21CallStaticVoidMethodVP7_JNIEnvP7_jclassP10_jmethodIDPc)
ãã[00000000b75b19da]Â /system/lib/libandroid_runtime.so (_ZN7_JNIEnv20CallStaticVoidMethodEP7_jclassP10_jmethodIDz)
ãã[00000000b75b3004]Â /system/lib/libandroid_runtime.so (_ZN7android14AndroidRuntime5startEPKcS2_)
ãã[00000000080491d7]Â /system/bin/app_process (main)
ãã[00000000b778cc0e]Â /system/lib/libc.so (__libc_init)
ãã[0000000008048c0a]Â /system/bin/app_process
ãã
ãã
ãã
ãã=========
ããtid : 22752, name: Binder Thread #
ããppid: 22740, name: droid.apps.maps
ããKernel Stack:
ãã[<c16afcf8>] binder_thread_read+0x9b8/0xc60
ãã[<c16b43c1>] binder_ioctl+0x331/0x6b0
ãã[<c133f966>] do_vfs_ioctl+0x86/0x660
ãã[<c133ff7e>] sys_ioctl+0x3e/0x70
ãã[<c18d6ad1>] syscall_call+0x7/0xb
ãã[<ffffffff>] 0xffffffff
ããUser Stack:
ãã[00000000b778d776]Â /system/lib/libc.so
ãã[00000000b7698d60]Â /system/lib/libbinder.so (_ZN7android14IPCThreadState14talkWithDriverEb)
ãã[00000000b7699e3c]Â /system/lib/libbinder.so (_ZN7android14IPCThreadState14joinThreadPoolEb)
ãã[00000000b76a4dd0]Â /system/lib/libbinder.so (_ZN7android10PoolThread10threadLoopEv)
ãã[00000000b76ff954]Â /system/lib/libutils.so (_ZN7android6Thread11_threadLoopEPv)
ãã[00000000b75b1870]Â /system/lib/libandroid_runtime.so (_ZN7android14AndroidRuntime15javaThreadShellEPv)
ãã[00000000b770044e]Â /system/lib/libutils.so (_ZN13thread_data_t10trampolineEPKS_)
ãã[00000000b77875a2]Â /system/lib/libc.so (__thread_entry)
ãã[00000000b2915590]ÂÂ (__thread_entry)
ãã
ãã
ãã=========
ããtid : 22759, name: its
ããppid: 22740, name: droid.apps.maps
ããKernel Stack:
ãã[<c136e3df>] sys_epoll_wait+0x2bf/0x360
ãã[<c18d6ad1>] syscall_call+0x7/0xb
ãã[<ffffffff>] 0xffffffff
ããUser Stack:
ãã[00000000b778ebb7]Â /system/lib/libc.so
ãã[00000000b770cc98]Â /system/lib/libutils.so (_ZN7android6Looper8pollOnceEiPiS1_PPv)
ãã[00000000b75e6f8f]Â /system/lib/libandroid_runtime.so (_ZN7android18NativeMessageQueue8pollOnceEi)
ãã[00000000b75e6fc5]Â /system/lib/libandroid_runtime.so (_ZN7androidL38android_os_MessageQueue_nativePollOnceEP7_JNIEnvP8_jobjectii)
ãã[00000000b632fc40]Â /system/lib/libdvm.so (dvmPlatformInvoke)
ãã[00000000b637cf59]Â /system/lib/libdvm.so (_Z16dvmCallJNIMethodPKjP6JValuePK6MethodP6Thread)
ãã[00000000b6424214]Â /system/lib/libdvm.so
ãã[00000000b6341f86]Â /system/lib/libdvm.so (_Z11dvmMterpStdP6Thread)
ãã[00000000b633e626]Â /system/lib/libdvm.so (_Z12dvmInterpretP6ThreadPK6MethodP6JValue)
ãã[00000000b63aad32]Â /system/lib/libdvm.so (_Z14dvmCallMethodVP6ThreadPK6MethodP6ObjectbP6JValuePc)
ãã[00000000b63aaef2]Â /system/lib/libdvm.so (_Z13dvmCallMethodP6ThreadPK6MethodP6ObjectP6JValuez)
ãã[00000000b639242a]Â /system/lib/libdvm.so (_ZL17interpThreadStartPv)
ãã[00000000b77875a2]Â /system/lib/libc.so (__thread_entry)
ãã[0000000000001d49]ÂÂ (__thread_entry)
ãã
ãã

2. Parse stack for all the processes, using âparse_stackâ
Attachment is the file store the stack info.

Br
XiaoBing Tu
PSI@System Integration Shanghai

Attachment: parse_stack_upstream.tar.gz
Description: parse_stack_upstream.tar.gz

Next message: Mark Brown: "Re: [PATCH] regmap: fix compile errors in regmap-irq.c due to stridechanges"
Previous message: Mark Brown: "[PATCH] trace: Fix build breakage without CONFIG_PERF_EVENTS"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]