wrong count of CPU usage

From: Azat Khuzhin
Date: Wed May 16 2012 - 10:33:34 EST


Hi all

I have machine that have uptime 212 days, kernel version:ÂLinux
2.6.35.14-97.fc14.x86_64 #1 SMP Sat Sep 17 00:15:37 UTC 2011 x86_64
x86_64 x86_64 GNU/Linux
Before top (1) utility shows some activity per process,Âas it should
be (sort by %CPU column)

But from some time, no activity in top (1) per process at all
 PID USER   ÂPR ÂNI ÂVIRT ÂRES ÂSHR S %CPU %MEM  ÂTIME+ ÂCOMMAND
  1 root   Â20  0 19420 1548 1240 S Â0.0 Â0.0 Â17:48.09 init
  2 root   Â20  0   0  Â0  Â0 S Â0.0 Â0.0  0:09.72 kthreadd
  3 root   Â20  0   0  Â0  Â0 S Â0.0 Â0.0 Â20:34.76 ksoftirqd/0

But load avg > 10
top (1) shows some activity on cores (I mean not processes, that
created by kernel)
top - 09:58:26 up 212 days, Â7:06, Â8 users, Âload average: 13.27, 12.20, 11.37
Tasks: 815 total, Â 9 running, 803 sleeping, Â 2 stopped, Â 1 zombie
Cpu0 Â: 47.5%us, Â8.0%sy, Â0.3%ni, 26.4%id, 16.9%wa, Â0.0%hi, Â0.9%si, Â0.0%st
Cpu1 Â: 39.7%us, Â5.2%sy, Â0.0%ni, 48.0%id, Â6.8%wa, Â0.0%hi, Â0.3%si, Â0.0%st
Cpu2 Â: 29.8%us, Â3.1%sy, Â0.0%ni, 59.7%id, Â7.4%wa, Â0.0%hi, Â0.0%si, Â0.0%st
Cpu3 Â: 15.9%us, Â2.8%sy, Â0.0%ni, 77.3%id, Â3.7%wa, Â0.0%hi, Â0.3%si, Â0.0%st
Cpu4 Â: 39.3%us, Â5.2%sy, Â0.0%ni, 37.4%id, 17.8%wa, Â0.0%hi, Â0.3%si, Â0.0%st
Cpu5 Â: 29.6%us, Â4.0%sy, Â0.0%ni, 62.3%id, Â3.7%wa, Â0.0%hi, Â0.3%si, Â0.0%st
Cpu6 Â: 20.4%us, Â5.0%sy, Â0.0%ni, 74.3%id, Â0.0%wa, Â0.0%hi, Â0.3%si, Â0.0%st
Cpu7 Â: 17.3%us, Â2.8%sy, Â0.0%ni, 76.8%id, Â3.1%wa, Â0.0%hi, Â0.0%si, Â0.0%st
Cpu8 Â: 32.8%us, Â6.5%sy, Â0.0%ni, 31.6%id, 29.1%wa, Â0.0%hi, Â0.0%si, Â0.0%st
Cpu9 Â: 27.6%us, Â3.1%sy, Â0.0%ni, 69.3%id, Â0.0%wa, Â0.0%hi, Â0.0%si, Â0.0%st
Cpu10 : Â6.8%us, Â2.5%sy, Â0.0%ni, 87.9%id, Â2.8%wa, Â0.0%hi, Â0.0%si, Â0.0%st
Cpu11 : Â8.7%us, Â1.9%sy, Â0.0%ni, 89.4%id, Â0.0%wa, Â0.0%hi, Â0.0%si, Â0.0%st
Cpu12 : 27.3%us, Â2.8%sy, Â0.0%ni, 52.8%id, 17.1%wa, Â0.0%hi, Â0.0%si, Â0.0%st
Cpu13 : 18.2%us, Â3.7%sy, Â0.0%ni, 77.5%id, Â0.6%wa, Â0.0%hi, Â0.0%si, Â0.0%st
Cpu14 : 21.4%us, Â3.8%sy, Â0.0%ni, 74.8%id, Â0.0%wa, Â0.0%hi, Â0.0%si, Â0.0%st
Cpu15 : 10.0%us, Â1.9%sy, Â0.0%ni, 87.9%id, Â0.3%wa, Â0.0%hi, Â0.0%si, Â0.0%st

So seems like after some time kernel not updating /proc/PID/stat properly

I wrote shell script (attach), that count CPU usage, and it also shows 0
This script base on 14, 15 columns from /proc/PID/stat (as man proc
says, that it utime and stime clocks, and /proc/stat summary), but for
newly created processes

Summarize
/proc/PID/stat seems not change counters utime, stime
/proc/stat updated

--
Azat Khuzhin

Attachment: cpu_usage
Description: Binary data