---876492614-1047046357-874747376=:202
Content-Type: TEXT/PLAIN; charset=US-ASCII
Hi Gurus..
This if fully reproducable using Dave Miller's fun_program with top -d 0.
This kernel is pre-2.1.56.. any kernel since the appearance of dentries
will do it though.
If you manage to get enough short term processes spawning, eventually
you will see a filetable overflow. After this happens, top ceases
displaying process info.
[This is a bit extreme :), but it will do it under less load too if you are
patient enough.]
In this instance I started 155 fun_progs, and then a make -j bzImage and a
make -j modules. The makes failed after appearance of vfork: try again when
#processes reached ~400 and a load of ~200. I started the makes again, and
almost immediately received a filetable overflow from one of the makes.
[from top.. much later]
9:04am up 1:52h, 6 users, load average: 0.83, 0.95, 11.9353549
0 processes: 0 sleeping, 0 running, 0 zombie, 0 stopped
CPU states: 0.0% user, 0.0% system, 0.0% nice, 100.0% idle
Mem: 78580K av, 15736K used, 62844K free, 3428K shrd, 2248K buff
Swap: 261496K av, 2740K used, 258756K free 2172K cached
USER PID %CPU %MEM NI VSZ RSS SHRD TT STAT TIME COMMAND
top is displaying no process info due to too many entries in
/proc/4225/fd.. exactly 1024.
mikeg:# strace -p4225
[SNIP]
stat("/proc", {st_mode=S_IFDIR|0555, st_size=0, ...}) = 0
open("/proc", O_RDONLY) = 9
fcntl(9, F_SETFD, FD_CLOEXEC) = 0
getdents(9, /* 58 entries */, 1024) = 976
stat("/proc/1", {st_mode=S_IFDIR|0555, st_size=0, ...}) = 0
open("/proc/1/stat", O_RDONLY) = -1 EMFILE (Too many open files)
stat("/proc/2", {st_mode=S_IFDIR|0555, st_size=0, ...}) = 0
open("/proc/2/stat", O_RDONLY) = -1 EMFILE (Too many open files)
stat("/proc/3", {st_mode=S_IFDIR|0555, st_size=0, ...}) = 0
[SNIP]
mikeg:# /bin/ls /proc
1/ 10/ 125/ 127/ 129/ 131/
133/ 135/ 149/ 155/ 156/ 157/
158/ 159/ 160/ 163/ 2/ 3/
31510/ 31511/ 31525/ 31529/ 4225/ 45/
57/ 59/ 9/ cmdline cpuinfo devices
dma filesystems interrupts ioports kcore kmsg
ksyms loadavg locks mdstat meminfo misc
modules mounts net/ pci profile rtc
scsi/ self@ slabinfo stat swaps sys/
tty/ uptime version
now watch this..
[/proc/17239 selected from top's fd directory]
mikeg:# /bin/ls -l /proc/17239
/bin/ls: /proc/17239/exe: No such file or directory
/bin/ls: /proc/17239/root: No such file or directory
/bin/ls: /proc/17239/cwd: No such file or directory
total 0
-r--r--r-- 1 root root 0 Sep 20 09:15 cmdline
lrwx------ 1 root root 0 Sep 20 09:15 cwd
-r-------- 1 root root 0 Sep 20 09:15 environ
lrwx------ 1 root root 0 Sep 20 09:15 exe
dr-x------ 1 root root 0 Sep 20 09:15 fd
pr--r--r-- 1 root root 0 Sep 20 09:15 maps
-rw------- 1 root root 0 Sep 20 09:15 mem
lrwx------ 1 root root 0 Sep 20 09:15 root
-r--r--r-- 1 root root 0 Sep 20 09:15 stat
-r--r--r-- 1 root root 0 Sep 20 07:46 statm
-r--r--r-- 1 root root 0 Sep 20 09:15 status
mikeg:#
see attachment for the complete listing of /proc/4225/fd.
If you need more info, just holler.
-Mike
---876492614-1047046357-874747376=:202
Content-Type: TEXT/PLAIN; charset=US-ASCII; name="proc_4225_fd.gz.uu"
Content-Transfer-Encoding: BASE64
Content-ID: <Pine.LNX.3.95.970920112256.202B@mikeg.weiden.de>
Content-Description: strange stuff
YmVnaW4gNjQ0IHByb2NfNDIyNV9mZC5neg0KTSdYTCgiLF4tKFMwYGBXIVI7
Vi0/LSMoUi01XUY5YCJNVzxWTiZcPDFBTiZdS0gpYFVMKjlbT0tXUEtGKA0K
TTcoJDxSODgwUjEqTCRSPlleVSJWKic8KTAxOU9eMlAsK1wzIkQvUUpJR04+
R047W01fXVhcXS1XP1tEXQ0KTV8/I1ZZWj1XR1ZYT1ddVj4vT1tSWD5dL05L
PF1fP0NaUT8uJ1lVP087TT4rPVtfXFpdXE8/X05bVzZbSw0KTV1MTicjXF5X
U1ddP19eP1cvXT87V11ZXE8uV0s9TjVXPk9fT1s+N1dNWj83O1dZXT5HW15D
XSxKWl1MSg0KTStVJjVXUF9GTVs+NzJfPzNJXj03U15dWU06LTw5PjlPWTVb
XF0+N0NbX102TldcUFdVOlshSl09QF0/Tg0KTVA2TUVcJUg9Ty04JktfNyE6
Vi9QVkFSXE1HQSo1VE1JIzUuWiFCRT1AWTJOMDRLNygqNUtELShVMi5EOg0K
TUknMC00S0gmKjVWI0UuWF80S0pOViMmLFoxWyQ9YF1CTkA8UVc8KF8vTkQ/
V1k9SyQyWkdPNVAtUlVESw0KTSldPlBHIV9FRUVfMy1RTl1XLzMtOUJOV0lG
XlY+S0RVKlc9PCMpOUxcNiZVSlFfPCdJOTtPOVAsUl5VPg0KTTNIP0VJKT5T
ODtGU1FbOk0mKTpTN0ZbOCUsTVtONiUzSyZDRV1LYElVTUVDSENbLTc/NVJQ
XVI9TDcvUQ0KTTg0W1ZVMF1ORi4pXV0tQFZTNidMXU5bRUFLJztUTEwtOFs+
VUU6TUFbKztVPEwtU1w/OT47SUNCJztXPA0KTS0sNzlSVFVTNVtXPEwiTy48
VydKRVZMVkdcSytVOF1OPiVWNFdGMFU7IytJMzU7IylJLz45IzVMLEZFLQ0K
TTlNPlBSPDFaTjYmM0I/PVJQUjozWi42JjMyOzlSUFY7MypKNyZTOzlePDZO
RihZMF0uSCctXlBRNztXPA0KTV0uQVYrUz9MLDk1PjtNQUNWR01MLkpHMFdG
LzMyODdWJ0lNLmBbM1dWJzA6SCtXJ0lNLGBbM1RWRzA6PA0KTVdcMy4rXCxR
P0ImU1dGMUtWJjM2RlZRWiJLIz45LS0pQk80RkZUWTJLIz85PSkpQk88R1ZM
LEZMLV1EPg0KTS1JR1UpTU8jKUsvPjknTzg5LTo6Kyo5R1hfLkNEX1dFWCdC
Mz42TlJGJ1lWV0lITDkhQUM7VFQ2LU9QSg0KTTcnSllYP0cuTTk8O0dKIzw+
S0dBIjxKXUVZT0ZbRlBST1o4QzszXjtTLT1USS5XNVJQVS80LjxNR05NKw0K
TTNfYD5CWkw/Vy8hVCVaTjcmWzlMWyVZTlYmLEFPPVJQUVQpW042Ji9BPzVS
UFFYK1teNiYvMTsxUlBVWw0KTSsnSi8zND86VDdNTC5NKF4jRFk2M0dMTD5Y
XS1RXlc5PlZQWjtMXz44XS1RPl84PkZYWztMXz84PSoiPQ0KTU88PkZgXldM
LzM4PTomP09MPkVgLldOLzMwPzpWN01MLk1gXkpLRUkjR05MPkhdLUFeVzU+
VlBaXSpTPg0KTThdLUE+XzQ+VlwsPkpaLyc5J05MWjc9USZFRlozXE9VXl8/
My1XT1RGLlo8V00mTkhcPlQ4R0lQMVUvOA0KTV08PmBgQldGVldfKjhLW0Y4
RE1DTilJT0daT1lbRy5QWDs1VUdcTk5bRVZfQVM9Vj1FTEAoWzcmUVc8Kg0K
TUJNRjAqJzk7LDEqL01cTytNNFotWjZWUVc5OixFLk9MLys/LykyLVtCOFAv
W1tQQD9POkw6O1dTQkdDXw0KTTtGOzdaXVQ2Qz9APjdBK1dONklEQlxMVFJO
PlJEOlQ6WF4tSyVcNDwsTlxeSzcqR1JbMzw+NSZcI1spRw0KTWBYSV1XSCJe
RVxPOUE2Klc9Mi1LPi1HOUpdVkQnKFlCXVxIX1w3Sl1WU0NRJkxZQlNWNDQy
Vk1YJlZOPw0KTVVUOT0sQ1ZWSFtdNDszO0RXLj07QzhJQT8sXCUoUkRaLzk2
PCJUO0osPCdGVThbPipLJ0FEJlo/WlQ3Ng0KTUk2TTxLUl1cVS4mKVo1UFA4
TSxZWENbN0JdQi5WPzFaTV40QihMLzYvVT4rNiRUJyRdKSZCRiszQl9eWQ0K
TTZGMU1HVztfTjVJRFsxSi9MXF9FKE8/MTEkUTtdRVBPPENdVVpPIiQ8Slg3
NjVdN0xQUC5bVlJSXlMhWw0KTT1KXUNHUE0mW0JVS1RbJy5ONilEPjBRTzkr
P1U9Tyg4PV8qQy5ZPixLKEBdUz1aWTlOMV4nSENJRVdMTg0KTSZLRS80WDtL
WSs9XjxZSjpFQ0xbSzpbQUBMIV0rQUk5STVfJj1ILC9LXFcpSU5MNV1bRUo5
LTVUWz0tTg0KTUpUOk4lPS5DLlY9RFVWLFUpP19QJkdOTkYjOTo2UzlSJ1Ng
LitbPU1WPEMyRy85TTZTPlI8R0lTOys+JQ0KTShbTUQ+QVlIKlQ/JDkvU01H
RV5PUS4uVisnUlI6IU1eRUZQOz9JMUwmV1oyOyFNXkQqUk1KPzsnKEwvVw0K
TV4kQDwvVD5WJzNdJk1BVF8xNzg/W18mN1hEPD1bWVxUP1JEL0lfLVAuQF5H
XFciPk1TLSpBTF9CW04jOQ0KTSMpWy1YLUQsR0xXQFYwUj5TPiM5IylbLVgt
RCxHTFVTTyMvPlM2YEdTVjtSOyI7LzlPKUwpTF1GXEZQRg0KTVNWO1I7Ijsv
OU8pTEdJLzQ/IlFbSFw9MS8pSyVIVURcRkw2QzYzUjpROi05LylLJUhVRFxG
SDZDXj9NJA0KTVhIP1svV0taXi5HVUpePTcvW1NaXWA4Ny5eXEtGT0hIWVwr
V0RRJl5HOFNQVzYyJDtSOEM/Ilw5WDVPKQ0KTSItXSkxT0EmLEwrV0QxJl5D
PFFZRF1LMzkzOk0kS1orQy8hLTkoM08oMi1cIlFHQS5cQChXVCEmXi9YUQ0K
TVBLPi8kO1lbQy8jLThTSVg7KVUtSlgxTycyLVxZUUNBJlw/KFlNRzwvKU47
OVcvUzsmWj5TPFZTTjdEVg0KTUk2RjdTXltEQj8hTCJMXkZcJlAqU1o7UDtg
Sy9JTyFMIkxeRlwmUCpTVjk/TFI8VU8qQEsjWj9SPCJILw0KTUlfKVAqQF5H
XEdgSiNaP1I8IkgvSV8pUFZPXzQwN0hAUUstSS8pTyZMVkRcRlw6UzozUjsn
KUYkKFktUA0KTTkhKi4zJlUvRD81WFBJOj8uI0RSIjQ8Rlg8QEQnKUYkKFkt
UDkhKi4zLCoxMjNAUiI0PkZPRC8tVU4mKQ0KTURSLjM8JjAyQ0RTIkQ0RFgs
QEUnKU4nKCkhUjlBIi4zPCYzWjlLTkNGLldSKkFSOUEiLjM8JjAyQ0RTIg0K
TUQ0RFgsQEUnKU4nKCkhUjlBIi0zVlNPKz1dR1BITFo5MjNAUyI2PEZYPFBE
RylGJCxZLVA5QSsuMywqOQ0KTTIzQFMnMy9VP1dZXD9PTD4kWS5WUVknIzla
LCNZPDpEVylCNCZZLVI4VSlOMyxKLTI7RFEqMzxGWTw6RA0KTVcpQzpDR1o1
PjxXLlAsSjEyM0RSKjQ8Rlk8QkQnKUY0KFktUjklKi4zLEoxMjNEUk0/VEFR
Xk5DRSIuMw0KTTxGMTJDRFMqRDREWSxCRScpTjcoSSFSOUUiLjM8RjBaR1FO
MVNTTlUjRF48NylGNCpZLVI5NSpOMyxKNQ0KTTI7RFIqNTxGWTxKRDcpRjQq
XS9ZJSkrWVwoPT1FIU4zPEYtMjtEUypDNEZZLDJEVylONyZJLVI4RSFOMw0K
TTxGLFpHVkIrUVNYL18rMylDNEZZLDJEVylONyZJLVI4RSFOMzxGLTI7RFMq
QzRGWSw7NkcoXT9DKVRZWA0KTS4rRFEqMzxGWTw6RFcpQjQmWS1SOFUpTjMs
Si0yO0RRKjM+Rl1KU01VW1ZXXCglUDhVKU4zLEotMjtEUQ0KTSozPEZZPDpE
VylCNCZZLVI4VSlOMy1JXiZSS1ZcI1pePEYxMkNEUypENERZLEJFJylONyhJ
IVI5RSIuMw0KTTxGMTJDRFNHLUAjK0pHKDgzSFksUkklKS4zKEkxUjtFUiow
PEY5MENEVylENEhZLFJJJ0lGYFc9U1NeUw0KTVM0Py4jMjU2K0lMXUxWPzxG
OFBbRFclRyxOWSxRSVcpTiMsOT1SO0NTRjM8RjhQW1RbRFtSMko7U09OLQ0K
TS5ZLVE5UytOMyw6PVI7QFMmNzxGWFxZRFcpRiwuWS1ROV9JRkpZT0FcQ1dD
U0YzPEY4UFtEVyVHLE5ZLA0KTVFJVylOIyw5PVI7Q1NGMzxGPUpWMjdLRVsl
Wl48NjhSU0RTJkY8RFgsUUVHKU4vLDkpUjlDIy4zPDY4Ug0KTVNEUU0iUls/
Q1FdOkgwPyJHPEZYLFFFVylOLy45LVI5QyNOMzw2PFJbRFMmRzxGWCxbN01X
JygsMzw6QQ0KTVIzQFQmODxGWF0hRCcpSiwwWS1ROiMoLjM8OkFSM0BURztc
S14/RlckOD9XMlhVI0RXJUgsQFktUUonKQ0KTS4jMDlBUjtDVCYwPEZIUSNE
VyVILkdeRTUvO0NAMSQ+M0JZLVFKNylOIzA5RVI7Q1RGMTxGSFErRFclSQ0K
TSxCWS1RSjdJXyxXOz0+NVBKVSdDVCYwPEZIUSNEVyVILEBZLVFKJykuIzA5
QVI7Q1QmMDxGTUgvKiNdXg0KTSUpLUdEU04zPDY8UltEUyZHPEZYLFFFVylO
Ly45LVI5QyNOMzw2PFohSVNPL0tRWl8+SzdHVyJFPFwuXA0KTUtTXSshXV4x
PFYxUkNEUy5EPERZLENFJylOPyhZIVI5RyIuMzxWMVJDRFM+LUxSW0FBT0Yu
MzxGWVw7RA0KTVcpQjwmWS1TOFcpTjMsWi1SO0RRLjM8RllcO0RBUyc1OTwv
MylCPEZZXDNERylCPCRZLVM4RykuMyxaKQ0KTVIzRFEuMjxGWVwzRFlXWVlF
X0FQNE43PEYpUDtEVy1DPEZZLFNIVylOMyRZLVI7R1FOMzxGKVA7RFlcOw0K
TVlKVj0sVi1SO0RTLkM8RlksM0RXKU4/JlktUjhHIU4zPFYtUjtEUy5DPEcv
Xz8pQlY/MyRSOFcpTjMkWQ0KTS1SO0dRTjM8RilQO0RXLUM8RlksU0hXKU4z
J1lONic+NTMkXDw3KUI8RFksU0hHKS4zJFkpUjtHUS4yPA0KTUYpUDNEVy1C
PERZLFdHWys/PE1QMjtZUzhHKS4zLFopUjNEUS4yPEZZXDNERylCPCRZLVM4
RykuMydZTg0KTUYrPUwjPl9DLlM8RllcO0RXKUI8JlktUzhXKU4zLFotUjtE
US4zPEZZXDtESVMmOSMpPzwuMzxGWVw7RA0KTVcpQjwmWS1TOFcpTjMsWi1S
O0RRLjM8RllcOVQ3LTM/T1dHX11OPD8vXiEqWzRGRlJNRiJfLiMoJSFSOQ0K
TUBCLTM8JjAqQ0RTIUQyRFgsQDUnSU4jKCUhUjlYRCJGXl9EVzs5OS1DRFMh
RDJEWCxANSdJTiMoJSFSOQ0KTUBCLTM8JjAqQ0RTIUQyRS45KkpBP1A5J0lO
IyglIVI5QEItMzwmMCpDRFMhRDJEWCxANSdJTiMoJWBWOQ0KTV1PIipTSERJ
LiMkJSlaO0BRITI8RigoMzRXIUIiRFksUDhESS4jJiVHQE4+OltAJTtHIUIi
RFksUDhESQ0KTS4jJCUpWjtAUSEyPEYoKDM0VyFCIkRZLDg+PjJJTkQmXjwm
JSo7QFAhMT5GWCwoNDcpQiIiVS1QODBITg0KTTMsJiUqO0BQUTIlLE0/OUw9
XjtAUCEwPEYoKCM0VyFAIkBZLFA4JEkuI2AlIVo7QFAhMDxGLis8KyRdXA0K
TUZEV04yXCVdKjtATyE/PkVYK1g0Vyk+Il5VKVA3UEtOMlwlXSo8WSdGLlgz
U11HL1RQO0dJPiJcJSlSNw0KTUBPLTI8JVgqU0RPIT4yRFgrUDdHST4iXCUt
NlY6O0omLVM8WStQN0dJPiJcJSlSN0BPLTI8JVgqU0RPIQ0KTT4yRFgrUDdH
ST4uVFo9PjVIWldSXE9RKSlHRFxKVD8/NScpQDJAWSxSOCRJLjNgRSFaO0RQ
KTA8RikoIw0KTTRXKUAyQFksPjAhMzpgU09YQjwnSU4zYEUhUjhEQC0zPEYh
KiNEUylAMkRZLCI0J0lOM2BFK01FPFM5VA0KTTMwWSxSOCRJLjNgRSFaO0RQ
KTA8RikoIzRXKUAyQFksUjgkSUk0NVMtRyktI0RTKUAyRFksIjQnSU4zYA0K
TUUhUjhEQC0zPEYhKiNEUylAMkZVMTcvVlsmPVI3VEtOMlxFXSo7RE8pPz5F
WStaNFcpPjJeVSlSN1RLTg0KTTJWRE1GSy89MTkvUzRHKT4yTFkrUjdESS4y
XEVZWjdETykyPEVZK1M0Ryk+UkguN1tGLkBYMDY9XFUpUg0KTTdESy4yXEVZ
KjNETyk+PkVZK1I0Ryk+MlxVKVI3TElIVDFRPlQrRE4pPT5FWStKNDcpPjJa
VSlSNzRKTg0KTTJcRVUqO0ROKT0+RVMnLj8kOV9dMkVBUjckSi4yXEVRKjNE
Tik8PkVZK0I0Jyk+MlhVKVI3JEouMlVETQ0KTUZLLEVILEVNKjtETSk7PkVZ
Kzo0Vyk6MlZVKVI2VElOMlxFTSo7RE1VPTZCLjtOTzY5UiNCRy0wPDBYSg0K
TVNEJyUuOkBYIVE3R0guKDw1KVIjQkctMEsxOy1WMzJITmA4NVVaI0ImRTE8
QFhJSzQnJC0qSlkhUTM2SA0KTU5gODVVWiFKPl1LWTshSTQ3KC4qOlUhUSMy
Sk4wPDRVSitAJiU9PkBYQUk0NyguKjpVYD0mRTBZVyM2TA0KTU5gODVVWiNC
JkUxPEBYSUs0JyQtKkpZIVEzNkhOYDg1VVojMiVMVzkjK1ZYIUE3N0guKDo1
JVIjQkZNMA0KTTwwVEpLRCclLTpCWCFBNzdIKyg2Uz1ELE87QCYlPT5AWEFJ
NDcoLio6VSFRIzJKTjA8NFVKK0AmJT0+QA0KTVxBOy1YMFI9OlUhUSMySk4w
PDRVSitAJiU9PkBYQUk0NyguKjpVIVEjOklIVDFTLlQrRCYlPT5AWEFJNA0K
TTcoLio6VSFRIzJKTjA8NFVKK0AmJT0+QFJBOy1YMFI9OlUhUSMySk4wPDRV
SitAJiU9PkBYQUk0NyguKg0KTTpVIVEjOklKVDlSTS8iSk4wPDRVSitAJiU9
PkBYQUk0NyguKjpVIVEjMkpOMDU3VVhLXjVMN1wnOj9cYA0KIWAlIVENCmAN
CmVuZA0K
---876492614-1047046357-874747376=:202--