2.6.20, Java, unresponsive userspace

From: Alistair John Strachan
Date: Sat Mar 03 2007 - 23:07:50 EST


Hi,

I've had some problems with 2.6.20 (but also with earlier kernels, like
Debian's 2.6.18) where one of the users on our system, running a Java6
application, is somehow able to make the userspace on the machine totally
unresponsive, denying all remote _and_ local access to the machine.

- The AMD64 port of Linux and Java are in use
- The process runs as an unprivileged user
- The OOM killer doesn't kick in
- No swap is enabled on the machine
- I'm told that the program doesn't fork bomb
(but I've not got a reproducible test case yet)
- The program is able to deny service even to a getty login.

If I SAK the machine, init respawns getty and everything is fine again.

alistair@clarent:~$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
max nice (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) unlimited
max locked memory (kbytes, -l) 1000
max memory size (kbytes, -m) 1000
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) unlimited
max rt priority (-r) 0
stack size (kbytes, -s) 2048
cpu time (seconds, -t) unlimited
max user processes (-u) 100
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

Any ideas with how to proceed for debugging this issue?

--
Cheers,
Alistair.

Final year Computer Science undergraduate.
1F2 55 South Clerk Street, Edinburgh, UK.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/