More XFS resource starvation?

From: J.H.
Date: Mon Nov 15 2010 - 13:30:40 EST


So apparently I'm having fun tripping over all kinds of bugs lately.
I've seen this a couple of times now on the box in question. Usually
happens after a few days, or after particularly heavy rsync traffic on
the box.

http://pastebin.osuosl.org/36014

Christoph seemed to think it's a memory exhaustion problem, so I've
included the /proc/meminfo and as you can see there's plenty of memory
around on the system.

Loads have, expectedly, climbed currently around 1250.05 but growing slowly.

Quick overview of the underlying storage:

xfs -> md (raid 0) -+--> P812 hardware raid6 (cciss driver)
|
+--> P812 hardware raid6 (cciss driver)

This is running on an HP DL380 G7.

I saw this both on an older 2.6.30.10-105.2.23.fc11.x86_64, and
currently on 2.6.34.7-61.fc13.x86_64 (both being Fedora stock kernels)

I have not seen this on a very similar DL380 G6, with the same storage
setup and it is currently running the 2.6.30 kernel from above.

Christoph suggest increasing the nr_request values for each of the
underlying devices, but this didn't seem to change anything
significantly on the system.

Anyone have any ideas on what's going on?

- John 'Warthog9' Hawley
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/