Re: 2.6.1 IO lockup on SMP systems

[Mike Fedyk]

Sergey S. Kostyliov wrote:
> Hello Andrew,
>
> Now this happens for the third time.
>
>
> > > > I've just reproduced this lockup with 2.6.3.
> > > >
> > > >
> > > > > You may need a serial console to be able to capture all the output.
> > > > >
> > > > > Also, it would be useful to know what sort of load the machines are
> > > > > under, and what filesystems are in use.
> > > >
> > > > The machine is a http server. The main applications are:
> > > > 1) apache 1.3 which serves php pages (mod_php):
> > > > 	 15.3 requests/sec - 111.9 kB/second - 7.3 kB/request
> > > > 	 54 requests currently being processed, 19 idle servers
> > > > 2) mysql:
> > > > 	Threads: 19  Questions: 26922012  Slow queries: 9799  Opens: 64980
> > > > 	Flush tables: 1  Open tables: 630  Queries per second avg: 143.547
> > > >
> > > > This is an IO bound machine in general. All filesystems are reiserfs.
> > > >
> > > > Here is a sysrq-T output obtained from a locked box via serail console:
> > >
> > > OK, so everything is stuck trying to allocate memory.  Perhaps you ran out
> > > of swapspace, or some process has gone berzerk allocating memory.
>
>
> The memory exhaustion is indeed possible for this box. I'll double check
> ulimit and /etc/security/limits.conf stuff. The only thing which worries
> me that this box had been running for months without any problems with
> 2.4.23aa1.
>
> I have added another 2Gb to swap space (hope this give enough time
> to find the memory hungry process(es)).

Also check how much memory is being used for slab in /proc/meminfo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/