Hi Ted, thanks for the reply...
> *Something* is probably chewing up a lot of memory; this could either be
> the result of a programming bug (some daemon as a memory leak, and is
> gradually growing in size until the system gets desperate), or some kind
> of external attack on your system by a hacker.
This never happened before I upgraded to MySQL 3.22.30 and Apache 1.3.11
several days ago... hmm.
It seems to be a sudden thing. In fact yesterday I found 3 mysqld's each
eating 25% of the CPU, and I had 80MB swap usage. I waited 10 minutes and
they were still going -- I killed them and MySQL continued to function fine.
I'll try 3.22.32...
Thanks again
Simon Garner
----- Original Message -----
From: "Theodore Y. Ts'o" <tytso@MIT.EDU>
To: "Simon Garner" <sgarner@expio.co.nz>
Sent: Thursday, February 17, 2000 3:21 AM
Subject: Re: VM: killing process
> From: "Simon Garner" <sgarner@expio.co.nz>
> Date: Wed, 16 Feb 2000 17:46:29 +1300
>
> Can someone tell me what the following means: (from syslog)
>
> Feb 16 16:53:40 hyperion kernel: VM: killing process httpd
> Feb 16 16:53:50 hyperion kernel: VM: killing process pop3d
>
> Does VM stand for Virtual Memory? Am I out of RAM/swap?
>
> The machine is running Red Hat 6.0 (kernel 2.2.14), and has 128MB (ECC
> PC-100) RAM, and a 128MB swap partition. Normally I have anywhere from
3 to
> 20MB RAM free, and about 2MB of swap used (126MB free).
>
> Yes, this means that your system became chronically out of memory, and
> so the kernel was desparately trying to kill processes to get the memory
> utilization down to something where the rest of the processes could try
> to get work done.
>
> This is a highly abnormal situation and given what you reported about
> your normal amounts of memory and swamp free, it doesn't sound like your
> box is underprovisioned. It does sound like something seriously wrong
> is going on, though.
>
> *Something* is probably chewing up a lot of memory; this could either be
> the result of a programming bug (some daemon as a memory leak, and is
> gradually growing in size until the system gets desperate), or some kind
> of external attack on your system by a hacker.
>
> The first case is easier to detect. Something which I like to do when I
> run such heavy servers it to run a process out of cron every 10-15
> minutes, which appends a line to a log file. The shell script would be
> customized for each server, but the typical information it would have
> would be:
>
> * time
> * load (instaneous 5 minute load as reported by uptime)
> * free memory
> * swap space used
> * number of connections (for each service: httpd, pop, nntpd, etc.)
> * disk space usage
>
> etc. This gives me a 24-hour perspective of how much resources the
> server is needed, and it's a great way to gather the data need to
> justify to management that you need a bigger news server every 18
> months. :-)
>
> In your case, if you see the free memory and swap space declining, the
> pattern will be very useful to determine what's going on. If it's
> steady and then suddly the free memory goes low and swap space goes up,
> that tells you one thing. It's either a very interesting bug where some
> process suddenly tries to grab lots of memory, or it may be an external
> attack. If the free memory declines and the swap space numbers
> gradually icnrease over time, then you may have a memory leak bug in one
> of your daemons. Once you know this, hopefully you'll know when to
> look for the problem using "ps".
>
> I hope this helps, and wasn't too obvious (I don't know how much
> experience you have)....
>
> - Ted
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Wed Feb 23 2000 - 21:00:16 EST