Re: Very rapid _INODE_LEAK_ in 2.2.14|13...

From: Joe Cooper (joe@swelltech.com)
Date: Sat Jan 15 2000 - 18:31:02 EST


Chip Salzenberg wrote:

> > The rate of growth is about 30-60 per second depending on how much
> > data is flowing in and out on the network.
>
> My observed rate is more like one per minute, with occasional faster
> bursts. I haven't managed to correlate any particular activity to the
> bursts.

Count your blessings brother! What I wouldn't give for 1 per minute!
;-)

> To summarize what we know so far:
>
> * The leak affects 2.2.13 and 2.2.14, at least.
>
> * The leak occurs when the high-activity filesystem is either ext2
> or reiserfs, so it's probably in the fs-independent code.
>
> * The leak rate varies depending on system activity.
>
> Anything else?

Yeah, I've confirmed all of the above also. I can without doubt say
that squid under a polygraph load is a great way to reproduce the
problem.

I've compiled a pristine 2.2.14 kernel without Reiser (at AC's
suggestion) and found the problem to persist.

Mickulas suggested I umount the drive and see what happened to inodes to
find out just who was leaking. The inodes remained high, suggesting a
socket inodes leak, according to him.

I've looked at netstat -a and lsof output and found that they remain
stable after reaching a reasonable maximum (I originally thought lsof
kept climbing, but have been proven wrong by longer tests). file-nr
also remains stable (though extremely high under polygraph load...that
is to be expected however and is necessary).

squid under polygraph load is all about lots of opened net connections
(little ones) and lots of opened files (little ones), I'd say it's got
something to do with one or both of those things.

It's been hidden this long though (since at least 2.2.13), so it must be
pretty specific to the workload of a caching proxy under high load.

-- 
Joe Cooper <joe@swelltech.com>
http://www.swelltech.com

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Sun Jan 23 2000 - 21:00:13 EST