Re: direct i/o problem with 2.4.21

From: Larry Auton (
Date: Tue Jun 17 2003 - 07:20:00 EST

Andrea Arcangeli wrote:
> On Mon, Jun 16, 2003 at 06:33:00PM -0400, Larry Auton wrote:
>>>Message-ID: <>
>>>Date: Sat, 14 Jun 2003 14:18:47 -0400
>>>From: Larry Auton <>
>>>Subject: direct i/o problem with 2.4.20 and 2.4.21rc7
>>>I have an application that requires direct i/o to thousands of files.
>>>On 2.4.19 the open's would eventually fail (at around 7200 files).
>>>On 2.4.20 and 2.4.21rc7 the machine hangs.
>>>Here's a sample program to do the deed:
>>> wget
>>> cc -o crashme crashme.c # compile it
>>> ./crashme 4000 # OK
>>> ./crashme 9999 # CRASH
>>>It's a little obfuscated to eliminate the need for root privileges to
>>>mess with rlimit. It simply opens a bunch of files with O_DIRECT and,
>>>when enough files are open, the system will hang.
>>>The system hangs when '/proc/slabinfo' reports that 'kiobuf' reaches
>>>just over 7230 active objects. I don't believe that this problem is
>>>specific to any particular file system as the failure occurs when
>>>using both ext2 and reiserfs.
>>>Larry Auton
>>The hang I reported on 2.4.21rc7 persists in the released version 2.4.21.
> can you try to reproduce with 2.4.21rc8aa1? (you can apply my full patch
> to 21 final too of course since rc8 is the same as final) The fix for
> this that also fixes the performance problem with rawio from the same
> file but multiple fd is already in tree since several months, we should
> push it into mainline ASAP. The basic idea of the patch that moves the
> bh allocation flood down to the slab to take advantage of a shared
> shrinkable cache was developed at intel AFIK.
> the single patch is this but I doubt it would apply cleanly as 90% of
> the other patches:
> so you can make a quick test with the full 2.4.21rc8aa1.gz applied.
> as for the hang, that's because the vm in mainline probably isn't
> capable of returning -ENOMEM from syscalls under a zone normal shortage
> (previously it wouldn't touch the vm side because it used vmalloc that
> returns -ENOMEM w/o entering the VM). With the vm in my tree you
> shouldn't experience hangs even w/o the fix for the kiobuf bh flood
> allocation patch applied.
> Andrea

Problem solved. I after applying patch

to the vanilla linux-2.4.21, both the "crashme" program and my genuine
application run as intended. The machine is stable. Here's a line from
/proc/slabinfo showing the small, well-managed kiobuf allocation:

kiobuf 168 168 68 3 3 1 : 168 168 3 0 0 : 252 126 : 756227 6 756230 0

This state reflects my application using direct i/o to over 8300 files.

Well done! Many thanks.

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to
More majordomo info at
Please read the FAQ at

This archive was generated by hypermail 2b29 : Mon Jun 23 2003 - 22:00:20 EST