Re: fsync on large files

Malcolm Beattie (mbeattie@sable.ox.ac.uk)
Mon, 1 Mar 1999 12:29:14 +0000 (GMT)


hagopiar@vuser.vu.union.edu writes:
> On Thu, 18 Feb 1999, Richard Jones wrote:
>
> > Extrapolating using an *extremely dodgy* linear
> > approximation gives the following estimates:
> >
> > 50% 40m (2400s)
> > 90% 66m40s (4000s)
>
> That's too nice, or you have faster HDs than I did... I had a 30GB RAID-5
> that when at about 80-90% full took over an hour to fsck.

There's been a thread on fsck times recently on the server-linux
mailing list. Extracting from a message I sent there:

Hardware: 350 MHz Pentium II PC, 512 MB RAM, BT958D SCSI adapter.
Sun D1000 disk array with 6 x 9 GB 10000 RPM disks.
Software: Linux 2.0.36 + latest RAID patch.
Filesystem configured as a single 43 GB RAID5 ext filesystem with
4k blocks and 64k RAID5 chunk-size.

I created 25 subdirectories on the filesystem and in each untarred
four copies of the Linux 2.2.1 source tree (each is ~4000 files
totalling 63 MB untarred).

fsck took 8 minutes.

Then I added 100 subdirectories in each of those subdirectories and
into each of those directories put five 1MB files. (The server is
actually going to be an IMAP server and this mimics half-load quite
well). The result is 18 GB used on the filesystem.

fsck took 10.5 minutes.

Then I added another 100 subdirectories in each of the 25 directories
and put another five 1MB files in each of those. The result is 30 GB
used on the filesystem.

fsck took 13 minutes.

That's 70% full. I rather doubt that going up to 80% would take it
anywhere near one hour. The main requirements for an adequately fast
e2fsck on filesystems of this size appears to be 4k blocksize and
nice fast random I/O seek times, which are also the attributes which
make the array perform nicely in real life in many situations.

--Malcolm

-- 
Malcolm Beattie <mbeattie@sable.ox.ac.uk>
Unix Systems Programmer
Oxford University Computing Services

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/