Re: File system performance, hardware performance, ext3, 3ware RAID1,etc.

[Timothy Miller]

Thanks for your suggestions.  They have aided me in my tests... read on.

Willy Tarreau wrote:
> On Thu, Feb 12, 2004 at 06:32:31PM -0500, Timothy Miller wrote:
>  
>
> > For writes, iozone found an upper bound of about 10megs/sec, which is 
> > abysmal.  Typically, I'd expect writes to be faster (on a single drive) 
> > than reads, because once the write is sent, you can forget about it. 
> > You don't have to wait around for something to come back, and that 
> > latency for reads can hurt performance.  The OS can also buffer writes 
> > and reorder them in order to improve efficiency.
>
>
> It depends on the disk too. Lots of disks (specially IDE) are far slower
> on writes than they are on reads.

I wonder why that is.  What adds extra overhead to the writes?  Verify? 
   Given the lousy quality of today's hard drives, I can see why that 
might be necessary.

> > The 3ware has this write cache that you can turn on or off.  With it 
> > off, it ensures that writes make it to the disks in order.  With it on, 
> > it will reorder writes more efficiently.  However, I noticed that the 
> > performance only went up to about 16meg/sec with the cache ON.
>
>
> I don't think that the FS type has much importance once the cache is ON.

It might, but from my tests (see below), you're right.

> > What's the command?  How about this:
> >    time dd if=/dev/zero of=/dev/sga3 bs=1024 count=1024
>
>
> do it like this, but use higher values, particularly for bs which is only
> 1kB here. Using something like bs=65536 and count=4096 will give you a 256 MB
> file.

I mistyped.  I did "bs=1024k count=1024".

> > Will that do it?  Should I use an offset to avoid any kind of header or 
> > metadata?
>
>
> not needed. Just ensure that you write to the right partition, and better
> check twice.

Done.  Here are my results:

As I had mentioned, I did raw read performance by timing a dd from 
/dev/sda to /dev/null.  That demonstrated a throughput of 47 megs/sec 
which is pretty close to the benchmarks that I find in reviews.

I was able to perform the write performance test last night.  The swap 
partition was actually /dev/sda2.  I think / and /boot are sda1 and 
sda3, but I forget the order.  Either way, the swap partition is not 
worst case.  In my test, I wrote 1 gigabyte in 1 meg blocks in 73.522 
seconds which translates into 13.92 megs/sec.  That's terrible -- worse 
because the 3ware's write cache is ON.

According to Tom's Hardware, raw write throughput for the WD1200JB 
varies from 39500 k/sec at the outer tracks to 14200 k/sec at the inner 
tracks.  Something is seriously bottle-necking the performance through 
the 3ware.

For comparison, my wife's computer has the same model of drive as its 
primary IDE.  This is a single drive, and the box runs Windows XP Pro 
with 1GB of RAM.  The disk was defragmented this morning at 5AM, and I 
ran the test this morning at about 9AM.  I ran SiSoft SANDRA, and these 
are the results:

Buffered Read     73 mb/s
Seqential Read    37 mb/s
Random read        6 mb/s
Buffered write    60 mb/s
Sequential write  39 mb/s
Random write      12 mb/s

Assuming that the "buffered" speeds are being buffered by the OS, we'll 
ignore those.  I am therefore observing that the writes to a single 
drive are 3 to 4 times faster than they are through the RAID controller, 
even with the 3ware write cache ON.

Does that make any sense?


[Kernel version:  Most recent Red Hat update, 2.4.20-something.
 3ware 7000-2 with two 120GB WD drives in RAID 1 array.]

Thanks.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/