Re: Bad SSD performance with recent kernels

[Herbert Poetzl]

On Sat, Jan 28, 2012 at 08:51:08PM +0800, Wu Fengguang wrote:
> Hi Herbert,

> On Fri, Jan 27, 2012 at 07:00:34AM +0100, Herbert Poetzl wrote:

>> Dear Community!

>> Recently I decided to update the kernel on a Dell Laptop to
>> a more recent version than 2.6.38.x, but experienced bad
>> I/O performance with the new kernel, so I started to dig
>> a little deeper and ended up with the following test:

>>   1) download and extract kernel (on 2.6.38.8)
>>   2) make defconfig
>>   3) make localmodconfig
>>   4) make
>>   5) make modules_install install

>> Then I booted each kernel in single user and ran the following
>> test script:

>>   echo noop >/sys/class/block/sda/queue/scheduler
>>   for n in 1 2 3; do sync; echo $n > /proc/sys/vm/drop_caches; done
>>   /usr/bin/time -f "real = %e, user = %U, sys = %S, %P cpu" \
>>         ionice -c0 nice -20 \
>>         dd if=/dev/sda of=/dev/null bs=1M count=20480

>>   echo deadline >/sys/class/block/sda/queue/scheduler
>>   for n in 1 2 3; do sync; echo $n > /proc/sys/vm/drop_caches; done
>>   /usr/bin/time -f "real = %e, user = %U, sys = %S, %P cpu" \
>>         ionice -c0 nice -20 \
>>         dd if=/dev/sda of=/dev/null bs=1M count=20480

>>   echo cfq >/sys/class/block/sda/queue/scheduler
>>   for n in 1 2 3; do sync; echo $n > /proc/sys/vm/drop_caches; done
>>   /usr/bin/time -f "real = %e, user = %U, sys = %S, %P cpu" \
>>         ionice -c0 nice -20 \
>>         dd if=/dev/sda of=/dev/null bs=1M count=20480


>> note that the Laptop is a relatively modern Latitude E6400
>> with a Samsung 830 Series 256GB SSD

>> here are the surprising results:

>> @ linux 2.6.38.8
>>   248 MB/s  real = 86.74, user = 0.01, sys = 21.65, 24% cpu
>>   248 MB/s  real = 86.81, user = 0.02, sys = 21.75, 25% cpu
>>   251 MB/s  real = 85.63, user = 0.01, sys = 22.24, 25% cpu

>> @ linux 2.6.39.4
>>  49.0 MB/s  real = 438.79, user = 0.01, sys = 19.79, 4% cpu
>>  25.7 MB/s  real = 836.70, user = 0.02, sys = 18.39, 2% cpu
>>  27.7 MB/s  real = 776.53, user = 0.01, sys = 16.03, 2% cpu

>> @ linux 3.0.18
>>  48.9 MB/s  real = 439.07, user = 0.01, sys = 17.55, 4% cpu
>>  25.0 MB/s  real = 859.03, user = 0.01, sys = 16.97, 1% cpu
>>  49.8 MB/s  real = 431.61, user = 0.01, sys = 16.68, 3% cpu

>> @ linux 3.1.10
>>  54.0 MB/s  real = 398.23, user = 0.01, sys = 17.36, 4% cpu
>>  29.4 MB/s  real = 731.47, user = 0.01, sys = 17.14, 2% cpu
>>  25.0 MB/s  real = 859.35, user = 0.01, sys = 14.51, 1% cpu

>> @ linux 3.2.2
>>  45.8 MB/s  real = 468.85, user = 0.01, sys = 17.11, 3% cpu
>>  44.8 MB/s  real = 478.92, user = 0.01, sys = 17.02, 3% cpu
>>  45.0 MB/s  real = 476.91, user = 0.01, sys = 16.14, 3% cpu

> Thanks for the through tests! However I cannot reproduce the
> regressions, but see slightly better performance in 3.2:

> Linux lkp-nex04 3.2.0-rc7-shli+ #121 SMP Thu Jan 19 18:10:45 CST 2012 x86_64 x86_64 x86_64 GNU/Linux

> 21474836480 bytes (21 GB) copied, 90.9307 s, 236 MB/s real = 91.03, user = 0.01, sys = 25.80, 28% cpu
> 21474836480 bytes (21 GB) copied, 90.8864 s, 236 MB/s real = 90.90, user = 0.01, sys = 25.54, 28% cpu
> 21474836480 bytes (21 GB) copied, 93.4684 s, 230 MB/s real = 93.47, user = 0.02, sys = 25.12, 26% cpu


> Linux lkp-nex04 2.6.38 #334 SMP Sat Jan 28 20:16:25 CST 2012 x86_64 GNU/Linux

> 21474836480 bytes (21 GB) copied, 96.2382 s, 223 MB/s real = 96.29, user = 0.01, sys = 57.64, 59% cpu
> 21474836480 bytes (21 GB) copied, 96.5796 s, 222 MB/s real = 96.60, user = 0.02, sys = 58.08, 60% cpu
> 21474836480 bytes (21 GB) copied, 95.486  s, 225 MB/s real = 95.51, user = 0.00, sys = 58.20, 60% cpu

> My test box has 64GB memory and a dozen 80GB INTEL
> SSDSA2M080G2GN SSD drives attached to

so we have intel vs samsung here ...

> 87:00.0 Serial Attached SCSI controller: LSI Logic / Symbios
> Logic SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 02)

... and sas vs sata

>> I have no idea why the I/O performance is that bad on any
>> kernel newer than 2.6.38.x, but I'm happy to test and/or
>> try various configurations as time permits ...

> Would you please create a filesystem and large file on sda and
> run the tests on the file? There was some performance bug on
> reading the raw /dev/sda device file..

will do tonight!

>> The detailed test results as well as the dmesg and config
>> of each kernel can be found here:
>> http://vserver.13thfloor.at/Stuff/SSD

> It would be better for such files be attached in future.
> (For one thing I get timeout when trying to open the link...)

attached the files as .txz

thanks,
Herbert

> Thanks,
> Fengguang

>> here some technical information:

>> http://www.dell.com/us/dfb/p/latitude-e6400/pd
>> SATA controller: Intel Corporation ICH9M/M-E SATA AHCI Controller (rev 03)

>> http://www.samsung.com/uk/consumer/memory-cards-hdd-odd/ssd/ssd/MZ-7PC256N/EU-spec
>> Model=SAMSUNG SSD 830 Series, FwRev=CXM03B1Q
>> supposed 520MB/s seq. read, 320MB/s seq. write, 75K IOPS


>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>> the body of a message to majordomo@xxxxxxxxxxxxxxx
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> Please read the FAQ at  http://www.tux.org/lkml/

Attachment: SSD.txz
Description: Binary data