RE: Mainline kernel OLTP performance update

[Styner, Douglas W]

Our analysis of the interrupts shows that rescheduling interrupts are up 2.2x from 2.6.24.2 --> 2.6.30-rc3.  Qla2xxx interrupts are roughly the same.

Doug

>-----Original Message-----
>From: Andrew Morton [mailto:akpm@xxxxxxxxxxxxxxxxxxxx]
>Sent: Wednesday, April 29, 2009 12:30 AM
>To: Styner, Douglas W
>Cc: linux-kernel@xxxxxxxxxxxxxxx; Tripathi, Sharad C;
>arjan@xxxxxxxxxxxxxxx; Wilcox, Matthew R; Kleen, Andi; Siddha, Suresh B;
>Ma, Chinang; Wang, Peter Xihong; Nueckel, Hubert; Recalde, Luis F; Nelson,
>Doug; Cheng, Wu-sun; Prickett, Terry O; Shunmuganathan, Rajalakshmi; Garg,
>Anil K; Chilukuri, Harita; chris.mason@xxxxxxxxxx
>Subject: Re: Mainline kernel OLTP performance update
>
>On Tue, 28 Apr 2009 10:08:22 -0700 "Styner, Douglas W"
><douglas.w.styner@xxxxxxxxx> wrote:
>
>> Summary: Measured the mainline kernel from kernel.org (2.6.30-rc3).
>>
>> The regression for 2.6.30-rc3 against the baseline, 2.6.24.2 is 1.91%.
>Oprofile reports 71.1626% user, 28.8295% system.
>>
>> Linux OLTP Performance summary
>> Kernel#            Speedup(x)   Intr/s  CtxSw/s us%     sys%    idle%
>iowait%
>> 2.6.24.2                1.000   22106   43709   75      24      0       0
>> 2.6.30-rc3              0.981   30645   43027   75      25      0       0
>
>The main difference there is the interrupt frequency.  Do we know which
>interrupt source(s) caused this?
>
>> Server configurations:
>> Intel Xeon Quad-core 2.0GHz  2 cpus/8 cores/8 threads
>> 64GB memory, 3 qle2462 FC HBA, 450 spindles (30 logical units)
>>
>>
>> ======oprofile CPU_CLK_UNHALTED for top 30 functions
>> Cycles% 2.6.24.2                   Cycles% 2.6.30-rc3
>> 74.8578 <database>                 69.1925 <database>
>
>ouch, that's a large drop in userspace CPU occupancy.  It seems
>inconsistent with the 1.91% above.
>
>> 1.0500 qla24xx_start_scsi          1.1314 qla24xx_intr_handler
>> 0.8089 schedule                    1.0031 qla24xx_start_scsi
>> 0.5864 kmem_cache_alloc            0.8476 __schedule
>> 0.4989 __blockdev_direct_IO        0.6532 kmem_cache_alloc
>> 0.4357 __sigsetjmp                 0.4490 __blockdev_direct_IO
>> 0.4152 copy_user_generic_string    0.4199 __sigsetjmp
>> 0.3953 qla24xx_intr_handler        0.3946 __switch_to
>> 0.3850 memcpy                      0.3538 __list_add
>> 0.3596 scsi_request_fn             0.3499 task_rq_lock
>> 0.3188 __switch_to                 0.3402 scsi_request_fn
>> 0.2889 lock_timer_base             0.3382 rb_get_reader_page
>> 0.2750 memmove                     0.3363 copy_user_generic_string
>> 0.2519 task_rq_lock                0.3324 aio_complete
>> 0.2474 aio_complete                0.3110 try_to_wake_up
>> 0.2460 scsi_alloc_sgtable          0.2877 ring_buffer_consume
>> 0.2445 generic_make_request        0.2683 mod_timer
>> 0.2263 qla2x00_process_completed_re0.2605 qla2x00_process_completed_re
>> 0.2118 blk_queue_end_tag           0.2566 blk_queue_end_tag
>> 0.2085 dio_bio_complete            0.2566 generic_make_request
>> 0.2021 e1000_xmit_frame            0.2547 tcp_sendmsg
>> 0.2006 __end_that_request_first    0.2372 lock_timer_base
>> 0.1954 generic_file_aio_read       0.2333 memmove
>> 0.1949 kfree                       0.2294 memset_c
>> 0.1915 tcp_sendmsg                 0.2080 mempool_free
>> 0.1901 try_to_wake_up              0.2022 generic_file_aio_read
>> 0.1895 kref_get                    0.1963 scsi_device_unbusy
>> 0.1864 __mod_timer                 0.1963 plist_del
>> 0.1863 thread_return               0.1944 dequeue_rt_stack
>> 0.1854 math_state_restore          0.1924 e1000_xmit_frame

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/