Re: [RFC][PATCH 6/8] mm: handle_speculative_fault()

[Minchan Kim]

At last your patient try makes the problem solve
although it's from not your patch series.

Thanks for very patient try and testing until now, Kame. :)
I learned lot of things from this thread.

Thanks, all.

On Wed, Jan 6, 2010 at 4:06 PM, KAMEZAWA Hiroyuki
<kamezawa.hiroyu@xxxxxxxxxxxxxx> wrote:
> On Tue, 5 Jan 2010 20:20:56 -0800 (PST)
> Linus Torvalds <torvalds@xxxxxxxxxxxxxxxxxxxx> wrote:
>
>>
>>
>> On Wed, 6 Jan 2010, KAMEZAWA Hiroyuki wrote:
>> > >
>> > > Of course, your other load with MADV_DONTNEED seems to be horrible, and
>> > > has some nasty spinlock issues, but that looks like a separate deal (I
>> > > assume that load is just very hard on the pgtable lock).
>> >
>> > It's zone->lock, I guess. My test program avoids pgtable lock problem.
>>
>> Yeah, I should have looked more at your callchain. That's nasty. Much
>> worse than the per-mm lock. I thought the page buffering would avoid the
>> zone lock becoming a huge problem, but clearly not in this case.
>>
> For my mental peace, I rewrote test program as
>
> Âwhile () {
> Â Â Â Âtouch memory
> Â Â Â Âbarrier
> Â Â Â Âmadvice DONTNEED all range by cpu 0
> Â Â Â Âbarrier
> Â}
> And serialize madivce().
>
> Then, zone->lock disappears and I don't see big difference with XADD rwsem and
> my tricky patch. I think I got reasonable result and fixing rwsem is the sane way.
>
> next target will be clear_page()? hehe.
> What catches my eyes is cost of memcg... (>_<
>
> Thank you all,
> -Kame
> ==
> [XADD rwsem]
> [root@bluextal memory]# Â/root/bin/perf stat -e page-faults,cache-misses --repeat 5 ./multi-fault-all 8
>
> ÂPerformance counter stats for './multi-fault-all 8' (5 runs):
>
>    33029186 Âpage-faults        Â( +-  0.146% )
>   Â348698659 Âcache-misses        ( +-  0.149% )
>
>  60.002876268 Âseconds time elapsed  ( +-  0.001% )
>
> # Samples: 815596419603
> #
> # Overhead     ÂCommand       Shared Object ÂSymbol
> # ........ Â............... Â........................ Â......
> #
> Â Â41.51% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] clear_page_c
> Â Â 9.08% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] down_read_trylock
> Â Â 6.23% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] up_read
> Â Â 6.17% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] __mem_cgroup_try_charg
> Â Â 4.76% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] handle_mm_fault
> Â Â 3.77% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] __mem_cgroup_commit_ch
> Â Â 3.62% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] __rmqueue
> Â Â 2.30% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] _raw_spin_lock
> Â Â 2.30% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] page_fault
> Â Â 2.12% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] mem_cgroup_charge_comm
> Â Â 2.05% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] bad_range
> Â Â 1.78% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] _raw_spin_lock_irq
> Â Â 1.53% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] lookup_page_cgroup
> Â Â 1.44% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] __mem_cgroup_uncharge_
>   1.41% Âmulti-fault-all Â./multi-fault-all     [.] worker
> Â Â 1.30% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] get_page_from_freelist
> Â Â 1.06% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] page_remove_rmap
>
>
>
> [async page fault]
> [root@bluextal memory]# Â/root/bin/perf stat -e page-faults,cache-misses --repeat 5 ./multi-fault-all 8
>
> ÂPerformance counter stats for './multi-fault-all 8' (5 runs):
>
>    33345089 Âpage-faults        Â( +-  0.555% )
>   Â357660074 Âcache-misses        ( +-  1.438% )
>
>  60.003711279 Âseconds time elapsed  ( +-  0.002% )
>
>
> Â Â40.94% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] clear_page_c
> Â Â 6.96% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] vma_put
> Â Â 6.82% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] page_add_new_anon_rmap
> Â Â 5.86% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] __mem_cgroup_try_charg
> Â Â 4.40% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] __rmqueue
> Â Â 4.14% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] find_vma_speculative
> Â Â 3.97% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] handle_mm_fault
> Â Â 3.52% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] _raw_spin_lock
> Â Â 3.46% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] __mem_cgroup_commit_ch
> Â Â 2.23% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] bad_range
> Â Â 2.16% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] mem_cgroup_charge_comm
> Â Â 1.96% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] _raw_spin_lock_irq
> Â Â 1.75% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] mem_cgroup_add_lru_lis
> Â Â 1.73% Âmulti-fault-all Â[kernel] Â Â Â Â Â Â Â Â Â[k] page_fault
>



-- 
Kind regards,
Minchan Kim
N‹§²æìr¸›yúèšØb²X¬¶ÇvØ^–)Þ{.nÇ+‰·¥Š{±‘êçzX§¶›¡Ü}©ž²ÆzÚ&j:+v‰¨¾«‘êçzZ+€Ê+zf£¢·hšˆ§~†­†Ûiÿûàz¹®w¥¢¸?™¨è­Ú&¢)ßf”ù^jÇy§m…á@A«a¶Úÿ0¶ìh®å’i