Re: MAP_SHARED bizarrely slow

[Ian Wienand]

On Wed, Oct 27, 2004 at 05:59:44PM +1000, David Gibson wrote:
> With the entire input matrices all copies of the zero page, cache
> performance, oddly enough, would have been rather better...

I agree with your analysis.  Just for my own fun playing with
profiling, I ran the two tests on a 1.5Ghz Itanium 2 (I compiled with
the Intel compiler lest you all laugh at how slow the "Itanic" is at
multiplying matrices).

ianw@baci:/usr/src/tmp/maptest$ /usr/bin/time ./mm-sharemmap
9.58user 0.02system 0:09.61elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+1540minor)pagefaults 0swaps
ianw@baci:/usr/src/tmp/maptest$ /usr/bin/time ./mm-sharemmap
9.47user 0.02system 0:09.50elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+1540minor)pagefaults 0swaps
ianw@baci:/usr/src/tmp/maptest$ /usr/bin/time ./mm-privmmap
8.63user 0.00system 0:08.63elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+1541minor)pagefaults 0swaps
ianw@baci:/usr/src/tmp/maptest$ /usr/bin/time ./mm-privmmap
8.63user 0.00system 0:08.63elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+1541minor)pagefaults 0swaps

Both close, certainly nothing like some other reports.  But as with
all benchmarking the devil is in the details; watching the cache
misses:

ianw@baci:/usr/src/tmp/maptest$ pfmon --events=L3_MISSES ./mm-privmmap
                    112678 L3_MISSES
ianw@baci:/usr/src/tmp/maptest$ pfmon --events=L3_MISSES ./mm-sharemmap
                  68600586 L3_MISSES

So it's no wonder shared mmap takes a little longer.  And indeed,
modifying your program to touch the memory in the privmmap call brings
the two into line.

Also looking at the kernel profiling via q-syscollect, the only
significant difference is private mapping spends about 19% of it's
time in clear_page, whilst shared spends around 29% of it's time in
clear_page.  All things being equal, you can thus expect to run ~10%
slower with shared, which is pretty close to what you actually see.

-i
ianw@xxxxxxxxxxxxxxxxxx
http://www.gelato.unsw.edu.au

Attachment: signature.asc
Description: Digital signature