Re: [Xen-devel] Re: [PATCH 3/5] x86/pvclock: add vsyscall implementation

From: Avi Kivity
Date: Sun Nov 01 2009 - 04:33:38 EST

On 10/29/2009 05:55 PM, Dan Magenheimer wrote:
From: Avi Kivity [mailto:avi@xxxxxxxxxx]
Sent: Thursday, October 29, 2009 9:07 AM
To: Dan Magenheimer
Cc: Jeremy Fitzhardinge; Glauber Costa; Jeremy Fitzhardinge; Kurt
Hackel; the arch/x86 maintainers; Linux Kernel Mailing List;
Glauber de
Oliveira Costa; Xen-devel; Keir Fraser; Zach Brown; Chris Mason; Ingo
Subject: Re: [Xen-devel] Re: [PATCH 3/5] x86/pvclock: add vsyscall

On 10/29/2009 04:46 PM, Dan Magenheimer wrote:
No, the apps I'm familiar with (a DB and a JVM) need a timestamp
not a monotonic counter. The timestamps must be relatively
accurate (e.g. we've been talking about gettimeofday generically,
but these apps would use clock_gettime for nsec resolution),
monotonically increasing, and work properly across a VM
migration. The timestamps are taken up to a 100K/sec or
more so the apps need to ensure they are using the fastest
mechanism available that meets those requirements.
Out of interest, do you know (and can you relate) why those apps need
100k/sec monotonically increasing timestamps?
I don't have any public data available for this DB usage, but basically
assume it is measuring transactions at a very high throughput, some
of which are to a memory-resident portion of the DB. Anecdotally,
I'm told the difference between non-vsyscall gettimeofday
and native rdtsc (on a machine with Invariant TSC support) can
affect overall DB performance by as much as 10-20%.

Sorry, that doesn't explain anything.

I did find the following public link for the JVM:

Search for "flight recorder". This feature is intended to
be enabled all the time, but with non-vsyscall gettimeofday
the performance impact is unacceptably high, so they are using
rdtscp instead (on those machines where it is available). With
rdtscp, the performance impact is not measureable.

Though the processor/server vendors have finally fixed the
"unsynced TSC" problem on recent x86 platforms, thus allowing
enterprise software to obtain timestamps at rdtsc performance,
the problem comes back all over again with virtualization
because of migration. Jeremy's vsyscall+pvclock is a great
solution if the app can ensure that it is present; if not,
the apps will instead continue to use rdtsc as even emulated
rdtsc is 2-3x faster than non-vsyscall gettimeofday.

Does that help?

For profiling work fast timestamping is of course great, but surely there is no monotonicity requirement?

I don't think we'll be able to provide monotonicity with vsyscall on tsc-broken hosts, so we'll be limited to correcting the tsc frequency after migration for good-tsc hosts.

error compiling committee.c: too many arguments to function

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at
Please read the FAQ at