From: john stultz
Date: Tue Oct 25 2005 - 13:17:14 EST
On Tue, 2005-10-25 at 17:44 +0200, Ingo Molnar wrote:
> i found one source of timekeeping bugs on SMP boxes, it's the
> non-monotonicity of the TSC:
> ... time warped from 1270809453 to 1270808096.
> ... MTSC warped from 0000000a731a8c3c  to 0000000a731a899c .
> ... MTSC warped from 0000000a7c93baec  to 0000000a7c93b7a8 .
> ... MTSC warped from 0000000a881d6afc  to 0000000a881d67d0 .
> ... MTSC warped from 0000000a924217a0  to 0000000a924216ac .
> ... MTSC warped from 0000000a9c592788  to 0000000a9c59232c .
> ... MTSC warped from 0000000aa7aa95c8  to 0000000aa7aa9338 .
> ... MTSC warped from 0000000b33206d60  to 0000000b33206a48 .
> ... time warped from 26699635824 to 26699633144.
> ... MTSC warped from 00000013f379cb88  to 00000013f379c7e0 .
> ... MTSC warped from 0000001413df8660  to 0000001413df8200 .
> ... MTSC warped from 00000014194f5360  to 00000014194f51b0 .
> ... time warped from 60775269225 to 60775266727.
> the number in square brackets is the CPU#. I.e. CPUs on this 4-CPU box
> have small TSC differences, which ends up leaking into the generic TOD
> code, causing real time warps, which causes ktimer weirdnesses (timers
> failed to expire, etc.).
> (the above output tracks TSC results globally, under a spinlock. It also
> detects time-warps that propagate into the monotonic clock output.)
> unfortunately, there's no easy solution for this. We could make
> cycle_last per-CPU, but that again brings up the question of how to set
> up the per-CPU 'TSC offset' values - those would need similar technique
> that the current clear-all-TSCs-on-all-CPUs code does - which as we can
> see failed ...
Indeed. This is a nasty issue can affect a number of different systems.
The best solution in my mind is to utilize alternative clocksources when
necessary (one of the main reasons for creating the flexible clocksource
interface: so we can easily use something else).
In my patches, I have a function mark_tsc_unstable(), when called will
drop the tsc's rating value and will cause another clocksource to be
chosen (as long as one is available). Right now we call it when we know
the TSC is going to have problems. But maybe we should be more dynamic
in our detection.
Do you have any details about the hardware? Are the TSCs not being
synced well enough, or are they falling out of sync? i386 is a bit more
aggressive about using the TSC in SMP systems, where x86-64 has more
conditionals. Maybe some of the x86-64 logic should be moved to i386 as
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/