Ingo Molnar wrote:
* George Anzinger <george@xxxxxxxxxx> wrote:I was rather thinking of only comparing a cup's TSC with the SAME cup's TSC. We only use the TSC to bridge from the latest update of the the clock to now and when we update the clock we capture (i.e. update this cup's last TSC value) the TSC on that CPU. If we also capture the system time then we have a set of (last TSC, sys clock) for each CPU. If we further, keep 64-bits of TSC, we don't really require each cpu to update the clock very often, or, we could force such and update from time to time as needed. This would not require the TSC to be synced at all and even if they had different rates we could add that to the per cpu data and cover that too.
The TSC is such a fast and, usually, accurate answer, I think it deserves a little effort to save it. With your new clock code I think we could use per cpu TSC counters, read the full 64 bits and, in real corner cases, even per cpu conversion "constants" and solve this problem.
the problem is, this is the same issue as 'boot-time TSC syncing', but in disguise: to get any 'per CPU TSC offset' you need to do exactly the same type of careful all-CPUs-dance to ensure that the TSCs were sampled at around the same moment in time!
The box where i have these small TSC inconsistencies shows that it's the bootup synchronization of TSCs that failed to be 100% accurate. Even a 2 usecs error in synchronization can show up as a time-warp - regardless of whether we keep per-CPU TSC offsets or whether we clear these offsets back to 0. So it is not a solution to do another type of synchronization dance. The only solution is to fix the boot-time synchronization (where the hardware keeps TSCs synchronized all the time), or to switch TSCs off where this is not possible.
What this does require is that we do a really good job of figuring the TSC multiplier, something we don't do at all well today (rc5-rt5 on my box is fast by ~15 sec/ day). We might also want to verify that we are not letting monotonic time be other than monotonic :)
As you might suspect, I have some ideas about how to do a much better job of figuring out the TSC multiplier.