Re: [RFC PATCH V2 2/9] perf: Extend ABI to support post-processing monotonic raw conversion

[John Stultz]

On Mon, Feb 13, 2023 at 1:40 PM Liang, Kan <kan.liang@xxxxxxxxxxxxxxx> wrote:
> On 2023-02-13 2:37 p.m., John Stultz wrote:
> > On Mon, Feb 13, 2023 at 11:08 AM <kan.liang@xxxxxxxxxxxxxxx> wrote:
> >>
> >> From: Kan Liang <kan.liang@xxxxxxxxxxxxxxx>
> >>
> >> The monotonic raw clock is not affected by NTP/PTP correction. The
> >> calculation of the monotonic raw clock can be done in the
> >> post-processing, which can reduce the kernel overhead.
> >>
> >> Add hw_time in the struct perf_event_attr to tell the kernel dump the
> >> raw HW time to user space. The perf tool will calculate the HW time
> >> in post-processing.
> >> Currently, only supports the monotonic raw conversion.
> >> Only dump the raw HW time with PERF_RECORD_SAMPLE, because the accurate
> >> HW time can only be provided in a sample by HW. For other type of
> >> records, the user requested clock should be returned as usual. Nothing
> >> is changed.
> >>
> >> Add perf_event_mmap_page::cap_user_time_mono_raw ABI to dump the
> >> conversion information. The cap_user_time_mono_raw also indicates
> >> whether the monotonic raw conversion information is available.
> >> If yes, the clock monotonic raw can be calculated as
> >> mono_raw = base + ((cyc - last) * mult + nsec) >> shift
> >
> > Again, I appreciate you reworking and resending this series out, I
> > know it took some effort.
> >
> > But oof, I'd really like to make sure we're not exporting timekeeping
> > internals to userland.
> >
> > I think Thomas' suggestion of doing the timestamp conversion in
> > post-processing was more about interpolating collected system times
> > with the counter (tsc) values captured.
> >
>
> Thomas, could you please clarify your suggestion regarding "the relevant
> conversion information" provided by the kernel?
> https://lore.kernel.org/lkml/87ilgsgl5f.ffs@tglx/
>
> Is it only the interpolation information or the entire conversion
> information (Mult, shift etc.)?
>
> If it's only the interpolation information, the user space will be lack
> of information to handle all the cases. If I understand John's comments
> correctly, it could also bring some interpolation error which can only
> be addressed by the mult/shift conversion.

"Only" is maybe too strong a word. I think having the driver use
kernel timekeeping accessors to CLOCK_MONONOTONIC_RAW time with
counter values will minimize the error.

But again, it's not yet established that any interpolation error using
existing interfaces is great enough to be problematic here.

The interpoloation is pretty easy to do:

do {
    start= readtsc();
    clock_gett(CLOCK_MONOTONIC_RAW, &ts);
    end = readtsc();
    delta = end-start;
} while (delta  > THRESHOLD)   // make sure the reads were not preempted
mid = start + (delta +(delta/2))/2; //round-closest

and be able to get you a fairly close matching of TSC to
CLOCK_MONOTONIC_RAW value.

Once you have that mapping you can take a few samples and establish
the linear function.

But that will have some error, so quantifying that error helps
establish why being able to get an atomic mapping of TSC ->
CLOCK_MONOTONIC_RAW would help.

So I really don't think we need to expose the kernel internal values
to userland, but I'm willing to guess the atomic mapping (which the
driver will have access to, not userland) may be helpful for the fine
granularity you want in the trace.

thanks
-john