RE: [PATCH 2.6.x] additional kernel event notifications

[Villacis, Juan]

Hi,

"John Levon" <levon@xxxxxxxxxxxxxxxxx> writes:
> > In some cases, a profiler can figure out information regarding
> > Dynamically Generate Code (DGC) with help from the generator of the
> > code, but in other cases it cannot.
> > 
> > In the case of Java jitted code, our userspace tools obtain
sufficient
> > information through JVMPI, when it is implemented by the JVM.
> 
> I would argue that any JVM that doesn't yet implement
> JVMPI_EVENT_COMPILED_METHOD_LOAD is broken - JVMPI has been around for
> a long time now.
> 
> I don't see why the kernel is the correct place to fix such lacking
> functionality.

There are several languages and tools besides Java which can jit code.
Perl6 (using interpreters like Parrot) and Python (using interpreters
like Psycho) are examples of this and it is not yet clear what
interfaces they will provide to allow for JVMPI like functionality.  We
would like to be able to profile applications using these interpreters
(as well as others).  If userspace interpreters, for whatever reason, do
not provide functionality for tracking jitted code, would it not be
useful to provide help in the kernel to allow profilers to be able to
track this? 

Although we understand your belief that a DGC generator that doesn't
provide hooks for determining the DGC information may not be optimal, we
would still like to be able to provide something useful to the users of
those environments.  Perhaps this philosophical discussion is better
handled offline per an earlier suggestion.

> > for DGC which does not have such userspace support, it is important
to
> > be able to spot and accurately attribute samples to DGC.  The 4
> > additional profiling hooks we proposed can be used for such
purposes.
> 
> Please be specific about which *actual* cases you're worried about,
and
> why they shouldn't be fixed in userspace.

>From what we can see in the Oprofile driver code, if there is no dcookie
for a corresponding EIP, the EIP information is discarded and the
"oprofile_stats.sample_lost_no_mapping" count is incremented (see
add_us_sample() in drivers/oprofile/buffer_sync.c).  Did we
misunderstand something?  Is a DENTRY (and a corresponding dcookie)
being created somewhere for DGC?

> > If the generator of DGC frees memory used for DGC that subsequently
gets
> > a loaded image (or reuses memory that may have once had an
executable
> > image), you can mis-attribute samples so that instead of attributing
the
> > samples to the DGC, you will attribute the samples to an image. The
> > dcookie mechanism will indicate information about an image, but
doesn't
> > help prevent mis-attribution of samples if DGC is intermixed with
images
> > that are loaded/unloaded in the same memory region.
> 
> Simply flush the sample buffer by echo 1 >/dev/oprofile/dump when you
> receive a COMPILED_METHOD_LOAD/UNLOAD that conflicts with a previous
> mapping.

As long as there exists a userspace API similar to the JVMPI for a
particular DGC generator, and as long as the profiling tool is using
that API, then yes, it seems like this method would work.  But this
isn't the case we are worried about.

-juan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/