Re: FTRACE_WARN_ON((rec->flags & ~FTRACE_FL_MASK) == 0))

From: Dave Jones
Date: Wed Aug 28 2013 - 12:23:37 EST


On Wed, Aug 28, 2013 at 10:31:01AM -0400, Steven Rostedt wrote:
> Dave,
>
> BTW, is there a way to run trinity on a subset of syscalls. Basically,
> I would like to run it on just the perf code, and nothing else. I have
> a feeling that the bug you see is not caused by other operations
> happening (although it could be), but from just out of order perf
> calls. If I can reproduce it, I'll have a much better way of debugging
> this.

indeed. -c perf_open_event for eg. (You can also specify multiple -c's
if you want to thow in some others)

There's a few other options in there for narrowing things down.
Say you think it's an interaction between perf and and unknown other syscall
for eg, you can do -c perf_open_event -r5. This will pick 5 random
syscalls, and fuzz those plus perf. See scripts/find.sh for an example of
how I've used this in the past.

> Perhaps another thing you may think of adding to trinity (if it doesn't
> already exist), is a log of what it is doing. That is, to log somewhere
> the commands it writes, and that way, if something goes wrong, you have
> a clue to how it got there. Because this is one of those bugs that
> triggered before the code crashes, and the crash is just the symptom of
> what went wrong and does not give you much clue to how it happened.

It does have logging already, though for a bug that takes hours, or days to
hit, they can grow to unmanagable sizes, and there's a problem if we have
a situation like..

syscall A
<24 hours of boring syscalls>
syscall B
oops as a result of B's interaction with A.

Quite often just rerunning that last syscall that caused the oops/warn
isn't sufficient to trigger an issue. (Though it may be for this specific
bug that may not be the case..)

Vince has a variant of trinity focussed just on perf which also has some
neat replay/bisecting capabilities to narrow down test cases.
I think I might need to add something like that at some point.

Dave

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/