Re: [RFC][PATCH] ftrace: Use schedule_on_each_cpu() as a heavysynchronize_sched()

[Steven Rostedt]

On Wed, 2013-05-29 at 09:52 +0200, Peter Zijlstra wrote:

> Just to be clear, its the idle part that's a problem, right?

Actually, it's the userspace part that's triggered the bugs.


>  Being stuck
> in userspace isn't a problem since if that CPU is in userspace its
> certainly not got a reference to whatever list entry we're removing.

The issue is coming out of and going into userspace. For example, we
trace the function user_exit(), which is the function that tells RCU
it's coming back into the kernel. At this point, RCU doesn't keep track
of the preempt disable/enable that's done there.

> 
> Now when the CPU really is idle, its obviously not using tracing either;
> so only the gray area where RCU thinks we're idle but we're not actually
> idle is a problem?

Right. It's going into and coming out of idle or userspace that's the
issue. There's a very small window where RCU is blind to this.

> 
> Is there something a little smarter we can do? Could we use
> on_each_cpu_cond() with a function that checks if the CPU really is
> fully idle?

One thing I thought about doing was both a synchronize_sched() and then
a msleep(5). As a single function trace should never last 5
milliseconds. But to me, that's just hacky, and this is the real
solution.

> 
> > To implement the RCU synchronization, instead of using
> > synchronize_sched() the use of schedule_on_each_cpu() is performed. This
> > means that when a dynamically allocated ftrace_ops, or a control ops is
> > being unregistered, all CPUs must be touched and execute a ftrace_sync()
> > stub function via the work queues. This will rip CPUs out from idle or
> > in dynamic tick mode. This only happens when a user disables perf
> > function tracing or other dynamically allocated function tracers, but it
> > allows us to continue to debug RCU and context tracking with function
> > tracing.
> 
> I don't suppose there's anything perf can do to about this right? Since
> its all on user demand we're kinda stuck with dynamic memory.

Right, and perf isn't the only one. SystemTap, lttng, and even other
parts of ftrace can have this problem. Anyone that does a dynamic
allocation needs a full synchronization.

-- Steve


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/