[PATCH 3/6] rcu: Exit RCU extended QS on preemption in irq exit

[Frederic Weisbecker]

Syscalls and exceptions low level handlers must be contained
between rcu_user_exit() and rcu_user_enter(). But irq high
level handlers are already protected inside rcu_irq_enter/exit()
so we don't need to call rcu_user_exit() on irq entry. But if
we preempt the running task in the end of the interrupt, we need
to exit the RCU userspace mode because schedule() itself and the
next task are in the kernel and may contain RCU read side critical
section..

To solve this, call rcu_user_exit() from irq exit preemption points:
- If we interrupted kernelspace, we call preempt_schedule_irq() and
rcu_user_exit() is called from it.

- If we interrupted userspace, we usually call schedule() directly.
Now we will rely on a new API schedule_user() which calls rcu_user_exit()
before calling schedule().

The low level irq handler may need to call rcu_user_enter() if we
resume userspace after the task is woken up from its preemption.

Signed-off-by: Frederic Weisbecker <fweisbec@xxxxxxxxx>
Cc: Alessio Igor Bogani <abogani@xxxxxxxxxx>
Cc: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
Cc: Avi Kivity <avi@xxxxxxxxxx>
Cc: Chris Metcalf <cmetcalf@xxxxxxxxxx>
Cc: Christoph Lameter <cl@xxxxxxxxx>
Cc: Geoff Levand <geoff@xxxxxxxxxxxxx>
Cc: Gilad Ben Yossef <gilad@xxxxxxxxxxxxx>
Cc: Hakan Akkan <hakanakkan@xxxxxxxxx>
Cc: H. Peter Anvin <hpa@xxxxxxxxx>
Cc: Ingo Molnar <mingo@xxxxxxxxxx>
Cc: Josh Triplett <josh@xxxxxxxxxxxxxxxx>
Cc: Kevin Hilman <khilman@xxxxxx>
Cc: Max Krasnyansky <maxk@xxxxxxxxxxxx>
Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
Cc: Stephen Hemminger <shemminger@xxxxxxxxxx>
Cc: Steven Rostedt <rostedt@xxxxxxxxxxx>
Cc: Sven-Thorsten Dietrich <thebigcorporation@xxxxxxxxx>
Cc: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
---
 kernel/sched/core.c |    7 +++++++
 1 files changed, 7 insertions(+), 0 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index d5594a4..1630532 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3332,6 +3332,12 @@ asmlinkage void __sched schedule(void)
 }
 EXPORT_SYMBOL(schedule);
 
+asmlinkage void __sched schedule_user(void)
+{
+	rcu_user_exit();
+	schedule();
+}
+
 /**
  * schedule_preempt_disabled - called with preemption disabled
  *
@@ -3433,6 +3439,7 @@ asmlinkage void __sched preempt_schedule_irq(void)
 	/* Catch callers which need to be fixed */
 	BUG_ON(ti->preempt_count || !irqs_disabled());
 
+	rcu_user_exit();
 	do {
 		add_preempt_count(PREEMPT_ACTIVE);
 		local_irq_enable();
-- 
1.7.5.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/