Re: printk badness with VMAP_STACK

From: Linus Torvalds
Date: Wed Oct 26 2016 - 19:34:47 EST


On Wed, Oct 26, 2016 at 3:55 PM, Laura Abbott <labbott@xxxxxxxxxx> wrote:
>
> I was playing around with overflowing stacks and I managed to generate a
> test
> case that hung the kernel with vmapped stacks. The test case is just
>
> static void noinline foo1(void)
> {
> pr_info("%p\n", (void *)current_stack_pointer());
> foo2();
> }
>
> where foo$n is the same function with the name changed. I'm super
> creative. I have a couple thousand of these for testing with the final
> one doing a WARN. The kernel eventually hangs in printk on logbuf_lock

So just to get this right - your test-case is intentionally doing that
mutually recursive thing with foo1/foo2 calling each other until they
run out of stack?

And yes, occasionally the stack will run out while in the middle of
"printk()", and then when we take a fault, we'll be screwed.

Note that we do *not* guarantee that "printk()" works in all contexts,
so it might not really be considered a bug. It's very much a "best
effort", but the scheduler and timekeeping, for example, uses
"printk_deferred()" exactly because one of the contexts where printk()
does *not* work is when you hold the rq lock.

And the reason for *that* is that printk() ends up relying on a few
different locks:

- logbuf_lock, obviously.
- console_sem for actual output
- cond_resched() requires rq->lock

And we do have some hacks on place - the recursive printk test
(logbuf_cpu, as you note) and oops_in_progress and that "zap_locks()".

But zap_locks only zaps logbuf_lock and console_sem, for example.

If you run out of stack somewhere in the middle of the scheduler when
the "cond_resched()" case of printk triggers, and we hold "rq->lock"
when the double fault occurs, the machine *will* be dead. It will
still try to print things out (thanks to that zap_locks thing), but
rq->lock will be wrong, and nothing will ever recover.

And it _sounds_ like that's the case you hit.

Basically, zap_locks and the other printk "try to at least print
things out" can only handle so much.

Linus