lockup when accessing serial port (and fix)

From: Stas Sergeev
Date: Mon Feb 07 2005 - 15:33:24 EST


Hello.

When I am trying to use the serial
port, I am getting the machine lockup.
This started to happen sometime in a
2.6.9 era I think, and is not fixed
in the latest 2.6.11 pre's.
The bug looks trivial, unless I am
missing something. "port.lock" is
acquired in serial8250_interrupt()
and later in uart_start(), and that's
where the system is dead.
The calltrace is as follows:

serial8250_interrupt->!spin_lock(port.lock)!->serial8250_handle_port->
receive_chars->tty_flip_buffer_push->flush_to_ldisc->n_tty_receive_buf->
uart_flush_chars->uart_start->!spin_lock(port.lock)!->deadlock:(

Attached is a quick works-for-me fix
that just releases the lock at one
point. Can someone please have a look
into it and see if the other ser drivers
needs the same (or another) treatment?
Well, I can understand that the legacy
ser ports may not be too popular these
days, but then it is a local DoS, so it
may still be worth having a look.
I'll very much appreciate if the serial
port is to work again. I am still using
it, really! :)

--- linux-2.6.10/drivers/serial/au1x00_uart.c 2004-12-31 10:16:15.000000000 +0300
+++ linux-2.6.10/drivers/serial/au1x00_uart.c 2005-02-07 15:42:42.000000000 +0300
@@ -320,7 +320,9 @@
ignore_char:
*status = serial_inp(up, UART_LSR);
} while ((*status & UART_LSR_DR) && (max_count-- > 0));
+ spin_unlock(&up->port.lock);
tty_flip_buffer_push(tty);
+ spin_lock(&up->port.lock);
}

static _INLINE_ void transmit_chars(struct uart_8250_port *up)
--- linux-2.6.10/drivers/serial/8250.c 2005-02-07 13:47:57.000000000 +0300
+++ linux-2.6.10/drivers/serial/8250.c 2005-02-07 15:43:47.000000000 +0300
@@ -1059,7 +1059,9 @@
ignore_char:
lsr = serial_inp(up, UART_LSR);
} while ((lsr & UART_LSR_DR) && (max_count-- > 0));
+ spin_unlock(&up->port.lock);
tty_flip_buffer_push(tty);
+ spin_lock(&up->port.lock);
*status = lsr;
}