klogd dies, possible emit_log_char() vs do_syslog(..READ..) race.

From: Peter Chubb
Date: Mon Jul 25 2011 - 21:09:10 EST


Hi,

Since mid May, my klogd has been turning into a 100% CPU hog at
regular intervals. Turns out that sometimes reads from /proc/kmsg
return with zero bytes read --- pointing to a race between
do_syslog(SYSLOG_ACTION_READ...) and emit_log_char().
(In Debian, klogd is invoked with a dd frontend, as
dd if=/proc/kmsg of=/var/run/klogd bs=1
where /var/run/klogd is a named pipe; this dd dies when it reads
zero bytes).

There's history in the Debian bug tracker,
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=308580

I tried an obvious and very ugly band-aid (appended) and the
problem went away.

It seems to me that log_buf_end must be wrapping and be a little
less than log_buf_start, then when emit_log_char() updates
log_buf_end it can become equal, causing do_syslog() to return zero
bytes read. This'd have to happen between the
wait_event_interruptible() and the taking of logbuf_lock in
do_syslog().

I can't see how this can happen though. emit_log_char() should
reset log_start so it can not happen.



diff --git a/kernel/printk.c b/kernel/printk.c
index 37dff34..0e44138 100644
--- a/kernel/printk.c
+++ b/kernel/printk.c
@@ -358,6 +358,7 @@ int do_syslog(int type, char __user *buf, int len, bool from_file)
error = -EFAULT;
goto out;
}
+ again:
error = wait_event_interruptible(log_wait,
(log_start - log_end));
if (error)
@@ -377,6 +378,8 @@ int do_syslog(int type, char __user *buf, int len, bool from_file)
spin_unlock_irq(&logbuf_lock);
if (!error)
error = i;
+ if (error == 0)
+ goto again;
break;
/* Read/clear last kernel messages */
case SYSLOG_ACTION_READ_CLEAR:


--
Dr Peter Chubb http://www.gelato.unsw.edu.au peterc AT gelato.unsw.edu.au
http://www.ertos.nicta.com.au ERTOS within National ICT Australia
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/