Re: Problem with 2.6 kernel and lots of I/O

From: Kyle Moffett
Date: Mon Jun 20 2005 - 18:41:17 EST

On Jun 20, 2005, at 18:19:19, Roy Keene wrote:
On Mon, 6 Jun 2005, Kyle Moffett wrote:
IIRC, because of the way the loopback delivers packets from the
same context as they are sent, it is possible (and quite easy)
to either deadlock or peg the CPU and make everything hang and
be unuseable. DRBD likewise used to have problems with testing
over the loopback until they added a special configuration
option to be extra careful and yield CPU.

Actually, the problem I have isn't specific to the using it over
the local device. Quite often I have the problem where the
secondary node goes down and comes back up after some time and
needs to be resyncd. This is done on the master (raid1_resync) by
hot-removing /dev/nbd1 and then hot-adding it back.

No, see, when you hot-add /dev/nbd1, the kernel md resync thread
begins processing the resync. The resync operation on two nbds
1) Send data request packet from nbd0
2) Wait for response
3) Send data packet to nbd1
4) Wait for response
5) Repeat until done

On a normal net device, the "Send data request packet" causes the
system to drop the packet on the wire and go away to do other stuff
for a while, whereas on the loopback, it can schedule immediately
to the process receiving the packet, which is the kernel itself.
The kernel then processes the packet and returns the result, over
the loopback. It then sends the response to the other server over
a real net connection. During most of this time, the kernel is
taking big locks and turning interrupts off and on and such, causing
massive hangs until resync finishes. Since you mentioned bad write
performance with your RAID controller, I suspect its driver may also
turn off interrupts, take excessive locks, or do other madness,
further worsening system responsiveness.

Kyle Moffett

There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies.
-- C.A.R. Hoare

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at
Please read the FAQ at