On Mon, 19 Jul 1999 23:55:22 +0200, Pavel Machek <pavel@bug.ucw.cz>
said:
> 1) our network layer allocates memory for incoming packets.
> 2) nbd will not work without network stack running.
> 3) all atomic memory can disappear instantly (improbable but happens)
> Therefore:
> All memory disappears (3). System decides it wants to swap out. But
> there's not enough memory to receive ACK. It even is not enough memory
> to respond to ARP requests (1). Therefore our machine looses
> networking which means nbd will stop working.
> We are "deadlocked".
Not necessarily. You are assuming that kswapd cannot make any progress
while the swapout is blocked.
This is exactly why we added the 2.2.2 kpiod thread. It is an
independent thread which carries out the blocking pageout operations on
behalf of kswapd. If that thread becomes totally blocked, kswapd simply
stops giving it more work to do, and devotes itself to rescuing the
situation by removing existing clean pages from memory.
Currently we only use the kpiod thread for filemapped pageouts, but
there is absolutely nothing to stop us from doing the same sort of thing
for swapout. It is not hard, and it _does_ prevent the deadlock you
describe. Remember, the networking is already resistent to momentary
loss of GFP_ATOMIC memory, so as long as kswapd can keep making
progress, nbd won't completely stall.
> If someone gave us few pages of memory, we could reply to ARPs, get
> acks and see that swapouts were successfull. But nobody is going to
> give us pages of memory.
That is where your argument is wrong --- we already have constructions
in place to make kswapd resistant to its IOs stalling.
> Do you see why swapping over nbd (or over nfs) is extremely nasty,
> now?
I've always known why it was nasty. We had very similar deadlock
problems when paging out regular mmaped files due to recursion on the
superblock lock: a write() took the superblock lock for an allocation,
but then needed more memory. kswapd tried to flush back a page from a
mmaped file to disk but needed the superblock lock to do so. Deadlock
between the allocator and kswapd, which arises because kswapd needs a
resource which cannot be released until the swapper makes progress.
We already fixed this in once case by moving the IO out of kswapd
itself. Now do you see why the difficulties of swapping over nbd are no
different, in principle, to the difficulties that kpiod already solves
for a related case?
--Stephen
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/