Re: nbd, nbdkit, loopback mounts and memory management

From: Pavel Machek
Date: Sun Feb 17 2019 - 18:15:19 EST


Hi!

> So not to dispute that this could be a bug, but I couldn't cause a
> deadlock. I wonder if you can see something wrong with my method?
>
> *** Set up ***
>
> - kernel 5.0.0-0.rc3.git0.1.fc30.x86_64
> - nbd-client 3.19-1.fc30
> - nbdkit 1.11.5 (git commit ef9d1978ce28)
>
> Baremetal machine was booted with mem=2G to artificially limit the
> RAM. The machine has 64G of swap.
>
> # free -m
> total used free shared buff/cache available
> Mem: 1806 329 1383 0 93 1357
> Swap: 65535 179 65356
>
> *** Method ***
>
> I started nbdkit as a 4G RAM disk:
>
> ./nbdkit memory size=4G
>
> This is implemented as a sparse array with a 2 level page table, and
> should allocate (using malloc) every time a new area of the disk is
> written to. Exact implementation is here:
> https://github.com/libguestfs/nbdkit/tree/master/common/sparse
>
> I started nbd-client using the -swap option which uses
> mlockall(MCL_FUTURE) to lock the client into RAM.
>
> nbd-client -b 512 -swap localhost /dev/nbd0
>
> I then created a filesystem on the RAM disk, mounted it, and copied a
> 3G file into it. I tried this various ways, but the variation I was
> eventually happy with was:
>
> mke2fs /dev/nbd0
> mount /dev/nbd0 /tmp/mnt
>
> dd if=/dev/zero of=/tmp/big bs=1M count=3000
> cp /tmp/big /tmp/mnt/big
>
> I couldn't get any kind of deadlock or failure in this test.
>
> (Note that if you repeat the test several times, in theory you could
> delete the file and fstrim the filesystem, but when I was testing it
> to be sure I unmounted everything and killed and restarted nbdkit
> between each test.)

This looks like quite a good try. I'd try to use mmap() to dirty
memory very quickly.

But Shaun reported it happened somehow often for them, so he might
have a practical test case... better than my theories :-).

Best regards,
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Attachment: signature.asc
Description: Digital signature