Re: REQ_FLUSH, REQ_FUA and open/close of block devices

[Alex Bligh]

--On 20 May 2011 08:20:10 -0400 Christoph Hellwig <hch@xxxxxxxxxxxxx> wrote:

> On Thu, May 19, 2011 at 04:06:27PM +0100, Alex Bligh wrote:
> > Should a close() of a dirty block device result in a REQ_FLUSH?
>
> No, why would it?  That's what fsync is for.

I had thought fsync() was meant to be implicit in a close of a raw device
though perhaps that's my faulty memory; I think you are saying it's
up to userspace to fix these; fair enough.

However, I'm also seeing writes to the device after the last flush when
the device is unmounted. Specifically, a sequence ending
 mount -t ext3 -odata=journal,barrier=1 /dev/nbd0 /mnt
 (cd /mnt ; tar cvzf /dev/null . ; sync) 2>&1 >/dev/null
 dbench -D /mnt 1 &
 sleep 10
 killall dbench
 sleep 2
 killall -KILL dbench
 sync
 umount /mnt

produces these commands (at the end):
 Sending command: NBD_CMD_WRITE [NONE] (0x00000001)
 Sending command: NBD_CMD_WRITE [NONE] (0x00000001)
 Sending command: NBD_CMD_FLUSH [NONE] (0x00000003)
 Sending command: NBD_CMD_WRITE [FUA] (0x00010001)
 Sending command: NBD_CMD_WRITE [NONE] (0x00000001)
 Sending command: NBD_CMD_WRITE [NONE] (0x00000001)
 Sending command: NBD_CMD_WRITE [NONE] (0x00000001)
 Sending command: NBD_CMD_WRITE [NONE] (0x00000001)
 Sending command: NBD_CMD_WRITE [NONE] (0x00000001)
 Sending command: NBD_CMD_WRITE [NONE] (0x00000001)
 Sending command: NBD_CMD_WRITE [NONE] (0x00000001)
 Sending command: NBD_CMD_WRITE [NONE] (0x00000001)
 Sending command: NBD_CMD_WRITE [NONE] (0x00000001)

(I'm testing this out by adding flush and fua support to nbd, see
git.alex.org.uk if this is interesting).

What I am concerned about is that relatively normal actions (e.g. unmount
a filing system) do not appear to be flushing all data, even though I
did "sync" then "umount". I suspect the sync is generating the FLUSH here,
and nothing is flushing the umount writes. How can I know as a block
device that I have to write out a (long lasting) writeback cache if
I don't receive anything beyond the last WRITE?

--
Alex Bligh
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/