Re: [Lsf-pc] [LSF/MM TOPIC] really large storage sectors - goingbeyond 4096 bytes

[Chris Mason]

On Thu, 2014-01-23 at 13:27 -0800, Joel Becker wrote:
+AD4- On Wed, Jan 22, 2014 at 10:47:01AM -0800, James Bottomley wrote:
+AD4- +AD4- On Wed, 2014-01-22 at 18:37 +-0000, Chris Mason wrote:
+AD4- +AD4- +AD4- On Wed, 2014-01-22 at 10:13 -0800, James Bottomley wrote:
+AD4- +AD4- +AD4- +AD4- On Wed, 2014-01-22 at 18:02 +-0000, Chris Mason wrote:
+AD4- +AD4- +AFs-agreement cut because it's boring for the reader+AF0-
+AD4- +AD4- +AD4- +AD4- Realistically, if you look at what the I/O schedulers output on a
+AD4- +AD4- +AD4- +AD4- standard (spinning rust) workload, it's mostly large transfers.
+AD4- +AD4- +AD4- +AD4- Obviously these are misalgned at the ends, but we can fix some of that
+AD4- +AD4- +AD4- +AD4- in the scheduler.  Particularly if the FS helps us with layout.  My
+AD4- +AD4- +AD4- +AD4- instinct tells me that we can fix 99+ACU- of this with layout on the FS +- io
+AD4- +AD4- +AD4- +AD4- schedulers ... the remaining 1+ACU- goes to the drive as needing to do RMW
+AD4- +AD4- +AD4- +AD4- in the device, but the net impact to our throughput shouldn't be that
+AD4- +AD4- +AD4- +AD4- great.
+AD4- +AD4- +AD4- 
+AD4- +AD4- +AD4- There are a few workloads where the VM and the FS would team up to make
+AD4- +AD4- +AD4- this fairly miserable
+AD4- +AD4- +AD4- 
+AD4- +AD4- +AD4- Small files.  Delayed allocation fixes a lot of this, but the VM doesn't
+AD4- +AD4- +AD4- realize that fileA, fileB, fileC, and fileD all need to be written at
+AD4- +AD4- +AD4- the same time to avoid RMW.  Btrfs and MD have setup plugging callbacks
+AD4- +AD4- +AD4- to accumulate full stripes as much as possible, but it still hurts.
+AD4- +AD4- +AD4- 
+AD4- +AD4- +AD4- Metadata.  These writes are very latency sensitive and we'll gain a lot
+AD4- +AD4- +AD4- if the FS is explicitly trying to build full sector IOs.
+AD4- +AD4- 
+AD4- +AD4- OK, so these two cases I buy ... the question is can we do something
+AD4- +AD4- about them today without increasing the block size?
+AD4- +AD4- 
+AD4- +AD4- The metadata problem, in particular, might be block independent: we
+AD4- +AD4- still have a lot of small chunks to write out at fractured locations.
+AD4- +AD4- With a large block size, the FS knows it's been bad and can expect the
+AD4- +AD4- rolled up newspaper, but it's not clear what it could do about it.
+AD4- +AD4- 
+AD4- +AD4- The small files issue looks like something we should be tackling today
+AD4- +AD4- since writing out adjacent files would actually help us get bigger
+AD4- +AD4- transfers.
+AD4- 
+AD4- ocfs2 can actually take significant advantage here, because we store
+AD4- small file data in-inode.  This would grow our in-inode size from +AH4-3K to
+AD4- +AH4-15K or +AH4-63K.  We'd actually have to do more work to start putting more
+AD4- than one inode in a block (thought that would be a promising avenue too
+AD4- once the coordination is solved generically.

Btrfs already defaults to 16K metadata and can go as high as 64k.  The
part we don't do is multi-page sectors for data blocks.

I'd tend to leverage the read/modify/write engine from the raid code for
that.

-chris

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/