Re: [RFC PATCH 00/31] netfs: [WIP] Allow the use of MSG_SPLICE_PAGES and use netmem allocator

[Enzo Matsumiya]

On 08/11, David Howells wrote:
> Hi Enzo,
>
> I now have encryption, compression and encryption+compression all working :-)
>
> I've pushed my patches here:
>
> 	https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=cifs-experimental
>
> It should work up to "cifs: Don't use corking".

Great! I'll try it out later.

> Btw, is is_compressible() actually worth doing?  It seems to copy a lot of
> data (up to 4M) to an extra buffer and then do various analyses on it,
> including doing a sort.

Compression, as a whole, is actually only worth doing if one is paying
more for network traffic than computing.  is_compressible() tries to
balance that to avoid a "compress/fail/send original" cycle, as it takes
0-4ms on a 4M payload (on my machine) whereas, without it, a failing
cycle can take up to 40ms.

> I need to extract a fix for collect_sample(), which I can do tomorrow, but it
> should look something like:
>
> /*
> * Collect some 2K samples with 2K gaps between.
> */
> static int collect_sample(const struct iov_iter *source, ssize_t max, u8 *sample)
> {
> 	struct iov_iter iter = *source;
> 	size_t s = 0;
>
> 	while (iov_iter_count(&iter) >= SZ_2K) {
> 		size_t part = umin(umin(iov_iter_count(&iter), SZ_2K), max);
> 		size_t n;
>
> 		n = copy_from_iter(sample + s, part, &iter);
> 		if (n != part)
> 			return -EFAULT;
>
> 		s += n;
> 		max -= n;
>
> 		if (iov_iter_count(&iter) < PAGE_SIZE - SZ_2K)
> 			break;
>
> 		iov_iter_advance(&iter, SZ_2K);
> 	}
>
> 	return s;
> }
>
> What's currently upstream won't work and may crash because it assumes that
> ITER_XARRAY is in use - which should now never be true.

Yes, compression was merged when that was the only case.

> Also, there's a bug in wireshark's LZ77 decoder.  See attached patch.

Good catch :)
There are several, actually... if you vary the compression parameters
defined (min len, min/max dist, hash log) within acceptable limits,
you'll notice that, even though wireshark might show some as malformed
packets, Windows is able to decode them just fine.

I really need to reserve some time to work on this again :(


Cheers,

Enzo