Re: [PATCH] block: bios with an offset are always gappy

[Ming Lei]

On Thu, Apr 13, 2017 at 10:06:29AM +0200, Johannes Thumshirn wrote:
> Doing a mkfs.btrfs on a (qemu emulated) PCIe NVMe causes a kernel panic
> in nvme_setup_prps() because the dma_len will drop below zero but the
> length not.

Looks I can't reproduce the issue in QEMU(32G nvme, either partitioned
or not, just use 'mkfs.btrfs /dev/nvme0n1p1'), could you share the exact
mkfs command line and size of your emulated NVMe?

> 
> A git bisect tracked the behaviour down to commit 729204ef49ec ("block:
> relax check on sg gap"). Since commit 729204ef49ec a bio's offsets are not
> taken into account in the decision if the bio will gap any more. Restore
> the old behavior of checking bio offsets as well for the decision if a
> bio will gap.
> 
> Signed-off-by: Johannes Thumshirn <jthumshirn@xxxxxxx>
> Fixes: 729204ef49ec ("block: relax check on sg gap")
> Cc: Ming Lei <ming.lei@xxxxxxxxxx>
> ---
>  include/linux/blkdev.h | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
> index 7548f332121a..a03b7196209e 100644
> --- a/include/linux/blkdev.h
> +++ b/include/linux/blkdev.h
> @@ -1677,11 +1677,14 @@ static inline bool bio_will_gap(struct request_queue *q, struct bio *prev,
>  {
>  	if (bio_has_data(prev) && queue_virt_boundary(q)) {
>  		struct bio_vec pb, nb;
> +		bool offset;
>  
>  		bio_get_last_bvec(prev, &pb);
>  		bio_get_first_bvec(next, &nb);
>  
> -		if (!bios_segs_mergeable(q, prev, &pb, &nb))
> +		offset = pb.bv_offset || nb.bv_offset;

We don't consider pb's offset here, because if pb.bv_offset
isn't zero, pb should be the only bvec in 'prev' and will be
put in a standalone segement, and we still can make 'nb' into
this segment if both are mergeable.

But the following issue might be caused by commit 729204ef49ec
("block: relax check on sg gap"):

	- if the 'next' has more than one segment
	- the segment merged from 'pb' and the 1st segment of 'next'
	may end at un-aligned virt boundary

Could you try the following patch to see if it fixes your issue?

---
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 7548f332121a..65d1510681c6 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -1659,16 +1659,28 @@ static inline bool bvec_gap_to_prev(struct request_queue *q,
  * and the 1st bvec in the 2nd bio can be handled in one segment.
  */
 static inline bool bios_segs_mergeable(struct request_queue *q,
-		struct bio *prev, struct bio_vec *prev_last_bv,
+		struct bio *prev, struct bio *next,
+		struct bio_vec *prev_last_bv,
 		struct bio_vec *next_first_bv)
 {
 	if (!BIOVEC_PHYS_MERGEABLE(prev_last_bv, next_first_bv))
 		return false;
 	if (!BIOVEC_SEG_BOUNDARY(q, prev_last_bv, next_first_bv))
 		return false;
-	if (prev->bi_seg_back_size + next_first_bv->bv_len >
+	if (prev->bi_seg_back_size + next->bi_seg_front_size >
 			queue_max_segment_size(q))
 		return false;
+
+	/*
+	 * if 'next' has multiple segments, we need to make
+	 * sure the merged segment from 'pb' and the 1st segment
+	 * of 'next' ends at aligned virt boundary.
+	 */
+	if ((next->bi_seg_front_size < next->bi_iter.bi_size) &&
+	    ((prev_last_bv->bv_offset + prev_last_bv->bv_len +
+	     next->bi_seg_front_size) & queue_virt_boundary(q)))
+		return false;
+
 	return true;
 }
 
@@ -1681,7 +1693,7 @@ static inline bool bio_will_gap(struct request_queue *q, struct bio *prev,
 		bio_get_last_bvec(prev, &pb);
 		bio_get_first_bvec(next, &nb);
 
-		if (!bios_segs_mergeable(q, prev, &pb, &nb))
+		if (!bios_segs_mergeable(q, prev, next, &pb, &nb))
 			return __bvec_gap_to_prev(q, &pb, nb.bv_offset);
 	}
 

-- 
Ming