Re: request baset device mapper in Linux

From: Lukas Hejtmanek
Date: Thu Jul 21 2011 - 09:26:19 EST

On Thu, Jul 21, 2011 at 08:11:32PM +0900, Kiyoshi Ueda wrote:
> I don't understand why you are saying request-based device-mapper makes
> serious troubles.
> BIO merging is done in the block layer. Don't you see the same thing
> if you use sd devices (/dev/sdX)?

no, if I use sd devices directly, it does not overload ksoftirqd and this is
obvious. The problem with overloading ksoftirqd has roots in request based
stuff in dm layer, in particular in dm_softirq_done() call.

> Also, what do you see with the latest upstream kernel (say, 3.0-rc7)?

I didn't check it with this kernel. I don't see many changes in request based
stuff in dm.

> If you see your problem only with request-based device-mapper, please
> elaborate about below:
> - end_clone_bio() has someting like quadratic complexity.
> * What do you mean the "quadratic complexity"?

end_clone_bio calls blk_update_request which calls __blk_recalc_rq_segments
which has code:
for_each_bio(bio) {
bio_for_each_segment(bv, bio, i) {

in total, it seems that *whole* bio list is traversed again and again as some parts
are done and some not which leads to comlexity O(n^2) with respect to number
of bio and segments. But this is just wild guess. The real problem could be

However, oprofile or sysprof show that ksoftirq spends most time in

> - each request contains more then 100 bios which makes serious
> troubles for ksoftirqd call backs.
> * What do you mean the "serious troubles for ksoftirqd call backs"?

serious troubles means that ksoftirqd eats 100% CPU and slows down I/O
significantly (from 2.8GB/s to 500MB/s).

Lukáš Hejtmánek
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at
Please read the FAQ at