Re: per device queues for cciss 2.6.0
From: Jeff Garzik
Date: Wed Mar 10 2004 - 13:26:22 EST
Miller, Mike (OS Dev) wrote:
Yes, the controller has a single command buffer. It can hold 1024 outstanding commands.
Ok, great. Well then the carmel.c I sent you should be a good model --
carmel.c has per-device queues, and there are no starvation issues. The
code is contained within only a few LOC, in carm_push_q(), carm_pop_q(),
and carm_round_robin().
As an aside, you should probably make the call to cciss_round_robin()
conditional on the hardware's command buffer being at least 1/2 empty.
(or pick whatever low water mark you like)
The command buffer size, 1024, is quite nice. Given the same model as
carmel.c, I predict that blk_{start,stop}_queue will be called quite
infrequently -- that translates to _high_ performance on the cciss hardware.
Note the blk_{start,stop}_queue() were only recently fixed (grab latest
2.6.4-rc), so that may have introduced noise into whatever testing and
design you've done.
Now, per-queue locking, rather than per-HBA locking, definitely
introduces some additional complexity. I've got a good idea how to do
that, which involves the each queue's request function kicking a common
tasklet that queues commands to hardware. But there's a lot of deadlock
potentional if it's not done right, since you still need a common lock
for the HBA when submitting and completing hardware commands. So I
would be interested to see some evidence of actual SMP contention on the
per-HBA lock...
Regards,
Jeff
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/