Re: per device queues for cciss 2.6.0

From: Jeff Garzik
Date: Wed Mar 10 2004 - 13:26:22 EST

Miller, Mike (OS Dev) wrote:
Yes, the controller has a single command buffer. It can hold 1024 outstanding commands.

Ok, great. Well then the carmel.c I sent you should be a good model -- carmel.c has per-device queues, and there are no starvation issues. The code is contained within only a few LOC, in carm_push_q(), carm_pop_q(), and carm_round_robin().

As an aside, you should probably make the call to cciss_round_robin() conditional on the hardware's command buffer being at least 1/2 empty. (or pick whatever low water mark you like)

The command buffer size, 1024, is quite nice. Given the same model as carmel.c, I predict that blk_{start,stop}_queue will be called quite infrequently -- that translates to _high_ performance on the cciss hardware.

Note the blk_{start,stop}_queue() were only recently fixed (grab latest 2.6.4-rc), so that may have introduced noise into whatever testing and design you've done.

Now, per-queue locking, rather than per-HBA locking, definitely introduces some additional complexity. I've got a good idea how to do that, which involves the each queue's request function kicking a common tasklet that queues commands to hardware. But there's a lot of deadlock potentional if it's not done right, since you still need a common lock for the HBA when submitting and completing hardware commands. So I would be interested to see some evidence of actual SMP contention on the per-HBA lock...



To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at
Please read the FAQ at