Re: [patch] Real-Time Preemption, -RT-2.6.10-rc2-mm3-V0.7.32-6

[Ingo Molnar]

* Mark_H_Johnson@xxxxxxxxxxxx <Mark_H_Johnson@xxxxxxxxxxxx> wrote:

> In my "real" application (a large real time simulation running on a
> cluster) I cannot necessarily assign one batch of IRQ's higher than
> any others (nor above / below the main RT tasks). The character of
> my RT application is something like this:
> [...]

> CPU load is pretty steady at up to 20% for any of the two CPU nodes in
> the cluster. The upper bound for OS overhead (latency) I need is about
> 1 msec (out of a 12.5 msec / 80 Hz frame). I do have some long CPU
> runs / PCI shared memory traffic in the 80 Hz task at a one per second
> rate that might take up to 10 msec of the 12.5 msec frame.

so the 1 msec latency is needed by this 80 Hz task? I'd thus make this
task prio 90 (higher than most IRQ handlers), and make the 80 Hz
timesource's [timer IRQ? RTC? special driver?] IRQ thread prio 91. All
other IRQ threads should be below prio 90. Whatever else this task
triggers will be handled either by PI handling, or is started enough in
advance (such as disk IO or network IO) to be completed by the time the
80 Hz task needs it.

> I could set the IRQ priority of the shared memory interface to be the
> highest (since I do task scheduling based on it) but after that there
> is also no preset assignment of priority to I/O activity.

but if this is the task that needs to do its work within 1 msec when
signalled, it should be the highest prio one nevertheless, and no IRQ
(except the signal IRQ) must be allowed to preempt it.

(The other tasks can 'feed' this master task with whatever scheduling
pattern, as long as the 'master task' provides frames with a precise 80
Hz frequency. Any jitter to the execution of these other threads is
handled by buffering enough stuff in advance.)

> Some form of priority inheritance may be "better" but I understand
> that is not likely to be implemented (nor worth the effort).

the master task's priority will be inherited across most of the
dependencies that might happen at the kernel level. [ If it doesnt then
it should show up in traces and i'm most interested in fixing it ... ]

> By setting the IRQ threads to RT FIFO 99, I also get something closer
> to PREEMPT_DESKTOP w/o IRQ threading (or for that matter, closer to
> the 2.4 kernel I use today). It shows more clearly the overhead of
> adding the threads.

i believe this is the wrong model for this workload.

> [...] As Ingo noted in a private message
>   "IRQ-threading will always be more expensive than direct IRQs,
>    but it should be a fixed overhead not some drastic degradation."
>
> I agree the overhead should be modest but somehow the test cases I run
> don't show that (yet). There is certainly more work to be done to fix
> that.

have you tried it with all debugging turned off? I'd like to fix any
performance problems related to IRQ/softirq threading. (If you mean the
'lost pings' problem, that one looks like to be more of a priority
inversion problem than a real performance issue.)

	Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/