You make it sound like I don't know what I'm measuring. This is
incorrect. I am trying to determine two things:
- what is the lower limit on the context switch time (two processes on
the run queue)
- what is the effect of extra processes on the run queue on the above
measurement.
Using these values I can then look at where time can be
saved. Further, my tests have shown variability, which has led me to
investigate the cause of that variability. If the variability is due
to caching problems, I can look at how to minimise the effects.
So the benchmarks I've run have provided useful data. I've been able
to reorder struct task_struct and reduce the context switch latency as
a result of my analysis (from 0.2 us per run queue process to 0.15
us). I've also been able to reduce the variability this way.
> As to taking processes on and off the run queue, yup, that's by
> design. In my experience that is part of a context switch cost.
> Linus said this a different way, but I'll try it like this: in the
> cases where I might be context switching without removing myself
> from the run queue (i.e., my time slice ran out), I just ran for
> 10,000 usecs. Whether the context switch is 10 or 100 usecs is in
> the noise. In the case where I ran less than my time slice, I am
> removing myself from the run queue because I'm blocking on I/O. In
> that case I may very well have run for an extremely short time and
> then context switched, where the context switch does indeed include
> a run queue removal. And that's exactly what I wanted to measure, I
> never intended to factor that out and you shouldn't want to either.
Part of my motivation comes from the question (which I've been asked
by some pSOS+ users here considering switching to Linux), is "how long
after an interrupt is delivered will a blocked RT process start
running"?
Part of this is the interrupt latency. I've seen people looking at
this on the list, but I've not seen much attention being paid to how
long it takes to wake a process up and have it start running
(i.e. switch out whatever is running now and switch in my RT task). So
hence I'm looking at this part.
I am well aware there are other factors (like !##@$ 8390-based network
drivers which disable interrupts globally while reading the packet,
which is up to 1.6 ms! Ouch). These need to be addressed too, but for
now I'm focussing on the context switch overhead.
> As to the realistic nature of the benchmark, I must disagre with
> your claim. You are claiming that systems have long run queues
> without actually showing that to be true, even in your case. At
> least everything you've stated so far is conjecture, not based on
> observation. In other systems, which sound quite similar to what
> you have described, I've seen similar claims of long run queues, but
> when I went and actually measured things, it was very rare to see a
> queue depth of more than 2/CPU.
I'm in luck: the observing programme is running right now. I've taken
a quick peek and I'm seeing 7 processes on the run queue (not
including my measurement process:-). That's with just one
observer/user running the online reduction software. That software
basically controls the data logging, processing and display. The
instrument control and data capture software is running elsewhere.
This is on a dedicated Ultra2 with two 167 MHz CPUs.
If we put the instrument control software on this machine, expect a
few more processes on the run queue. If we start adding more users,
the run queue gets even longer.
Also, please note that the thrust of my concern is how long it takes
to wake up a blocked RT process. It doesn't matter if I have some RT
process which is normally blocking. What matters is that when it has
to wake up, it does so as quickly as possible. One of the overheads
here is the length of the run queue. Having separate run queues will
mean you are one step closer to safely putting more "normal" load on a
machine without worrying about what it does to your RT latencies.
Where I work we are lucky to be able to afford a separate machine for
users and instrument control. This isn't always the case for smaller
institutions and small university departments.
Regards,
Richard....
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/