Asynchrony (was Re: Locking a process or thread onto a specific CPU)

David Wragg (dpw@doc.ic.ac.uk)
19 Feb 1999 02:50:40 +0000

Jamie Lokier <lkd@tantalophile.demon.co.uk> writes:
> There's another problem. Sometimes, one of your tasks blocks in the
> kernel. This happens however much effort you go to with poll(),
> O_NONBLOCK etc. Some things will just block anyway. Disk I/O is
> especially unpredictable at this; paging to disk also comes into this
> category. Ideally, you'd like to continue using that CPU for some more
> of your threads. A solution often used (see nfsd) is to run more than
> one task per CPU. But that's non-optimal because the number is often
> more than is optimal (= 1 per CPU), and sometimes less than is optimal
> (= 1 + number of blocked tasks, per CPU).
>
> IMO, the optimal mechanism is something which lets you schedule threads
> purely in userspace the way the kernel can do it for tasks. That is,
> have one task at a time running on each CPU at all times (as far as your
> program is concerned).
>
> [snip]
> So we'd like a mechnism to tell our program to restart
> another task and schedule threads into it. No problem: a task flag to
> say "if I block, wake up process X", plus user space code to say "when I
> carry on, I'll stop myself if X is running, or tell process X to stop".

To have a substitute task get woken with a signal just needs something
like:

pseudo-diff against kernel/sched.c, line 670 in 2.2.1
switch (prev->state) {
case TASK_INTERRUPTIBLE:
if (signal_pending(prev)) {
prev->state = TASK_RUNNING;
break;
}
+ /* fall through */
+ case TASK_UNINTERRUPTIBLE:
+ if (prev->substitute) {
+ spin_unlock_irq(&runqueue_lock);
+ kill_proc(prev->substitute,
+ prev->substitute_sig, 0);
+ spin_lock_irq(&runqueue_lock);
+ prev->subsitute = 0;
+ }
+ /* fall through */
default:
del_from_runqueue(prev);
case TASK_RUNNING:
}

Plus a new prctl option to set the substitute and substitute_sig for a
task. Probably support for passing the pid of the about-to-sleep task
to the substitute as signal info would be nice also.

Is this actually useful? Well, the cost of the kill_proc is low; we're
about to do a task switch anyway.

For the userspace thread scheduler Jamie mentions, the substitute
thread upon waking just calls the scheduler. It will even allow you
attempt to do work while the original thread suffers a page-in. It
should also be straightforward to implement a async IO API on top of
it. The thread returning from the call to initiate IO may be a
different task from the one that called it, but I think all the
problems this causes can be worked around.

Dave Wragg

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/