Re: Overscheduling DOES happen with high web server load.
Richard Gooch (firstname.lastname@example.org)
Thu, 6 May 1999 18:55:58 +1000
> Date: Thu, 6 May 1999 10:42:18 +1000
> From: Richard Gooch <email@example.com>
> Indeed. As a separate question, we may wonder why so many
> processes/threads are being used, and whether that number
> could/should be reduced. Perhaps the server is doing something
> silly. But that's an aside. Instead, I'd like to explore ways of
> reducing the (already low) scheduler overhead.
> No, it is more beneficial to find out why so many tasks are waking up
> so much in the first place, _THIS_ is the bug.
> This smells of galloping herd phenomenon to me. And my nose says it's
> accept() in this case, and my nose further states that implementing
> wake-one semantics in accept() might even make this part of the
> profiling go away and increase our TCP server performance
> dramatically. Several people have suggested this to me in private
> discussions, and I was skeptical at first, but now I'm starting to
> believe it.
> This has nothing to do with how many threads are being used, it has to
> do with threads getting woken up only to find they have no work to do,
> which is the only way the run queues can grow large enough for these
> sorts of user processes.
I agree that the thundering herd problem needs to be resolved.
Implementing wake-one for select() might be a good idea, although I
prefer the approach of using a POSIX.4 signal which gives the fd to
the signal handler/waiter. It's more scalable, so it's what should be
used for best performance. Still, that's not to say it isn't worth
optimising select() and poll() for "legacy" code. When 2.3 comes out,
I'll redo my select()/poll() optimisation patch (factor 3x speedup),
which will also help.
However, just now I'm interested in seeing what effect my scheduler
optimisation patch has. This seems like a good test environment, so
I'm curious to see the result.
But you're welcome to care more about improving select() :-)
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to firstname.lastname@example.org
Please read the FAQ at http://www.tux.org/lkml/