Hello list,
I have an application that creates a pool of 1000 threads that sleep() for
various intervals, and then wake up and do something. I also have another
version of the same app that uses a single thread together with program
logic to do essentially the same thing: sleeping until it is time to wake up
and do something. I'm surprised to see that the overhead of having 1000
threads that each use a timer is only about 60% of the overhead of the most
efficient attempt that I could muster using a single thread.
Since I'll need the pool size will go from 1000 to 6000 (technical hurdles
aside), I was wondering if I can expect the kernel to continue to scale as
well. I think that I understand why it has scaled so well thus far, and I'd
like some type of validation/invalidation: I have a boatload of threads, but
all except a very small number will be in the TASK_INTERRUPTIBLE state in
stead of TASK_RUNNABLE, and therefore the scheduler doesn't have to walk a
list of 1000+ processes, just 2 or 3.
My questions are these:
1) Are there any kernel structures that are O(n) or just as bad, where n is
the total number of processes, not just the total number of runnable
processes? i.e. will the performance of the system be adversely affected by
the number of tasks in wait queues?
2) If I have 1000 threads, and each calls sleep(), I assume that my
motherboard cannot handle 1000 timers in hardware, so the kernel has to
somehow keep track of a timer by polling and sending SIGALRM to the owning
process when each expires, which causes glibc's sleep() to somehow return.
The question is this - how can asking the kernel to manage 1000 timers be so
much more efficient than asking it to manage just 1? Or better yet, how can
I load the system down with 1000, and see absolutely *NO* demonstrable
difference in the system's responsiveness?
3) I am tempted to cajole linuxthreads to do that many, or get clone() to
play nicely with glibc, or still use linuxthreads and have 6 processes with
1000 each that share memory, but don't want to figure it all out, just to
see it not perform well. So, if the kernel impresses the hell out of me at
1000, will it at 6000? Or is a pool of 6000 threads, sleeping or not, just
crazy, which was my first impression?
Thanks to any and all,
Don Dade
"I'm not going back to win32. Ever."
_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Fri Aug 31 2001 - 21:00:13 EST