Re: Interesting analysis of linux kernel threading by IBM

From: Davide Libenzi (dlibenzi@maticad.it)
Date: Thu Jan 20 2000 - 19:44:32 EST


Hi Larry,

On Thu, 20 Jan 2000, Larry McVoy wrote:
> There are a number of unsubstantiated statements in this message and
> I'd like to request that the various people who are claiming that these
> statements are true, please back them up. All we need is vmstat/ps/top
> output on machines with these sorts of loads. Benchmark machines do
> not count, only real workloads, please.
>
> : > Hondreds of tasks is just not a typical (perhaps even realistic)
> : > workload.
> :
> : Yes it is.
> :
> : If you are running a webserver.
>
> ps -ax outout please.
>
> : Or a highly threaded application.
>
> Highly threaded == highly stupid in 99.9% of all cases. Before wacking the
> kernel to support these apps, please describe an application which would
> be be smaller and faster if it were select based.
>
> : Or a machine with a lot of users. (For example, a University unix server)
>
> vmstat & ps -axf please.
>
> : Or an ftp server. (Where is the Linux equivalent of FreeBSD's ftp.cdrom.com?)
>
> I'm with you here. I'd like to see this too. But before we "fix" the
> scheduler, I'd also like to see a some evidence that the scheduler is
> the issue, and benchmark evidence is of marginal interest.
>
> : It is really a question of "Where does Linux want to go?"
>
> I know where I don't want it to go. I don't want it to go down the
> "my dick is bigger than your dick" marketing rathole which has lead
> all the other Unix vendors into pretty much destroying their own kernel
> source base. All in the name of the mighty benchmark.
>
> : If it wants to be a high performance server, Linux needs a new scheduler.
>
> If it wants to be a high performance server, we need to design it to be
> a high performance server. My challenge to all of the "scheduler must
> be fixed" people is this: how are your proposed fixes going to help get
> us to performance levels 1000x what we are at today? They obviously
> aren't. So instead of going after the microwins, how about going after
> the big wins? In the past, I've outlined a plan that can get Linux
> kicking everyone's butt on 2048 processor (or bigger) boxes. You may or
> may not like my ideas, but the goals are certainly better than what you
> are doing. You're just falling into the same trap that has buggered up
> all of the existing Unix kernels: ``we're at performance level X, let's
> hack the system into performance level 2X''. That isn't designing or
> architecting, that's hacking. And that path leads to a crap source base.
> Please look at the big picture.
>
> : If it wants to be the most efficient desktop machine, then it doesn't
> : need it NOW. However, the average number of programs people are
> : running on their machine are increasing, not decreasing.
>
> This is a completely unsubstatiated statement. OK, everyone, do this:
>
> $ vmstat
> load free cach swap pgin pgou dk0 dk1 dk2 dk3 ipkt opkt int ctx usr sys idl
> 0.00 4.3 96.9 0.4 0 0 0 0 0 0 0 0 104 53 0 3 97
>
> Those people with the load field higher than 10, please tell us what you are
> doing.
>

have You seen that patch we're speaking about ?
It simply subdivide the runqueue in clusters and give the scheduler a
logarithmic low as function of the number N of runnable tasks.

1) It's very simple
2) Under home usage give practically the same results ( probably less then 1%
cost )
3) Under high load gives great results due to the pseudo-logarithmic low

> are doing. You're just falling into the same trap that has buggered up
> all of the existing Unix kernels: ``we're at performance level X, let's
> hack the system into performance level 2X''. That isn't designing or
> architecting, that's hacking. And that path leads to a crap source base.
> Please look at the big picture.

Not always patching codebase introduces new bugs.
AFAIK Linux codebase is grown about 10x form 1.0 to 2.3.X and not only bugs
were introduced into the sources.

Cheers,
        Davide.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Sun Jan 23 2000 - 21:00:24 EST