Re: How to increat [sic.] max open files?

Richard B. Johnson (root@analogic.com)
Fri, 3 Jan 1997 16:19:46 -0500 (EST)


On Fri, 3 Jan 1997, Baldur Norddahl wrote:
[SNIPPED]
> On Fri, 3 Jan 1997, Richard B. Johnson wrote:
[SNIPPED]
>
> > Now, it is not efficient to kick-off a separate child to handle each
> > connection. It is also not efficient to have a single task handle everything.
> > There is some design necessary to figure out what goes in between.
>
> In the mud case it is actually efficient to keep everything in a single
> process. At least that is what the profilers say. It is the simulation of
> the virtual world that hogs the CPU, not the client handling. Having to
> lock the objects in the virtual world would just complicate the simulation
> and thereby make it slower.
>
> What teoratical background do you have for concluding that using a single
> process to handle the clients is NEVER efficient?
>

The idea is that it is most efficient to have the kernel do as much work
as possible. The presumption is that the kernel code in any operating system
will be tuned to perform its function with the minimum possible overhead.
Indeed, in many architectures, there are even opcodes that are not allowed
in user-space programs, that are used by the operating system. In fact,
the operating system can often set some bit in a page-register, rather
than having to copy data, etc.

Of course we will always have user-mode programmers who think that they
can make better code than the kernel code, but you should know how that
works.

When user code has to keep track of many "sockets" it usually has to look
through a list (perhaps linked) of things to be done once some event
(such as an inquiry from a socket-connected client), It can't just use
socket values as indexes because clients disconnect and new out-of-order
sockets are assigned for new connections.

Once the list becomes large, much time is wasted just getting to the
code that is going to service the request. There might even be a context-
switch or two before your application actually does anything useful as
far as the client is concerned.

Now, suppose your code used a different "port" (after some initial
negotiation), for each Client. Then suppose your code wasn't even
executed until the kernel gave you control with the port (read index),
already found.

Don't you think that this would be a more efficient way to handle the
stereotypical Client/Server methodology?

Now, this is just one example. It is not a good example but it is one
that is easy to understand. Another example is the simple telnet daemon.
It issues an "listen", and when a "connect" occurs, it sets up I/O
descriptors, spawns a child, then goes back to sleep. The child is
entirely independent "setsid()", uses its own resources, etc. This
child only worries about one client and has, in fact, become the client.
Until the number of "connections" reaches some point, this seems very
efficient. However, memory is being wasted because each of the children
(now users), have a lot of code that is only used once. At some point,
having separate children perform tasks on behalf of separate Clients is
no longer efficient because of the wasted overhead. Note that the telnet
example could be accessing a database or serving files instead of being
a terminal server to a shell.

My theoretical background spans about 30 years, starting as a youngster
working for Los Alamos Scientific and NASA/Ames up to my present work
with very high speed data acquisition and number-crunching used in
3rd generation CAT-Scanners. I have a couple of degrees, but mostly one
has to learn from the school of hard-knocks when bumping against the
leading edge of technology.

Some of this things I say just might be helpful when attempting to make
some things work a bit better than the general case.

Cheers,
Dick Johnson
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Richard B. Johnson
Project Engineer
Analogic Corporation
Voice : (508) 977-3000 ext. 3754
Fax : (508) 532-6097
Modem : (508) 977-6870
Ftp : ftp@boneserver.analogic.com
Email : rjohnson@analogic.com, johnson@analogic.com
Penguin : Linux version 2.1.20 on an i586 machine (66.15 BogoMips).
Warning : It's hard to remain at the trailing edge of technology.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-