Syn Cookies

Robert Dinse (nanook@eskimo.com)
Fri, 24 Dec 1999 18:08:04 -0800 (PST)

On Thu, 23 Dec 1999 Chuck Phillips <cdp@peakpeak.com> wrote:
>
> What I can say is that a co-worker and I recently did tests of Apache,
> squid and telnet connections during massive syn-flooding with and without
> syn-cookies on a test network. (By massive, I mean random source addresses
> on a test network with 100BTX end-to-end connections.) Without syn
> cookies, *all* of these services became unavailable pretty quickly and it
> took almost ten minutes after the flooding ceased before the server fully
> recovered. With syn-cookies turned on, squid and Apache ran a little
> slower but otherwise were unaffected. Telnet "worked" but was so slow as
> to be unusable for all practical purposes. (I expect this has something to
> do with delays every 1-2 characters for telnet vs. per almost 1500
> characters for HTTP.)
>
> As a result, I'm thinking syn-cookies should be enabled by default,
> *perhaps* unless the server is underpowered (and there already is a
> separate configuration parameter for this). This means a documentation
> change at the very least and a small change at most.

There is something broken in the current Linux implementation of SYN
cookies. I don't know what but they are broken. Let me explain...

I run an ISP; hence both SYN floods and real heavy load have been common
occurances.

At one point we ran all SunOS 4.1.4 systems here. Apache would choke on
any load at all. Then I learned of some performance tweaks that basically
increased the size of the open connection cache or whatever you call the linked
list of structures where it has received a SYN, sent a SYN-ACK, but hasn't
received the ACK-ACK.

That helped, but on Sum4m architecture you are limited to 2MB of mbuf's by
the memory layout used by the kernel, so you couldn't increase them enough to
handle heavy traffic without running out of mbuf's and killing the whole
machine.

Then someone came out with a SYN cookie patch; installed that, worked
marvelously. Machine would then handle heavy load, or SYN floods, without
running out of mbuf's. At worst it would chew up a lot of CPU and slow things
down.

But I eventually moved to Linux for other reasons, the 2GB partition limit
of SunOS was too constraining, the 256fd limit of Sun's C libs was unworkable.
You could jack up the fd's in the kernel easy enough but the library functions
would choke.

Under Linux 2.0.x, SYN cookies seemed to work well, and we even modified
machines here so that they always used them instead of only after a threshold
was reached.

But under 2.2.13, they are definitely broken. If enabled, sooner or later
processes get "stuck" in a state where they are closing a connection but it
never finishes closing. With Apache, this results in as many children as it
will fork being forked, then as each process hangs, the whole thing gradually
grinds to a halt. This did not happen on 2.0.x at all, ever, even with SYN
cookies modified to always be used.

If I leave SYN cookies off, this behavior does not occur. There does seem
to be some architecture dependence; I have this problem on some small servers
running on LX's, but a larger server running on an SS-10 does not do this even
though it carriers about 4X heavier traffic. One factor, the LX's have only
96MB of RAM, the SS-10 has 384MB, so there might be some memory exhaustion
issue here.

Whatever it is, if I don't turn SYN cookies on, it doesn't happen and it
didn't used to happen under 2.0.x at all.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/