SPAM, ORBS, and VGER

From: Matti Aarnio (mea@nic.funet.fi)
Date: Mon Jan 17 2000 - 19:33:17 EST


Hi,

  After a plenty of heated discussion at lots of lists where it
doesn't belong to at all, I have decided to say something. (Again
at sort of wrong place, although most ``discussion'' participants
are at linux-kernel list...)

  At first about ORBS:

As people have noticed, VGER runs now with online ORBS tests, but
unlike most such things, these are ``somewhat'' more complicated ones.

At VGER's smtp policy rule dataset tells:

#| Third RBL variant: Late block with RBL+DUL+RSS+ORBS
_RBL0 rcpt-dns-rbl rbl.maps.vix.com:dul.maps.vix.com:relays.mail-abuse.org:ok.orbs.org:relays.orbs.org
_RBL1 test-rcpt-dns-rbl +

Decoding this is somewhat more complicated, though.

- "rcpt-dns-rbl" attribute lists RBL-like suffixes which can be looked
  up the same way as original RBL.

- Lookups are done in sequence, and first one that yields A record
  with 127.0.0.* type address is returned as the error.

- There exists an exception (of course), if during the scan there appears
  A record which does *not* match 127.0.0.*, then a flag is set "has_ok",
  and testing continues. If next test yields A of 127.0.0.4, that "has_ok"
  is looked for, and if it is set, the block (netblock) is ignored.

This method means that larger netblocks can be punctuated with "ok.orbs.org"
zone, HOWEVER: The ok.orbs.org zone can contain FALSE OK cases when tests
have not reached back to ORBS DB for whatever reason.
( - network level firewall blocks ORBS system
  - messages destined to ORBS are blocked at MTA by non-generic means )

  Of SMTP Relay Hijack - and Chess..

You may wonder what Chess has to do with SPAM ?

Not in itself, except that people who are trying to fight spam are often
doing things they would not do if they would think two-three envolution
rounds ahead. If these people are capable of playing chess at all, they
would be rather bad at it...

One of such sad examples is ABOVE.NET's (and other such peoples) attitude
towards bringer of the bad tidings -- in modern world we are not supposed
(to try) to kill the messanger when the message doesn't please us...

ORBS is just a bringer of the bad news, they are not bad news themselves.

You should realize that keeping VGER alive is a *hobby* for DaveM and me
(and others associated with it), it just happens that these hobby things
have some correlation with work we do elsewere.

On world-scale my employer isn't the biggest ISP there is, but locally we
are largest in the country. We are serving there SMTP relay for about ten
million IPv4 addresses worth of possible systems. We have thousands of
customer systems sending email out thru our smart-relay servers.
(``10M addresses'' as more than 1M and less than 100M)

If we receive a note from ORBS (or whoever) that one of our major smtp
relay hubs is becoming blocked due to sloppy customer, we pass the puck
quickly to that customer by means of a phone call (if we have still valid
contact information), AND by blocking them at all of our HUBs.
We are then giving them two choises:
        - Armour their MTA against Relay Hijack
        - Start sending directly to the world without using
          our smart-relay service
In the latter case ORBS-like blocks will be that customer's problem, not
our HUBs, nor thousands of other secure and happy systems our customers
have all around.

   About usefullness of ORBS probe blocking -- and more Chess..

If a MTA, which is trivially (see http://www.orbs.org/envelopes.html)
misusable, tries just simply block/divert any message destined to
<orbs-relaytest@manawatu.co.nz>, they for sure will get off the ORBS
list, however abusers will still be able to use the system.

This is almoast in same category as Teergrubing, and other ``Sour
the Milk'' methods used to make system abusing less attractive for
misuse, however nothing is more trivial than adding more systems to
the task of spreading the junk.

People who do such things don't think beyond their immediate case.

Over the years of this SPAM problem emergence I have pondered upon
several ways to ``Sour the Milk'' when certain type of abusive patterns
have been seen. In such cases I have taken a moment to consider a
predator-pray kind of evolutionary pressures applied to each suggested
method. I don't offhand remember *any* such method which I can't envolve
into non-effective method within 2-3 minutes of thinking of how a spam-
ware writer could counter the gambit.

Taking it granted that ``best'' spamware writers won't be entirely stupid
coders, they will be able to turn an open-but-teergrubed server into one
of many workers for spewing. Like some (ORBS or RSS) page says, such methods
are turning really open servers to one which is still open, but its abuse
level drops so much that it feels ``fixed''.

One of the problems in the countermeasures development is that they MUST
be compliant to RFC 821, and all extensions that the input server has
decided to implement. Smooth running of the normal SMTP protocol is primary
goal to keep in mind.

Now take abuser signature into play, for example that they seem to be
writing HELO+MAIL FROM+RCPT TO+DATA all in one go without stopping to
wait for the half-duplex replies. If input server capable to detect
this case now decides to sleep 60 seconds each time such a client sends
*any* protocol verb, what will happen in few months time ? Latter spam-
ware learns to behave properly per RFC 1854/2197 and not exhibit too early
write to socket before EHLO has been handled in half-duplex manner..
(Unfortunately some non-spamware system is also behaving exactly like this
 so implementing this ``signature trap'' would trap also some quite
 legitimate email..)

   Possible future of anti-spam measures:

As relay-control won't help at things like ``send this message to the local
user <linux-kernel@vger.rutgers.edu>'', and thus resulting spam thru lists,
good content analyzing system needs to be implemented. Such analysis must
be able to detect when a message is inappropriate for the recipient/system.
Of course it won't be trivial...

People are trying to do technical tests to block common spam-signatures,
like "To: Friends@Public.com" or several RFC 822 envelope headers carrying
syntactically invalid data. Some are doing (IMO inappropriate) DNS
lookups at domains on several RFC 822 headers of the message. While
such lookups are in theory ok, they do give false positives -- sometimes
because of temporary (?) DNS problems systems are rejecting email with
e.g. claim that "vger.rutgers.edu" is unknown domain/machine..

Applying evolutionary pressure test to existing rejection methods, I think
appropriate techniques are somewhere in between PROCMAIL regexprs and full
natural language AI.

Such facilities have not yet been added to ZMailer, but hooks are in place..

/Matti Aarnio

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Sun Jan 23 2000 - 21:00:16 EST