I'm not sure what you mean by "hard to tell who owns which service".
Every time you send a datagram, it goes through the networking code in
the kernel, which knows your UID, your GID, and your groups list. It
also knows the destination address and port. It is rather simple to
insert code which matches the address and port against a set of ACLs,
and then looks for the user in the ACL if it finds one.
(Hmm, I wonder if the folks working on the e2fs ACL support are going
to do it properly and provide an ACL module, so that any other kernel
object can use their ACL implementation for access control.)
-Marc