Re: [ofa-general] [PATCH v3] iw_cxgb3: Support"iwarp-only"interfacestoavoid 4-tuple conflicts.

From: Steve Wise
Date: Fri Sep 28 2007 - 15:47:11 EST

Kanevsky, Arkady wrote:
IB aside,
it looks like an ULP which is capable of being both RDMA aware and RDMA
like iSER and iSCSI, NFS-RDMA and NFS, SDP and sockets, will be treated as two separete ULPs.
Each has its own IP address, since there is a different IP address for
port and "regular" Ethernet port. So it falls on the users of ULPs to
"handle" it
via DNS or some other services.
Is this "acceptable" to users? I doubt it.

Recall that ULPs are going in opposite directions by having a different
port number for RDMA aware and RDMA unaware versions of the ULP.
This way, ULP "connection manager" handles RDMA-ness under the covers,
while users plug an IP address for a server to connect to.

Arkady, I'm confused about how this proposed design changes the behavior of the ULPs that run on TCP and iWARP. I don't see much difference from the point of view of the ULPs.

The NFS-RDMA server, for example, will not need to change since it binds to address which will translate into a bind/listen on the specific iwarp address for each iwarp device on the rdma side, and address for the TCP side.

Am I missing your point?

The real pain, IMO, with this solution is that it FORCES the admins to use 2 subnets when 1 is sufficient if the net maintainers would unify the port space...


Arkady Kanevsky email: arkady@xxxxxxxxxx
Network Appliance Inc. phone: 781-768-5395
1601 Trapelo Rd. - Suite 16. Fax: 781-895-1195
Waltham, MA 02451 central phone: 781-768-5300

-----Original Message-----
From: Sean Hefty [mailto:sean.hefty@xxxxxxxxx] Sent: Thursday, September 27, 2007 3:12 PM
To: Kanevsky, Arkady; Sean Hefty; Steve Wise
Cc: netdev@xxxxxxxxxxxxxxx; rdreier@xxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx; general@xxxxxxxxxxxxxxxxxxxxx
Subject: RE: [ofa-general] [PATCH v3] iw_cxgb3: Support"iwarp-only"interfacesto avoid 4-tuple conflicts.

What is the model on how client connects, say for iSCSI, when client and server both support, iWARP and 10GbE or 1GbE, and would like to setup "most" performant "connection" for ULP?
For the "most" performance connection, the ULP would use IB, and all these problems go away. :)

This proposal is for each iwarp interface to have its own IP address. Clients would need an iwarp usable address of the server and would connect using rdma_connect(). If that call (or rdma_resolve_addr/route) fails, the client could try connecting using sockets, aoi, or some other interface. I don't see that Steve's proposal changes anything from the client's perspective.

- Sean
general mailing list

To unsubscribe, please visit

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at
Please read the FAQ at