Fell down a rabbit hole today when I was looking at a data sheet for the Broadcom 5709c chipset. "RDMA over TCP (iWARP) - RDMAC 1.0 compliant". Huh?
This blog has a good overview of RDMA:
The rationale for RDMA is laid out in great detail in RFC 4297, but the basic idea is that allowing network messages to carry information about where they should be received and allowing the NIC to place the data directly in that buffer allows fundamentally better performance. [...]
With RDMA andiSCSI Extensions for (iSER, which is RFC 5046), the target can send the data in response to a read command and have it placed directly in the receive buffer on the initiator, which saves the copy and uses 3x less memory bandwidth (which is huge if the data is running at 10Gb/sec).
How about we just remove the RDMA stack altogether? I am not at all kidding. If you guys can't stay in your sand box and need to cause problems for the normal network stack, it's unacceptable. We were told all along the if RDMA went into the tree none of this kind of stuff would be an issue.
These are exactly the kinds of problems for which people like myself were dreading. These subsystems have no buisness using the TCP port space of the Linux software stack, absolutely none.
After TCP port reservation, what's next? It seems an at least bi-monthly event that the RDMA folks need to put their fingers into something else in the normal networking stack. No more.
I will NACK any patch that opens up sockets to eat up ports or anything stupid like that.
I'm not sure if this'll be useful at work, but it's interesting to read about.