RoCEE is RDMA over converged enhanced ethernet standard it is infiniband transport over ethernet, its highly light-weight transport, layered directly over Ethernet L2. Think FCoE equivalent for high performance IPC traffic. RDMA reduces endpoint latency by transferring data directly from the memory of one computer straight to the memory of another without being slowed by the operating system or the NIC memory. RoCEE should bring endpoint latency down from about 4.5 us that is standard with Ethernet to about 1.3 us. RoCEE layers InfiniBand’s layer 2 / 3 protocols on top of Ethernet’s physical and MAC layers.
RoCEE packets are standard Ethernet frames with an IEEE assigned Ethertype, a GRH, unmodified IB transport headers and payload. IB subnet management and SA services are not required for RoCEE operation; Ethernet management practices are used instead. RoCEE encodes IP addresses into its GIDs and resolves MAC addresses using the host IP stack. For multicast GIDs, standard IP to MAC mappings apply.
The OFA RDMA Verbs API is syntactically unmodified. The CMA is adapted to support RoCEE ports allowing existing RDMA applications to run over RoCEE with no changes.
Ron, come on, you can do better. Tell us something interesting. As an API, ibverbs is very difficult to use even for the experienced. RocEE is terrific but even better would be to get an API that allows programmers to use the tech without having to learn the complete nuts and bolts of ibverbs. I'm working on one but I have decided to make it C++ only (someone can JNI it for Java if they want). Keeping it brain dead simple and highly efficient is not an easy task.
ReplyDeleteJeff remember DAPL (Direct Access Programming Library) it was the result of an attempt early on to provide a common API for both InfiniBand and iWARP (Ethernet) devices. For whatever reason, DAPL just never really seem to have caught on.
ReplyDelete1. Calling the technology “RDMA over” instead of “InfiniBand over” is overly vague and intentionally deceptive. We already have “RDMA over Ethernet” — except we’ve been calling it iWARP. Choosing “RoCE” is somewhat like talking about “Storage over Ethernet” instead of “Fibre Channel over Ethernet.” Sure, FCoE is storage over ethernet, but so is iSCSI.
ReplyDelete