Network is one of the bottlenecks influencing the Ceph performance. Especially the Ceph cluster network requires a high throughput and a low latency to speedup the I/O operations. The performance can be increased employing a shared memory communication (SMC).

SMC is a separate and fast communication channel implemented in the server hardware. While SMC-R (Shared Memory Communications over Remote Direct Memory Access) use Ethernet, SMC-D (Shared Memory Communications - Direct Memory Access) does employ the hardware shared memory. SMC-D channel does offload Ethernet and increase the Ceph cluster network throughput resulting in higher Ceph I/O operations.

We explain in detail SMC-D stack (the user space, the kernel space, and the firmware level), implementation in Ceph, and integration tests. We detail the performance analysis, namely Ceph setup to generate a high load on the cluster network, the client load, and the whole stack performance analysis. Performance analysis protocol and the results will be presented in tables and graphs. Advantages and disadvantages of the SMC-D channel will be given too. Implementation of SMC-D in Ceph demonstrate a significant I/O throughput increase.