Hi Rick,
Thank you for creating and maintaining this excellent project. The design philosophy behind NVMe_CSD—hardware-agnostic, mainline Linux-based, and standards-compliant—is exactly what we believe the computational storage ecosystem needs.
Our Project
We are building a Computational Storage Device (CSD) targeting edge AI inference preprocessing. Our hardware stack is:
SoC: Rockchip RK3588 (4×Cortex-A76 + 4×Cortex-A55, 6 TOPS NPU, hardware video codec, RGA 2D accelerator)
Storage: ZNS SSD (Zoned Namespaces)
Software: Our own user-space CSD software stack that runs compute pipelines (video decode via MPP, image preprocess via RGA/NPU)
The device presents itself to the host as a standard NVMe block device with CSD extended command support, following SNIA CSD API conventions. Target applications are industrial machine vision, smart transportation, and autonomous driving data logging—scenarios where data preprocessing at the storage layer eliminates the CPU bottleneck between camera ingest and NPU inference.
Architectural Questions
We would greatly appreciate your guidance on the following:
Data Path Efficiency for High-Throughput Compute
Our preprocessing workloads involve multi-stream 1080p/4K video frames. The planned data path is: NVMe_CSD custom cmd → char device → user-space compute → char device write-back → NVMe_CSD → host DMA
We are concerned about memory copies between kernel and user space becoming the bottleneck. Have you explored or prototyped zero-copy mechanisms (e.g., DMABUF, shared hugepages) for transferring data between the pci-epf-nvme.c driver and user-space compute processes? We would be happy to contribute engineering effort in this direction if it aligns with your vision.
ZNS SSD as Backend Storage
We intend to use a Zoned Namespaces (ZNS) SSD as the backend. Our plan is to manage Zone allocation and data placement in our user-space scheduler to maximize data locality for compute tasks. Given that NVMe_CSD currently treats the backend as a generic block device via the -l parameter, do you foresee any fundamental incompatibilities with ZNS devices? Would you recommend exposing the ZNS directly, or layering a block abstraction (e.g., dm-zoned or our own FTL-like mapping) between ZNS and NVMe_CSD?
Upstreaming & Community Contributions
We plan to develop a Hardware Abstraction Layer (HAL) within our user-space stack to make the compute pipeline portable across different SoC platforms (RK3588, NVIDIA Jetson, x86 with VAAPI/QAT). We also intend to build a more sophisticated task scheduler with priority queues and resource management. If these modules prove useful beyond our specific use case, would there be interest in upstreaming or co-locating them as part of the NVMe_CSD ecosystem? We want to ensure our development direction is complementary to yours.
Looking forward to your thoughts and advice. Thank you again for this foundational work.
Best regards,
Dan ZHAO
Hi Rick,
Thank you for creating and maintaining this excellent project. The design philosophy behind NVMe_CSD—hardware-agnostic, mainline Linux-based, and standards-compliant—is exactly what we believe the computational storage ecosystem needs.
Our Project
We are building a Computational Storage Device (CSD) targeting edge AI inference preprocessing. Our hardware stack is:
SoC: Rockchip RK3588 (4×Cortex-A76 + 4×Cortex-A55, 6 TOPS NPU, hardware video codec, RGA 2D accelerator)
Storage: ZNS SSD (Zoned Namespaces)
Software: Our own user-space CSD software stack that runs compute pipelines (video decode via MPP, image preprocess via RGA/NPU)
The device presents itself to the host as a standard NVMe block device with CSD extended command support, following SNIA CSD API conventions. Target applications are industrial machine vision, smart transportation, and autonomous driving data logging—scenarios where data preprocessing at the storage layer eliminates the CPU bottleneck between camera ingest and NPU inference.
Architectural Questions
We would greatly appreciate your guidance on the following:
Data Path Efficiency for High-Throughput Compute
Our preprocessing workloads involve multi-stream 1080p/4K video frames. The planned data path is: NVMe_CSD custom cmd → char device → user-space compute → char device write-back → NVMe_CSD → host DMA
We are concerned about memory copies between kernel and user space becoming the bottleneck. Have you explored or prototyped zero-copy mechanisms (e.g., DMABUF, shared hugepages) for transferring data between the pci-epf-nvme.c driver and user-space compute processes? We would be happy to contribute engineering effort in this direction if it aligns with your vision.
ZNS SSD as Backend Storage
We intend to use a Zoned Namespaces (ZNS) SSD as the backend. Our plan is to manage Zone allocation and data placement in our user-space scheduler to maximize data locality for compute tasks. Given that NVMe_CSD currently treats the backend as a generic block device via the -l parameter, do you foresee any fundamental incompatibilities with ZNS devices? Would you recommend exposing the ZNS directly, or layering a block abstraction (e.g., dm-zoned or our own FTL-like mapping) between ZNS and NVMe_CSD?
Upstreaming & Community Contributions
We plan to develop a Hardware Abstraction Layer (HAL) within our user-space stack to make the compute pipeline portable across different SoC platforms (RK3588, NVIDIA Jetson, x86 with VAAPI/QAT). We also intend to build a more sophisticated task scheduler with priority queues and resource management. If these modules prove useful beyond our specific use case, would there be interest in upstreaming or co-locating them as part of the NVMe_CSD ecosystem? We want to ensure our development direction is complementary to yours.
Looking forward to your thoughts and advice. Thank you again for this foundational work.
Best regards,
Dan ZHAO