Support for Depth Map Sequences at the Input Level

[othmane42]

Is there a way to adapt the preprocessing step to datasets that contain depth map sequences, such as the NTU RGB+D dataset? Currently, the system lacks support for efficiently handling depth map data in input pipelines, limiting its compatibility with such multimodal settings.

I think it would be helpful to implement a new data pipeline that can seamlessly process depth map sequences alongside RGB sequences. This pipeline should take two input paths, one for the RGB clip path and another for the depth map clip-path. It should then generate two corresponding tensors: one for the RGB sequence and another for the depth map sequence. Do any of you have an idea to tweak the existing code to make it work seamlessly with this requirement?