Issues with Realsense D435i. IR cameras overexposed.

[pabloparamio]

I am trying to get OKVIS2-X running with my realsense D435i but am struggling a lot to get it to work. Since there isn't a config file for the D435i, I had to create my own based on the D455 file and my specific camera extrinsics.

I was able to launch okvis2-x with this command:
ros2 launch okvis okvis2x_node_realsense.launch.xml config_filename:=$HOME/okvis_ws/src/OKVIS2-X/config/D435i_new.yaml se_config_filename:=$HOME/okvis_ws/src/OKVIS2-X/config/rsD455/se2.yaml depth_image:=true

This launched correctly and without bugs, but the state estimation of the camera was quite awful (tons of drift). Looking at the feed from the IR cameras, I now see that the image looks extremely overexposed, which explains why the VIO state estimation was working so horribly.

I have attached screenshots of the IR feed in both OKVIS2-X (overexposed) and OKVIS2. Both of these were taken immediately after each other with the same lighting. OKVIS2 seems to handle exposure automatically, while OKVIS2-X seems to be fixed, causing it to be overexposed in bright areas and ok in dim-dark areas.

Has anybody else had this issue?
Does OKVIS2-X suport the D435i? Looking at the source code, i see a lot of lines mentioning the D455, but none for the D435i, which makes me doubt whether it even is supposed to work with it.

Thanks in advanced!

[SebsBarbas]

Hi @pabloparamio, we are looking into this!

Regarding the D435i, it should be usable. We always referecne the D455 since this is the sensor we have in the lab and the one we use in our internal tests. But as long as it has similar number of cameras and IMU it should be usable.

The main issue for drift are the small white dots you see in the infrared image. These are patterns emitted by the sensor to estimate the depth. The issue is that if these are visible, these are detected by the keypoint detector and then try to get matched, yielding incorrect matching (since all of them look very similar). We are looking into the Realsense driver to see if there are any major changes wrt OKVIS2 that influence this. Does the pattern appear every now and then or is it consistent?

Regarding the auto exposure, this is a controllable parameter in our config files. In OKVIS2 we had autoexposure on by default and now we let the user decide it. This can be modified in https://github.com/ethz-mrl/OKVIS2-X/blob/main/okvis_ros2/launch/okvis2x_node_realsense.launch.xml by modifying the argument in line 7 (rs_rgb_exposure). If you set it to a value of 0, you will have autoexposure on. Plese let us know if this helps you with your issues!

[SebsBarbas]

Hi! We have been looking into the issue but with a D435 (we do not have D435i). Unfortunately, we had the overexposure error in both versions of the code. Something that we have found is that the firmware version matters (we do not use the latest one).

An approach that we could recommend is that you use the ROS driver provided by the realsense and that you connect to OKVIS2-X through the subscriber nodes that we provide. We have used this method previously and it should work when all topics are correctly remapped.

Please let us know if this helps you!

[SebsBarbas]

@pabloparamio hi, we wanted to know if there are any updates on your side. Else, we will proceed to close the ticket due to inactivity.