CUDA is not found in latest Docker image

[amadeuszsz]

Checklist

• I've read the contribution guidelines.
• I've searched other issues and no duplicate issues were found.
• I'm convinced that this is not my fault but a bug.

Description

Building CUDA-based packages is not possible in Docker container. This issue occurred after update Docker image.
CUDNN and TensorRT seems exist in container, reported by

dpkg -s libcudnn8
dpkg -s libnvinfer8

however cmake macros can't find them.

Expected behavior

Can build package.

Actual behavior

Can't build package.

Steps to reproduce

./docker/run.sh  --devel
cd /workspace
sudo rm -r install/tensorrt_common/ build/tensorrt_common/
colcon build --packages-up-to tensorrt_common

Output:

CMake Warning at CMakeLists.txt:15 (message):
  cuda, cudnn, tensorrt libraries are not found

It applies for all packages which use CUDNN / TensorRT.

Versions

No response

Possible causes

No response

Additional context

No response

[amadeuszsz]

@youtalk
Base on last commit I switched to 20240903-devel-cuda tag and it works. Maybe it will helpful for you for issue investigation.

[youtalk]

Oh great! Could you close this issue?

[amadeuszsz]

Oh great! Could you close this issue?

@youtalk
I'm not sure if we can close this issue. If any breaking change within Autoware dependencies appear, 20240903-devel-cuda will become incompatible. universe-devel-cuda needs to allow CUDA-related packages to be built.

Is it an unknown bug or is it related to temporary state because of changes in Docker containers?

[youtalk]

@amadeuszsz It might be an unknown bug because I expect the current autoware:base-cuda image is installed CUDA drivers https://github.com/autowarefoundation/autoware/blob/main/docker/Dockerfile#L28.

[marioney]

Hi, I'm having this same issue. I'm working with universe-devel-cuda-20240927 and when I try build autoware_lidar_centerpoint I get a message similar to the one mentioned by the OP.

Furthermore, I try to run the preinstalled autoware_lidar_centerpoint I get this error

Caught exception in launch (see debug for traceback): package 'autoware_lidar_centerpoint' found at '/opt/autoware', but libexec directory '/opt/autoware/lib/autoware_lidar_centerpoint' does not exist

[amadeuszsz]

Hi @marioney
Packages which use CUDA does not work in Docker container for now. As mentioned in previous post, please use 20240903-devel-cuda image before fix release.

[marioney]

Hi @amadeuszsz.

Thanks for the reply.

I'm using 20240903-devel-cuda in the meantime, hopefully this can be solved soon. The main inconvenience of using an older version is that I need to compile most packages locally (e.g. the full autoware.universe stack) and to figure out the right version of the other packages.

[youtalk]

Thank you for both of your feedback. The fixed images will be built within few hours.