Segmentation fault in CUDA

[ha7sh17]

dlib ver : 19.24
server : NVIDIA L4

---

We are heavily using your dlib for face detection in a CUDA environment.
We sincerely appreciate your work.

We are using the mmod face detector with the following typical setup.

using MmodFaceDetector = loss_mmod<con<1,
                                       9,
                                       9,
                                       1,
                                       1,
                                       rcon5<rcon5<rcon5<downsampler<input_rgb_image_pyramid<pyramid_down<6>>>>>>>>;

Occasionally, when the GPU is under heavy load, we receive a SIGSEGV signal, causing the program to terminate, along with the following backtrace. (about pyramid_down)

Based on the backtrace, do you have any insights or possible reasons for this issue?

SIGSEGV Backtrace: 0 : /usr/lib64/libfacedetect.so.1.0.0(_Z14HANDLE_SIGSEGVi+0xe9) [0x7fab94055889] | 1 : /usr/lib64/libc.so.6(+0x54db0) [0x7fab93800db0] | 2 : /usr/lib64/libfacedetect.so.1.0.0(_ZNK4dlib23input_rgb_image_pyramidINS_12pyramid_downILj6EEEE9to_tensorIPKNS_6matrixINS_9rgb_pixelELl0ELl0ENS_33memory_manager_stateless_kernel_1IcEENS_16row_major_layoutEEEEEvT_SD_RNS_16resizable_tensorE+0x16c6) [0x7fab941c1e86] | 3 : /usr/lib64/libfacedetect.so.1.0.0(_ZNK4dlib9add_layerINS_4con_ILl16ELl5ELl5ELi2ELi2ELi0ELi0EEENS_23input_rgb_image_pyramidINS_12pyramid_downILj6EEEEEvE9to_tensorIPKNS_6matrixINS_9rgb_pixelELl0ELl0ENS_33memory_manager_stateless_kernel_1IcEENS_16row_major_layoutEEEEEvT_SH_RNS_16resizable_tensorE+0x21) [0x7fab941c1fc1] | 4 : /usr/lib64/libfacedetect.so.1.0.0(_ZN4dlib14add_loss_layerINS_10loss_mmod_ENS_9add_layerINS_4con_ILl1ELl9ELl9ELi1ELi1ELi4ELi4EEENS2_INS_5relu_ENS2_INS_7affine_ENS2_INS3_ILl45ELl5ELl5ELi1ELi1ELi2ELi2EEENS2_IS5_NS2_IS6_NS2_IS7_NS2_IS5_NS2_IS6_NS2_IS7_NS2_IS5_NS2_IS6_NS2_INS3_ILl32ELl5ELl5ELi2ELi2ELi0ELi0EEENS2_IS5_NS2_IS6_NS2_IS8_NS2_IS5_NS2_IS6_NS2_INS3_ILl16ELl5ELl5ELi2ELi2ELi0ELi0EEENS_23input_rgb_image_pyramidINS_12pyramid_downILj6EEEEEvEEvEEvEEvEEvEEvEEvEEvEEvEEvEEvEEvEEvEEvEEvEEvEEvEEvEEvEEEclIPKNS_6matrixINS_9rgb_pixelELl0ELl0ENS_33memory_manager_stateless_kernel_1IcEENS_16row_major_layoutEEEPSt6vectorINS_9mmod_rectESaIS18_EEEEvT_S1C_T0_+0xb9) [0x7fab941c24a9] | 5 : /usr/lib64/libfacedetect.so.1.0.0(_ZN8CudaCrop21processMmodFaceDetectERN4dlib14add_loss_layerINS0_10loss_mmod_ENS0_9add_layerINS0_4con_ILl1ELl9ELl9ELi1ELi1ELi4ELi4EEENS3_INS0_5relu_ENS3_INS0_7affine_ENS3_INS4_ILl45ELl5ELl5ELi1ELi1ELi2ELi2EEENS3_IS6_NS3_IS7_NS3_IS8_NS3_IS6_NS3_IS7_NS3_IS8_NS3_IS6_NS3_IS7_NS3_INS4_ILl32ELl5ELl5ELi2ELi2ELi0ELi0EEENS3_IS6_NS3_IS7_NS3_IS9_NS3_IS6_NS3_IS7_NS3_INS4_ILl16ELl5ELl5ELi2ELi2ELi0ELi0EEENS0_23input_rgb_image_pyramidINS0_12pyramid_downILj6EEEEEvEEvEEvEEvEEvEEvEEvEEvEEvEEvEEvEEvEEvEEvEEvEEvEEvEEvEEvEEEERNS0_6matrixINS0_9rgb_pixelELl0ELl0ENS0_33memory_manager_stateless_kernel_1IcEENS0_16row_major_layoutEEEb+0x1c2) [0x7fab941a0342] | 6 : /usr/lib64/libfacedetect.so.1.0.0(_ZN8CudaCrop15detectFaceAlignEP9PhoBufferRjS2_+0x1df) [0x7fab941a0c1f] | 7 : /usr/lib64/libfacedetect.so.1.0.0(_ZN8CudaCrop21resolveCropAlignPointEP9PhoBufferRjS2_+0x376) [0x7fab941a1316] | 8 : /usr/lib64/libfacedetect.so.1.0.0(_ZN8CudaCrop7executeEjRSt10shared_ptrI15PhoPooledObjectIP9PhoBufferEE+0xd5) [0x7fab941a1535] | 9 : /usr/lib64/libfacedetect.so.1.0.0(_ZN10PhoElement25exceptionThrowableExecuteEv+0x128) [0x7fab940e69f8] |