Error when trying to do face detection (code: 2, reason: CUDA Resources could not be allocated)

[marcjasner]

I am trying to port a python project that uses dlib (to do face detection) to C++ on a Jetson Nano (Jetpack 4.6.1). The input image is in OpenCV format (cv::mat), which gets converted to a matrix<bgr_pixel>, which is then passed to the detector. I used dnn_mmod_face_detection_ex.cpp for reference, and it compiles fine, but when I run it I get the following runtime error:

terminate called after throwing an instance of 'dlib::cudnn_error'
  what():  Error while calling cudnnFindConvolutionForwardAlgorithm( context(), descriptor(data), (const cudnnFilterDescriptor_t)filter_handle, (const cudnnConvolutionDescriptor_t)conv_handle, descriptor(dest_desc), num_possible_algorithms, &num_algorithms, perf_results.data()) in file /home/marc/src/dlib_github/dlib/dlib/cuda/cudnn_dlibapi.cpp:827. code: 2, reason: CUDA Resources could not be allocated.

When I was working on the original python code I had opened an issue against a similar error I was getting on the python side (2948) and it was suggested that this may be an issue with the cuDnn install? Is that maybe what's going on here? If so, can someone please help me figure out how to reinstall cuDnn on an official Nvidia Jetson image (Ubuntu 18.04 based)?

Thanks in advance

[arrufat]

It looks like you're running out of VRAM. How big is the image?

[marcjasner]

If it hasn't been resized it SHOULD be 1280x720, but I will certainly double check that. Sent from my Verizon, Samsung Galaxy smartphone Get Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Adrià ***@***.***> Sent: Thursday, October 3, 2024 9:04:00 PM To: davisking/dlib ***@***.***> Cc: Marc Jasner ***@***.***>; Author ***@***.***> Subject: Re: [davisking/dlib] Error when trying to do face detection (code: 2, reason: CUDA Resources could not be allocated) (Discussion #3025) It looks like you're running out of VRAM. How big is the image? — Reply to this email directly, view it on GitHub<#3025 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AX7CRNQ6X2EITDUBSJKTKLTZZXSQBAVCNFSM6AAAAABPK36TWSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAOBTHA3TEMY>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[marcjasner]

Confirmed the input image is 1280x720. It gets run through pyramid up, but otherwise is unchanged. Sent from my Verizon, Samsung Galaxy smartphone Get Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Marc Jasner ***@***.***> Sent: Thursday, October 3, 2024 9:06:22 PM To: davisking/dlib ***@***.***>; davisking/dlib ***@***.***> Cc: Author ***@***.***> Subject: Re: [davisking/dlib] Error when trying to do face detection (code: 2, reason: CUDA Resources could not be allocated) (Discussion #3025) If it hasn't been resized it SHOULD be 1280x720, but I will certainly double check that. Sent from my Verizon, Samsung Galaxy smartphone Get Outlook for Android<https://aka.ms/AAb9ysg>

________________________________ From: Adrià ***@***.***> Sent: Thursday, October 3, 2024 9:04:00 PM To: davisking/dlib ***@***.***> Cc: Marc Jasner ***@***.***>; Author ***@***.***> Subject: Re: [davisking/dlib] Error when trying to do face detection (code: 2, reason: CUDA Resources could not be allocated) (Discussion #3025) It looks like you're running out of VRAM. How big is the image? — Reply to this email directly, view it on GitHub<#3025 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AX7CRNQ6X2EITDUBSJKTKLTZZXSQBAVCNFSM6AAAAABPK36TWSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAOBTHA3TEMY>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[marcjasner]

I've done some further debugging and it looks like the pyramid_up is causing the problem. If I don't pyramid up the image then it runs fine. Maybe the 1800x1800 size that was used in the example code isn't appropriate for my platform (Since my platform is embedded. It's a Jetson Nano 4gb)?

[davisking]

Yeah you just have to pick the size that gives the speed you want. With smaller images you just won't be able to find smaller objects in them is all.