Bug: Existing Output File with Vray

[caleb-mabry]

Describe the bug

When submitting a job to my 3DSMax CMF (I am working with VRAY) using 3dsmaxbatch, the job will fail as a popup appears asking me if I want to overwrite a file. The job will hang indefinitely until I cancel it through Deadline Cloud Monitor

Expected Behaviour

I would expect the render to overwrite the existing file.

Current Behaviour

The job hangs and does not cancel itself.

Reproduction Steps

Submit a job to a specific output path.
Resubmit the same job with the specified output path and it will hang.
Delete the output path file.
Re-submit a job to the same output path.
Watch the submission succeed.

Environment

At minimum:

1. Operating system: Windows Server 2022
2. 3ds Max 2024
3. Version of this package: 0.1.5

Please share other details about your environment that you think might be relevant to reproducing the bug.

[caleb-mabry]

I will spend more time on this to see if I can create a small, reproducible file.

But if there are any temporary suggestions, I'm all ears.

[jericht]

Hi, thanks for creating this bug report! Do you have any logs you can share showing the error?

The Worker Agent should be cleaning up the session directory between sessions, so if your output file path is outside of it, you can try moving the output file path to be within the session directory.

[caleb-mabry]

@jericht here are the associated logs. I noticed Window displayed : File Exists and Window closed : File Exists.

So two different behaviors. If I RDP into the Windows instance and run .\start.bat then .\render.bat I can see the Window displayed: * lines.

2025/08/11 20:59:25 DBG: [10560] [05208] Interface8::LoadFromFile(C:\ProgramData\Amazon\OpenJD\session-5948de7059ea49d1bf9942d61668aafbkbnfgnhc\assetroot-0296407f32469695ab28\3DS\Scene\AWS_RenderTest_VRAYONLY.max)
2025/08/11 20:59:25 INF: [10560] [05208] Starting to load file: C:\ProgramData\Amazon\OpenJD\session-5948de7059ea49d1bf9942d61668aafbkbnfgnhc\assetroot-0296407f32469695ab28\3DS\Scene\AWS_RenderTest_VRAYONLY.max
2025/08/11 20:59:26 DBG: [10560] [05208] WARNING: The Populate data is missing on your system.
2025/08/11 20:59:27 INF: [10560] [05208] Done loading file:  C:\ProgramData\Amazon\OpenJD\session-5948de7059ea49d1bf9942d61668aafbkbnfgnhc\assetroot-0296407f32469695ab28\3DS\Scene\AWS_RenderTest_VRAYONLY.max 
2025/08/11 20:59:27 INF: [10560] [05208] SYSTEM: Production renderer is changed to V-Ray 7. Previous messages are cleared.
2025/08/11 20:59:27 ERR: [10560] [05208] MAXScript Evaluate MXSWrapperBase Exception: -- Unknown property: "count" in get_Item()
2025/08/11 20:59:27 ERR: [10560] [05208] MAXScript Evaluate MXSWrapperBase Exception: -- Unknown property: "count" in get_Item()
2025/08/11 21:00:25 INF: [10560] [05208] [V-Ray] ============================================================
2025/08/11 21:00:25 INF: [10560] [05208] [V-Ray] Console created, V-Ray 7.00.02 for x64 from Oct 30 2024, 16:50:11
2025/08/11 21:00:25 INF: [10560] [05208] [V-Ray] ============================================================
2025/08/11 21:00:25 INF: [10560] [05208] [V-Ray] Compiled with Microsoft Visual C++ compiler, version 19.4
2025/08/11 21:00:25 INF: [10560] [05208] [V-Ray] Host is 3ds Max, version 26
2025/08/11 21:00:25 INF: [10560] [05208] [V-Ray] V-Ray DLL version is 7.00.02
2025/08/11 21:00:26 INF: [10560] [05208]  Window displayed : File Exists
2025/08/11 21:00:26 INF: [10560] [05208]  Window closed : File Exists
2025/08/11 21:00:26 ERR: [10560] [05208] MAXScript ExecuteMAXScriptScript Exception: -- Runtime error:  Traceback (most recent call last):   File "C:\Program Files\Autodesk\3ds Max 2024\Python\lib\site-packages\deadline\max_adaptor\MaxClient\max_client.py", line 99   File "C:\Program Files\Autodesk\3ds Max 2024\Python\lib\site-packages\deadline\max_adaptor\MaxClient\max_client.py", line 94, in 'main' function   File "C:\Program Files\Autodesk\3ds Max 2024\Python\lib\site-packages\openjd\adaptor_runtime_client\base_client_interface.py", line 212, in 'poll' function   File "C:\Program Files\Autodesk\3ds Max 2024\Python\lib\site-packages\openjd\adaptor_runtime_client\base_client_interface.py", line 233, in '_perform_action' function   File "C:\Program Files\Autodesk\3ds Max 2024\Python\lib\site-packages\deadline\max_adaptor\MaxClient\render_handlers\default_max_handler.py", line 105, in 'start_render' function RuntimeError: MAXScript exception raised. -- Runtime error: Unable to open renderer, possible recursion
2025/08/11 21:00:26 ERR: [10560] [05208] MAXScript -mxs Switch Exception:  Traceback (most recent call last):   File "C:\Program Files\Autodesk\3ds Max 2024\Python\lib\site-packages\deadline\max_adaptor\MaxClient\max_client.py", line 99   File "C:\Program Files\Autodesk\3ds Max 2024\Python\lib\site-packages\deadline\max_adaptor\MaxClient\max_client.py", line 94, in 'main' function   File "C:\Program Files\Autodesk\3ds Max 2024\Python\lib\site-packages\openjd\adaptor_runtime_client\base_client_interface.py", line 212, in 'poll' function   File "C:\Program Files\Autodesk\3ds Max 2024\Python\lib\site-packages\openjd\adaptor_runtime_client\base_client_interface.py", line 233, in '_perform_action' function   File "C:\Program Files\Autodesk\3ds Max 2024\Python\lib\site-packages\deadline\max_adaptor\MaxClient\render_handlers\default_max_handler.py", line 105, in 'start_render' function RuntimeError: MAXScript exception raised. -- Runtime error: Unable to open renderer, possible recursion
2025/08/11 21:00:26 INF: [10560] [05208] StartupTemplate factoryTemplateFolder: C:\Program Files\Autodesk\3ds Max 2024\en-US\StartupTemplates
2025/08/11 21:00:26 INF: [10560] [05208] StartupTemplate userTemplateFolder: C:\Users\DeadlineWorkerUser\Documents\3ds Max 2024\startuptemplates
2025/08/11 21:00:26 INF: [10560] [05208] StartupTemplate appdataTemplateFolder: C:\Users\DeadlineWorkerUser\AppData\Local\Autodesk\3dsMax\2024 - 64bit\ENU\en-US\StartupTemplates

If I submit the job, my fleet auto-scales to handle it. The Window displayed : File Exists and Window closed : File Exists does not appear in the logs and it proceeds to hang on the line V-Ray DLL version

2025/08/11 21:08:42 INF: [11684] [09912] Done loading file:  C:\ProgramData\Amazon\OpenJD\session-16ff0be626e04eff81bed6e19fa75800bu3gydn7\assetroot-0296407f32469695ab28\3DS\Scene\AWS_RenderTest_VRAYONLY.max 
2025/08/11 21:08:42 INF: [11684] [09912] SYSTEM: Production renderer is changed to V-Ray 7. Previous messages are cleared.
2025/08/11 21:08:43 ERR: [11684] [09912] MAXScript Evaluate MXSWrapperBase Exception: -- Unknown property: "count" in get_Item()
2025/08/11 21:08:43 ERR: [11684] [09912] MAXScript Evaluate MXSWrapperBase Exception: -- Unknown property: "count" in get_Item()
2025/08/11 21:08:47 INF: [11684] [09912] [V-Ray] ============================================================
2025/08/11 21:08:47 INF: [11684] [09912] [V-Ray] Console created, V-Ray 7.00.02 for x64 from Oct 30 2024, 16:50:11
2025/08/11 21:08:47 INF: [11684] [09912] [V-Ray] ============================================================
2025/08/11 21:08:47 INF: [11684] [09912] [V-Ray] Compiled with Microsoft Visual C++ compiler, version 19.4
2025/08/11 21:08:47 INF: [11684] [09912] [V-Ray] Host is 3ds Max, version 26
2025/08/11 21:08:47 INF: [11684] [09912] [V-Ray] V-Ray DLL version is 7.00.02

[caleb-mabry]

https://help.autodesk.com/view/3DSMAX/2025/ENU/?guid=GUID-69290AFE-45B6-4206-A54B-4E1C3FFF3B89

I believe this document outlines the overwrite functionality appearing.

The dialog that displays is context-sensitive, meaning that the warning will indicate when either of these two conditions exist:
When you risk overwriting an existing file or if you have not specified an output path and output file name.

[jericht]

Thanks for the logs and the doc link! I have a hunch that the issue could be that the dialog box is not being handled correctly when we run the render in headless mode on SMF hosts. It probably worked for you because you had a GUI session via RDP.

We'll take a closer look but I think we may need to update the adaptor to handle these dialogs properly.

[leongdl]

I ran into this the other week, when we set the output path, we did not set the "override flag",

Global "Quiet Mode"
If you are doing a batch process and want to ensure nothing pops up (including scene warnings), you can wrap your code in a global quiet mode toggle:

Python
# Enable global quiet mode
pymxs.runtime.setQuietMode(True)

I will test out this fix and hopefully have a PR up.

[caleb-mabry]

Appreciate you looking into this.