Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Broken if JC2 Node is installed, model alway loading on default CUDA device, no matter what device is chosen in the loader nodes #34

Open
ChiliBen opened this issue Mar 7, 2025 · 2 comments
Open

Broken if JC2 Node is installed, model alway loading on default CUDA device, no matter what device is chosen in the loader nodes #34

ChiliBen opened this issue Mar 7, 2025 · 2 comments
Assignees
@ChiliBen

Comments

@ChiliBen
Copy link

ChiliBen commented Mar 7, 2025

By installing node https://github.com/TTPlanetPig/Comfyui_JC2 (currently node 137 in ComfyUI Manager), MultiGPU stops working, and all models and operations are executed on the default GPU, even if the JC2 (joycaption alpha 2) node is not even part of the workflow.

Disabling the JC2 node and restarting ComfyUI fixes the issue.

I am aware, that this is likely not something which can be fixed in this codebase, but thought I'd mention the issue here as well, in case someone desperately looks for a solution, and because the JC2 node keeps working, while this node is breaking. Also, I'm pretty sure the JC2 repo is "dead"

In my opinion, a suitable alert in ComfyUI manager and/or the logs should be added to warn users accordingly, if JC2 node is detected at startup.
@pollockjj pollockjj self-assigned this Mar 7, 2025
@pollockjj
Copy link
Owner

I am willing to take a look. Can you give me a full dump of the log that fails and your workflow?

@pollockjj pollockjj assigned ChiliBen and unassigned pollockjj Mar 7, 2025
@ChiliBen
Copy link
Author

ChiliBen commented Mar 7, 2025 •
edited
Loading

From a fresh ComfyUI install, in a clean venv, with ONLY MultiGPU and JC2 nodes installed:

#Starting ComfyUI form venv using main.py  [JC2 node installed]:

(venv) llm@llm-server:~/storage/comfy2/ComfyUI$ python main.py --listen 0.0.0.0 --port 1214

#ComfyUI Startup sequence - everything looks normal:

Checkpoint files will always be loaded safely.
Total VRAM 40326 MB, total RAM 257402 MB
pytorch version: 2.6.0+cu126
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA A100-SXM4-40GB : cudaMallocAsync
Using pytorch attention
ComfyUI version: 0.3.24
[Prompt Server] web root: /home/llm/storage/comfy2/venv/lib/python3.11/site-packages/comfyui_frontend_package/static
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/ComfyUI-LTXVideo
MultiGPU: Module ComfyUI-LTXVideo not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/comfyui-ltxvideo
MultiGPU: Module comfyui-ltxvideo not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/ComfyUI-Florence2
MultiGPU: Module ComfyUI-Florence2 not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/comfyui-florence2
MultiGPU: Module comfyui-florence2 not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/ComfyUI_bitsandbytes_NF4
MultiGPU: Module ComfyUI_bitsandbytes_NF4 not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/comfyui_bitsandbytes_nf4
MultiGPU: Module comfyui_bitsandbytes_nf4 not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/x-flux-comfyui
MultiGPU: Module x-flux-comfyui not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/x-flux-comfyui
MultiGPU: Module x-flux-comfyui not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/ComfyUI-MMAudio
MultiGPU: Module ComfyUI-MMAudio not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/comfyui-mmaudio
MultiGPU: Module comfyui-mmaudio not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/ComfyUI-GGUF
MultiGPU: Module ComfyUI-GGUF not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/comfyui-gguf
MultiGPU: Module comfyui-gguf not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/PuLID_ComfyUI
MultiGPU: Module PuLID_ComfyUI not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/pulid_comfyui
MultiGPU: Module pulid_comfyui not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/ComfyUI-HunyuanVideoWrapper
MultiGPU: Module ComfyUI-HunyuanVideoWrapper not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/comfyui-hunyuanvideowrapper
MultiGPU: Module comfyui-hunyuanvideowrapper not found - skipping
MultiGPU: Registration complete. Final mappings: DeviceSelectorMultiGPU, HunyuanVideoEmbeddingsAdapter, MergeFluxLoRAsQuantizeAndLoaddMultiGPU, UNETLoaderMultiGPU, VAELoaderMultiGPU, CLIPLoaderMultiGPU, DualCLIPLoaderMultiGPU, TripleCLIPLoaderMultiGPU, CheckpointLoaderSimpleMultiGPU, ControlNetLoaderMultiGPU

Import times for custom nodes:
   0.0 seconds: /home/llm/storage/comfy2/ComfyUI/custom_nodes/websocket_image_save.py
   0.0 seconds: /home/llm/storage/comfy2/ComfyUI/custom_nodes/ComfyUI-MultiGPU
   0.0 seconds: /home/llm/storage/comfy2/ComfyUI/custom_nodes/Comfyui_JC2

Starting server

To see the GUI go to: http://0.0.0.0:1214

#Prompt with cuda set to cuda0:   --> execution on A100 (cuda0)

got prompt
model weight dtype torch.float16, manual cast: None
model_type EPS
Using pytorch attention in VAE
Using pytorch attention in VAE


VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16    # <-----cuda0!!!


/home/llm/storage/comfy2/venv/lib/python3.11/site-packages/transformers/tokenization_utils_base.py:1617: FutureWarning: `clean_up_tokenization_spaces` was not set. It will be set to `True` by default. This behavior will be deprecated in transformers v4.45, and will be then set to `False` by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884
  warnings.warn(
CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
Requested to load SDXLClipModel
loaded completely 38684.175 1560.802734375 True
Requested to load SDXL
loaded completely 37009.24715881348 4897.0483474731445 True
100%|██████████████████████████████████████████████████████████████████| 25/25 [00:04<00:00,  5.05it/s]
Requested to load AutoencoderKL
loaded completely 28565.350997924805 159.55708122253418 True
Prompt executed in 7.54 seconds

#Prompt with cuda in checkpoint loader set to cuda1: --> also execution on A100 (cuda0)

got prompt
model weight dtype torch.float16, manual cast: None
model_type EPS
Using pytorch attention in VAE
Using pytorch attention in VAE


VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16     #<-----cuda0!!!


CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
Requested to load SDXLClipModel
loaded completely 38552.05 1560.802734375 True
Requested to load SDXL
loaded completely 36991.24715881348 4897.0483474731445 True
100%|██████████████████████████████████████████████████████████████████| 25/25 [00:04<00:00,  5.58it/s]
Requested to load AutoencoderKL
loaded completely 28557.350997924805 159.55708122253418 True
Prompt executed in 6.69 seconds

^C
Stopped server

#ComfyUI server stopped, removing JC2 custom node:

(venv) llm@llm-server:~/storage/comfy2/ComfyUI$ rm -rf custom_nodes/Comfyui_JC2/

#Starting ComfyUI form venv using main.py  [JC2 node installed]:

(venv) llm@llm-server:~/storage/comfy2/ComfyUI$ python main.py --listen 0.0.0.0 --port 1214

Checkpoint files will always be loaded safely.
Total VRAM 40326 MB, total RAM 257402 MB
pytorch version: 2.6.0+cu126
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA A100-SXM4-40GB : cudaMallocAsync
Using pytorch attention
ComfyUI version: 0.3.24
[Prompt Server] web root: /home/llm/storage/comfy2/venv/lib/python3.11/site-packages/comfyui_frontend_package/static
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/ComfyUI-LTXVideo
MultiGPU: Module ComfyUI-LTXVideo not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/comfyui-ltxvideo
MultiGPU: Module comfyui-ltxvideo not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/ComfyUI-Florence2
MultiGPU: Module ComfyUI-Florence2 not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/comfyui-florence2
MultiGPU: Module comfyui-florence2 not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/ComfyUI_bitsandbytes_NF4
MultiGPU: Module ComfyUI_bitsandbytes_NF4 not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/comfyui_bitsandbytes_nf4
MultiGPU: Module comfyui_bitsandbytes_nf4 not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/x-flux-comfyui
MultiGPU: Module x-flux-comfyui not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/x-flux-comfyui
MultiGPU: Module x-flux-comfyui not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/ComfyUI-MMAudio
MultiGPU: Module ComfyUI-MMAudio not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/comfyui-mmaudio
MultiGPU: Module comfyui-mmaudio not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/ComfyUI-GGUF
MultiGPU: Module ComfyUI-GGUF not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/comfyui-gguf
MultiGPU: Module comfyui-gguf not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/PuLID_ComfyUI
MultiGPU: Module PuLID_ComfyUI not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/pulid_comfyui
MultiGPU: Module pulid_comfyui not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/ComfyUI-HunyuanVideoWrapper
MultiGPU: Module ComfyUI-HunyuanVideoWrapper not found - skipping
MultiGPU: Checking for module at /home/llm/storage/comfy2/ComfyUI/custom_nodes/comfyui-hunyuanvideowrapper
MultiGPU: Module comfyui-hunyuanvideowrapper not found - skipping
MultiGPU: Registration complete. Final mappings: DeviceSelectorMultiGPU, HunyuanVideoEmbeddingsAdapter, MergeFluxLoRAsQuantizeAndLoaddMultiGPU, UNETLoaderMultiGPU, VAELoaderMultiGPU, CLIPLoaderMultiGPU, DualCLIPLoaderMultiGPU, TripleCLIPLoaderMultiGPU, CheckpointLoaderSimpleMultiGPU, ControlNetLoaderMultiGPU

Import times for custom nodes:
   0.0 seconds: /home/llm/storage/comfy2/ComfyUI/custom_nodes/websocket_image_save.py
   0.0 seconds: /home/llm/storage/comfy2/ComfyUI/custom_nodes/ComfyUI-MultiGPU

Starting server

To see the GUI go to: http://0.0.0.0:1214

#Prompt with cuda set to cuda0:   --> execution on A100 (cuda0)

got prompt
model weight dtype torch.float16, manual cast: None
model_type EPS
Using pytorch attention in VAE
Using pytorch attention in VAE


VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16     #<-----cuda0!!!


/home/llm/storage/comfy2/venv/lib/python3.11/site-packages/transformers/tokenization_utils_base.py:1617: FutureWarning: `clean_up_tokenization_spaces` was not set. It will be set to `True` by default. This behavior will be deprecated in transformers v4.45, and will be then set to `False` by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884
  warnings.warn(
CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
Requested to load SDXLClipModel
loaded completely 38684.175 1560.802734375 True
Requested to load SDXL
loaded completely 37009.24715881348 4897.0483474731445 True
100%|██████████████████████████████████████████████████████████████████| 25/25 [00:04<00:00,  5.03it/s]
Requested to load AutoencoderKL
loaded completely 28565.350997924805 159.55708122253418 True
Prompt executed in 7.84 seconds

#Prompt with cuda set to cuda1:   --> execution on RTX3090 (cuda1)

got prompt
model weight dtype torch.float16, manual cast: None
model_type EPS
Using pytorch attention in VAE
Using pytorch attention in VAE


VAE load device: cuda:1, offload device: cpu, dtype: torch.bfloat16     #<-----cuda1!!!


CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
Requested to load SDXLClipModel
loaded completely 38552.05 1560.802734375 True
Requested to load SDXL
loaded completely 22496.54990081787 4897.0483474731445 True
100%|██████████████████████████████████████████████████████████████████| 25/25 [00:06<00:00,  3.94it/s]
Requested to load AutoencoderKL
loaded completely 13994.5287399292 159.55708122253418 True
Prompt executed in 8.83 seconds

As you can see, there isn't ANY indicator that something is wrong. The only way to notice is by directly checking VRAM usage and processes spawned on the respective cards. (nvidia-smi)

Only way I found the issue is by manually disabling all custom nodes, then switching them on one by one. I then restarted+reloaded ComfyUI and ran the attached example workflow.

I'm 99% confident the issue is with the JC2 node and not yours. Among other things it is loading the model into VRAM twice when used more than once in the same workflow, so I strongly suspect it does something weird in how it handles GPU assignment.

used workflow, though I also checked in my main install with your provided flux1dev_2gpu.json:

MultiGPU_test.json

Edited to improve formatting

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ChiliBen ChiliBen

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@pollockjj @ChiliBen