Skip to content

Commit

Permalink
Revamp how to write a custom node.
Browse files Browse the repository at this point in the history
  • Loading branch information
robinjhuang committed Feb 12, 2025
1 parent 36e7ba1 commit 4066b95
Show file tree
Hide file tree
Showing 8 changed files with 107 additions and 71 deletions.
2 changes: 1 addition & 1 deletion custom-nodes/backend/lifecycle.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Installing custom nodes"
title: "Lifecycle"
---

## How Comfy loads custom nodes
Expand Down
File renamed without changes.
20 changes: 7 additions & 13 deletions custom-nodes/overview.mdx
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
title: "Custom Node Overview"
title: "Overview"
---

Custom nodes allow you to implement new features and share them with the wider community.

A custom node, like any Comfy node, essentially takes input, does something to it, and produces an output.
A custom node is like any Comfy node: it takes input, does something to it, and produces an output.
While some custom nodes perform highly complex tasks, many just do one thing. Here's an example of a
simple node that takes an image and inverts it.

Expand All @@ -13,29 +13,23 @@ simple node that takes an image and inverts it.

## Client-Server Model

Comfy runs in a client-server model. The server, written in Python, handles all the real work: data-processing,
models, image diffusion etc. The client, written in Javascript, handles the user interface.
Comfy runs in a client-server model. The server, written in Python, handles all the real work: data-processing, models, image diffusion etc. The client, written in Javascript, handles the user interface.

Comfy can also be used in API mode, in which a workflow is sent to the server by a non-Comfy client (such as
another UI, or a command line script).
Comfy can also be used in API mode, in which a workflow is sent to the server by a non-Comfy client (such as another UI, or a command line script).

Custom nodes can be placed into one of four categories:

### Server side only

The majority of Custom Nodes run purely on the server side, by defining a Python class that specifies
the input and output types, and provides a function that can be called to process inputs and produce an output.
The majority of Custom Nodes run purely on the server side, by defining a Python class that specifies the input and output types, and provides a function that can be called to process inputs and produce an output.

### Client side only

A few Custom Nodes provide a modification to the client UI, but do not add core functionality. Despite the name,
they may not even add new nodes to the system.
A few Custom Nodes provide a modification to the client UI, but do not add core functionality. Despite the name, they may not even add new nodes to the system.

### Independent Client and Server

Custom nodes may provide additional server features, and additional (related) UI features (such as a new
widget to deal with a new data type). In most cases, communication between the client and server can be handled
by the Comfy data flow control.
Custom nodes may provide additional server features, and additional (related) UI features (such as a new widget to deal with a new data type). In most cases, communication between the client and server can be handled by the Comfy data flow control.

### Connected Client and Server

Expand Down
5 changes: 5 additions & 0 deletions custom-nodes/tips.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
title: "Workflow templates"
---

### Recommended Development Lifecycle
147 changes: 91 additions & 56 deletions custom-nodes/walkthrough.mdx
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
---
title: "Walkthrough"
title: "Getting Started"
---

This page will take you step-by-step through the process of creating a custom node
that takes a batch of images, and returns one of the images. Initially, the node
This page will take you step-by-step through the process of creating a custom node.

Our example will take a batch of images, and return one of the images. Initially, the node
will return the image which is, on average, the lightest in color; we'll then extend
it to have a range of selection criteria, and then finally add some client side code.

Expand All @@ -12,21 +13,56 @@ This page assumes very little knowledge of Python or Javascript.
After this walkthrough, dive into the details of [server side code](./custom_node_server_overview),
[client side code](./custom_node_server_overview), or [client-server comms](./comms_overview).

## The Basic Node
## Write a basic node

### Prerequisites

- A working ComfyUI [installation](/get_started/manual_install). For development, we recommend installing ComfyUI manually.
- A working comfy-cli [installation](/comfy-cli/getting-started).

### Setting up

All code for this custom node will be in a single directory. So start by locating the
`custom_nodes` directory in your `ComfyUI` folder, and create a new directory in it,
named (for instance) `image_selector`. This new directory is the base directory for all
code related to the new custom node.
```bash
cd ComfyUI/custom_nodes
comfy node scaffold
```

After answering a few questions, you'll have a new directory set up.

```bash
~ % comfy node scaffold
You've downloaded .cookiecutters/cookiecutter-comfy-extension before. Is it okay to delete and re-download it? [y/n] (y): y
[1/9] full_name (): Comfy
[2/9] email ([email protected]): [email protected]
[3/9] github_username (your_github_username): comfy
[4/9] project_name (My Custom Nodepack): FirstComfyNode
[5/9] project_slug (firstcomfynode):
[6/9] project_short_description (A collection of custom nodes for ComfyUI):
[7/9] version (0.0.1):
[8/9] Select open_source_license
1 - GNU General Public License v3
2 - MIT license
3 - BSD license
4 - ISC license
5 - Apache Software License 2.0
6 - Not open source
Choose from [1/2/3/4/5/6] (1): 1
[9/9] include_web_directory_for_custom_javascript [y/n] (n): y
Initialized empty Git repository in firstcomfynode/.git/
✓ Custom node project created successfully!
```
### The Python Framework
If you start your ComfyUI server now, you should be able to find a node called `Example Node`.
The basic structure of a custom node is described in detail [later](./custom_node_server_overview).
We begin with the bare necessities:
<div align="center">
<img src="/images/example_node.png" alt="Example Node" width="150" />
</div>
```Python
### The Python class
Add the following code to the end of `src/nodes.py`:
```Python src/nodes.py
class ImageSelector:
CATEGORY = "example"
@classmethod
Expand All @@ -36,18 +72,20 @@ class ImageSelector:
FUNCTION = "choose_image"
```
<Info>The basic structure of a custom node is described in detail [here](/custom-nodes/backend/server_overview). </Info>
A custom node is a Python class, which must include these four things: `CATEGORY`,
which specifies where in the add new node menu the custom node will be located,
`INPUT_TYPES`, which is a class method defining what inputs the node will take
(see [later](./custom_node_server_overview#input-types) for details of the dictionary returned),
(see [later](./custom_nodes/backend/server_overview#input-types) for details of the dictionary returned),
`RETURN_TYPES`, which defines what outputs the node will produce, and `FUNCTION`, the name
of the function that will be called when the node is executed.
<Tip>Notice that the data type for input and output is `IMAGE` (singular) even though
we expect to receive a batch of images, and return just one. In Comfy, `IMAGE` means
image batch, and a single image is treated as a batch of size 1.</Tip>
### Add the main function
### The main function
The main function, `choose_image`, receives named arguments as defined in `INPUT_TYPES`, and
returns a `tuple` as defined in `RETURN_TYPES`. Since we're dealing with images, which are internally
Expand All @@ -64,11 +102,11 @@ this into a one dimensional tensor, of length `H*W*C`, `torch.mean()` takes the
turns a single value tensor into a Python float.
```Python
def choose_image(self, images):
brightness = list(torch.mean(image.flatten()).item() for image in images)
brightest = brightness.index(max(brightness))
result = images[brightest].unsqueeze(0)
return (result,)
def choose_image(self, images):
brightness = list(torch.mean(image.flatten()).item() for image in images)
brightest = brightness.index(max(brightness))
result = images[brightest].unsqueeze(0)
return (result,)
```
Notes on those last two lines:
Expand All @@ -77,35 +115,29 @@ Notes on those last two lines:
us `[B,H,W,C]` with `B=1`: a single image.
- in `return (result,)`, the trailing comma is essential to ensure you return a tuple.
### Deploy the node
### Register the node
To make Comfy recognize the new node, we need to turn the directory `image_selector` into a Python module,
by adding `__init__.py`, which looks like this:
To make Comfy recognize the new node, it must be available at the package level. Modify the `NODE_CLASS_MAPPINGS` variable at the end of `src/nodes.py`.
```Python
from .image_selector_node import ImageSelector
```Python src/nodes.py

NODE_CLASS_MAPPINGS = {
"Example" : Example,
"Image Selector" : ImageSelector,
}

__all__ = ['NODE_CLASS_MAPPINGS']
# Optionally, you can rename the node in the `NODE_DISPLAY_NAME_MAPPINGS` dictionary.
NODE_DISPLAY_NAME_MAPPINGS = {
"Example": "Example Node",
"Image Selector": "Image Selector",
}
```
Here we are just exporting `NODE_CLASS_MAPPINGS`,
which gives each new custom node a unique name, mapped to the class.

### Run Comfy

Start (or restart) the Comfy server and you should see, in the list of custom nodes, a line like this:

```
0.0 seconds: [your path]\ComfyUI\custom_nodes\image_selector
<Info>More information on how ComfyUI loads custom nodes is available [here](/custom-nodes/backend/lifecycle).</Info>
```
### Restart Comfy
Reload the Comfy page in your browser, and under `example` in the `Add Node` menu, you'll find `image_selector`.
If you don't, look in the Python console output for an error!
You must restart ComfyUI to see any changes.
## Add some options
Expand All @@ -114,28 +146,28 @@ choose the brightest image, or the reddest, bluest, or greenest. Edit your Pytho
so `INPUT_TYPES` looks like:
```Python
@classmethod
def INPUT_TYPES(s):
return { "required": { "images": ("IMAGE",),
"mode": (["brightest", "reddest", "greenest", "bluest"],)} }
@classmethod
def INPUT_TYPES(s):
return { "required": { "images": ("IMAGE",),
"mode": (["brightest", "reddest", "greenest", "bluest"],)} }
```
Then update the main function. We'll use a fairly naive definition of 'reddest' as being the average
`R` value of the pixels divided by the average of all three colors. So:
```Python
def choose_image(self, images, mode):
batch_size = images.shape[0]
brightness = list(torch.mean(image.flatten()).item() for image in images)
if (mode=="brightest"):
scores = brightness
else:
channel = 0 if mode=="reddest" else (1 if mode=="greenest" else 2)
absolute = list(torch.mean(image[:,:,channel].flatten()).item() for image in images)
scores = list( absolute[i]/(brightness[i]+1e-8) for i in range(batch_size) )
best = scores.index(max(scores))
result = images[best].unsqueeze(0)
return (result,)
def choose_image(self, images, mode):
batch_size = images.shape[0]
brightness = list(torch.mean(image.flatten()).item() for image in images)
if (mode=="brightest"):
scores = brightness
else:
channel = 0 if mode=="reddest" else (1 if mode=="greenest" else 2)
absolute = list(torch.mean(image[:,:,channel].flatten()).item() for image in images)
scores = list( absolute[i]/(brightness[i]+1e-8) for i in range(batch_size) )
best = scores.index(max(scores))
result = images[best].unsqueeze(0)
return (result,)
```
## Tweak the UI
Expand All @@ -154,8 +186,8 @@ and, at the end of the `choose_image` method, add a line to send a message to th
type, which should be unique, and a dictionary)
```Python
PromptServer.instance.send_sync("example.imageselector.textmessage", {"message":f"Picked image {best+1}"})
return (result,)
PromptServer.instance.send_sync("example.imageselector.textmessage", {"message":f"Picked image {best+1}"})
return (result,)
```
### Write a client extension
Expand All @@ -168,7 +200,7 @@ WEB_DIRECTORY = "./js"
__all__ = ['NODE_CLASS_MAPPINGS', 'WEB_DIRECTORY']
```
The client extension is saved as a `.js` file in the `js` subdirectory, so create `image_selector/js/image_selector.js` with the
The client extension is saved as a `.js` file in the `js` subdirectory, so create `image_selector/web/js/imageSelector.js` with the
code below. (For more, see [client side coding](./javascript_overview)).
```Javascript
Expand All @@ -190,3 +222,6 @@ and read the dictionary we sent (which is stored in `event.detail`)
Stop the Comfy server, start it again, reload the webpage, and run your workflow.
### The complete example
The complete example is available [here](https://gist.github.com/robinjhuang/fbf54b7715091c7b478724fc4dffbd03).
File renamed without changes.
Binary file added images/example_node.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 3 additions & 1 deletion mint.json
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,9 @@
"custom-nodes/js/javascript_examples"
]
},
"custom-nodes/workflow_templates"]
"custom-nodes/workflow_templates",
"custom-nodes/tips"
]
},
{
"group": "Registry",
Expand Down

0 comments on commit 4066b95

Please sign in to comment.