generated from mintlify/starter
-
Notifications
You must be signed in to change notification settings - Fork 66
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
36e7ba1
commit 4066b95
Showing
8 changed files
with
107 additions
and
71 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,5 @@ | ||
--- | ||
title: "Installing custom nodes" | ||
title: "Lifecycle" | ||
--- | ||
|
||
## How Comfy loads custom nodes | ||
|
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
--- | ||
title: "Workflow templates" | ||
--- | ||
|
||
### Recommended Development Lifecycle |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,9 +1,10 @@ | ||
--- | ||
title: "Walkthrough" | ||
title: "Getting Started" | ||
--- | ||
|
||
This page will take you step-by-step through the process of creating a custom node | ||
that takes a batch of images, and returns one of the images. Initially, the node | ||
This page will take you step-by-step through the process of creating a custom node. | ||
|
||
Our example will take a batch of images, and return one of the images. Initially, the node | ||
will return the image which is, on average, the lightest in color; we'll then extend | ||
it to have a range of selection criteria, and then finally add some client side code. | ||
|
||
|
@@ -12,21 +13,56 @@ This page assumes very little knowledge of Python or Javascript. | |
After this walkthrough, dive into the details of [server side code](./custom_node_server_overview), | ||
[client side code](./custom_node_server_overview), or [client-server comms](./comms_overview). | ||
|
||
## The Basic Node | ||
## Write a basic node | ||
|
||
### Prerequisites | ||
|
||
- A working ComfyUI [installation](/get_started/manual_install). For development, we recommend installing ComfyUI manually. | ||
- A working comfy-cli [installation](/comfy-cli/getting-started). | ||
|
||
### Setting up | ||
|
||
All code for this custom node will be in a single directory. So start by locating the | ||
`custom_nodes` directory in your `ComfyUI` folder, and create a new directory in it, | ||
named (for instance) `image_selector`. This new directory is the base directory for all | ||
code related to the new custom node. | ||
```bash | ||
cd ComfyUI/custom_nodes | ||
comfy node scaffold | ||
``` | ||
|
||
After answering a few questions, you'll have a new directory set up. | ||
|
||
```bash | ||
~ % comfy node scaffold | ||
You've downloaded .cookiecutters/cookiecutter-comfy-extension before. Is it okay to delete and re-download it? [y/n] (y): y | ||
[1/9] full_name (): Comfy | ||
[2/9] email ([email protected]): [email protected] | ||
[3/9] github_username (your_github_username): comfy | ||
[4/9] project_name (My Custom Nodepack): FirstComfyNode | ||
[5/9] project_slug (firstcomfynode): | ||
[6/9] project_short_description (A collection of custom nodes for ComfyUI): | ||
[7/9] version (0.0.1): | ||
[8/9] Select open_source_license | ||
1 - GNU General Public License v3 | ||
2 - MIT license | ||
3 - BSD license | ||
4 - ISC license | ||
5 - Apache Software License 2.0 | ||
6 - Not open source | ||
Choose from [1/2/3/4/5/6] (1): 1 | ||
[9/9] include_web_directory_for_custom_javascript [y/n] (n): y | ||
Initialized empty Git repository in firstcomfynode/.git/ | ||
✓ Custom node project created successfully! | ||
``` | ||
### The Python Framework | ||
If you start your ComfyUI server now, you should be able to find a node called `Example Node`. | ||
The basic structure of a custom node is described in detail [later](./custom_node_server_overview). | ||
We begin with the bare necessities: | ||
<div align="center"> | ||
<img src="/images/example_node.png" alt="Example Node" width="150" /> | ||
</div> | ||
```Python | ||
### The Python class | ||
Add the following code to the end of `src/nodes.py`: | ||
```Python src/nodes.py | ||
class ImageSelector: | ||
CATEGORY = "example" | ||
@classmethod | ||
|
@@ -36,18 +72,20 @@ class ImageSelector: | |
FUNCTION = "choose_image" | ||
``` | ||
<Info>The basic structure of a custom node is described in detail [here](/custom-nodes/backend/server_overview). </Info> | ||
A custom node is a Python class, which must include these four things: `CATEGORY`, | ||
which specifies where in the add new node menu the custom node will be located, | ||
`INPUT_TYPES`, which is a class method defining what inputs the node will take | ||
(see [later](./custom_node_server_overview#input-types) for details of the dictionary returned), | ||
(see [later](./custom_nodes/backend/server_overview#input-types) for details of the dictionary returned), | ||
`RETURN_TYPES`, which defines what outputs the node will produce, and `FUNCTION`, the name | ||
of the function that will be called when the node is executed. | ||
<Tip>Notice that the data type for input and output is `IMAGE` (singular) even though | ||
we expect to receive a batch of images, and return just one. In Comfy, `IMAGE` means | ||
image batch, and a single image is treated as a batch of size 1.</Tip> | ||
### Add the main function | ||
### The main function | ||
The main function, `choose_image`, receives named arguments as defined in `INPUT_TYPES`, and | ||
returns a `tuple` as defined in `RETURN_TYPES`. Since we're dealing with images, which are internally | ||
|
@@ -64,11 +102,11 @@ this into a one dimensional tensor, of length `H*W*C`, `torch.mean()` takes the | |
turns a single value tensor into a Python float. | ||
```Python | ||
def choose_image(self, images): | ||
brightness = list(torch.mean(image.flatten()).item() for image in images) | ||
brightest = brightness.index(max(brightness)) | ||
result = images[brightest].unsqueeze(0) | ||
return (result,) | ||
def choose_image(self, images): | ||
brightness = list(torch.mean(image.flatten()).item() for image in images) | ||
brightest = brightness.index(max(brightness)) | ||
result = images[brightest].unsqueeze(0) | ||
return (result,) | ||
``` | ||
Notes on those last two lines: | ||
|
@@ -77,35 +115,29 @@ Notes on those last two lines: | |
us `[B,H,W,C]` with `B=1`: a single image. | ||
- in `return (result,)`, the trailing comma is essential to ensure you return a tuple. | ||
### Deploy the node | ||
### Register the node | ||
To make Comfy recognize the new node, we need to turn the directory `image_selector` into a Python module, | ||
by adding `__init__.py`, which looks like this: | ||
To make Comfy recognize the new node, it must be available at the package level. Modify the `NODE_CLASS_MAPPINGS` variable at the end of `src/nodes.py`. | ||
```Python | ||
from .image_selector_node import ImageSelector | ||
```Python src/nodes.py | ||
|
||
NODE_CLASS_MAPPINGS = { | ||
"Example" : Example, | ||
"Image Selector" : ImageSelector, | ||
} | ||
|
||
__all__ = ['NODE_CLASS_MAPPINGS'] | ||
# Optionally, you can rename the node in the `NODE_DISPLAY_NAME_MAPPINGS` dictionary. | ||
NODE_DISPLAY_NAME_MAPPINGS = { | ||
"Example": "Example Node", | ||
"Image Selector": "Image Selector", | ||
} | ||
``` | ||
Here we are just exporting `NODE_CLASS_MAPPINGS`, | ||
which gives each new custom node a unique name, mapped to the class. | ||
|
||
### Run Comfy | ||
|
||
Start (or restart) the Comfy server and you should see, in the list of custom nodes, a line like this: | ||
|
||
``` | ||
0.0 seconds: [your path]\ComfyUI\custom_nodes\image_selector | ||
<Info>More information on how ComfyUI loads custom nodes is available [here](/custom-nodes/backend/lifecycle).</Info> | ||
``` | ||
### Restart Comfy | ||
Reload the Comfy page in your browser, and under `example` in the `Add Node` menu, you'll find `image_selector`. | ||
If you don't, look in the Python console output for an error! | ||
You must restart ComfyUI to see any changes. | ||
## Add some options | ||
|
@@ -114,28 +146,28 @@ choose the brightest image, or the reddest, bluest, or greenest. Edit your Pytho | |
so `INPUT_TYPES` looks like: | ||
```Python | ||
@classmethod | ||
def INPUT_TYPES(s): | ||
return { "required": { "images": ("IMAGE",), | ||
"mode": (["brightest", "reddest", "greenest", "bluest"],)} } | ||
@classmethod | ||
def INPUT_TYPES(s): | ||
return { "required": { "images": ("IMAGE",), | ||
"mode": (["brightest", "reddest", "greenest", "bluest"],)} } | ||
``` | ||
Then update the main function. We'll use a fairly naive definition of 'reddest' as being the average | ||
`R` value of the pixels divided by the average of all three colors. So: | ||
```Python | ||
def choose_image(self, images, mode): | ||
batch_size = images.shape[0] | ||
brightness = list(torch.mean(image.flatten()).item() for image in images) | ||
if (mode=="brightest"): | ||
scores = brightness | ||
else: | ||
channel = 0 if mode=="reddest" else (1 if mode=="greenest" else 2) | ||
absolute = list(torch.mean(image[:,:,channel].flatten()).item() for image in images) | ||
scores = list( absolute[i]/(brightness[i]+1e-8) for i in range(batch_size) ) | ||
best = scores.index(max(scores)) | ||
result = images[best].unsqueeze(0) | ||
return (result,) | ||
def choose_image(self, images, mode): | ||
batch_size = images.shape[0] | ||
brightness = list(torch.mean(image.flatten()).item() for image in images) | ||
if (mode=="brightest"): | ||
scores = brightness | ||
else: | ||
channel = 0 if mode=="reddest" else (1 if mode=="greenest" else 2) | ||
absolute = list(torch.mean(image[:,:,channel].flatten()).item() for image in images) | ||
scores = list( absolute[i]/(brightness[i]+1e-8) for i in range(batch_size) ) | ||
best = scores.index(max(scores)) | ||
result = images[best].unsqueeze(0) | ||
return (result,) | ||
``` | ||
## Tweak the UI | ||
|
@@ -154,8 +186,8 @@ and, at the end of the `choose_image` method, add a line to send a message to th | |
type, which should be unique, and a dictionary) | ||
```Python | ||
PromptServer.instance.send_sync("example.imageselector.textmessage", {"message":f"Picked image {best+1}"}) | ||
return (result,) | ||
PromptServer.instance.send_sync("example.imageselector.textmessage", {"message":f"Picked image {best+1}"}) | ||
return (result,) | ||
``` | ||
### Write a client extension | ||
|
@@ -168,7 +200,7 @@ WEB_DIRECTORY = "./js" | |
__all__ = ['NODE_CLASS_MAPPINGS', 'WEB_DIRECTORY'] | ||
``` | ||
The client extension is saved as a `.js` file in the `js` subdirectory, so create `image_selector/js/image_selector.js` with the | ||
The client extension is saved as a `.js` file in the `js` subdirectory, so create `image_selector/web/js/imageSelector.js` with the | ||
code below. (For more, see [client side coding](./javascript_overview)). | ||
```Javascript | ||
|
@@ -190,3 +222,6 @@ and read the dictionary we sent (which is stored in `event.detail`) | |
Stop the Comfy server, start it again, reload the webpage, and run your workflow. | ||
### The complete example | ||
The complete example is available [here](https://gist.github.com/robinjhuang/fbf54b7715091c7b478724fc4dffbd03). |
File renamed without changes.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters