Skip to content

Latest commit

 

History

History
95 lines (67 loc) · 5.94 KB

README.md

File metadata and controls

95 lines (67 loc) · 5.94 KB
Raw

koi 🎣

Open In Colab Join us on Discord GitHub

koi is an open source plug-in for Krita that allows you to use AI to accelerate your art workflow!

Disclaimer ✋

In the interest of getting the open source community on board--I have released this plug-in early. In its current state you may run into issues (particularly during the setup process). If you do, I encourage you to open an issue here on GitHub and describe your problems so that it can be fixed it for you and others!

Overview 😄

The goal of this repository is to serve as a starting point for building increasingly useful tools for artists of all levels of experience to use.

Link to original twitter thread

This plug-in serves as a working example of how new A.I. models like Stable Diffusion can lower the barrier of entry to art so that anyone can enjoy making their dreams a reality!

Because this is an open source project I encourage you to try it out, break things, and come back with suggestions!

Getting Started 🏁

If you are new to git, or get stuck during the installation process, Lewington made a nice step-by-step video.

The easiest way to get started is to follow the plug-in installation process for krita. Then use the google colab backend server (button at the top of this readme)! This should give you a good introduction to the setup process and get you up and running fast!

Installation 🔨

Krita has a few plug-in installation methods, however, I will refer you to the one I use.

  • Step 1: Find your operating system's pykrita folder reference
  • Step 2: Clone the repository, and copy the koi folder and koi.desktop to pykrita. (restart krita now if it is open)
  • Step 3: Open Krita and navigate to the python plug-in menu reference
  • Step 4: Enable the koi plugin and restart Krita to load the plug-in.

The next thing you will need to do is setup the backend server that do all the computation!

  • Step 0: Ensure you have a GPU-accelerated installation of pytorch. (You can skip this step if you are using Colab or already have it)

    • Follow the installation instructions on pytorch's official getting started.

  • Step 1: Get the latest version of HuggingFace's diffusers from source by going to the GitHub repo

    • From here you can clone the repo git clone https://github.com/huggingface/diffusers.git & cd diffusers to move into the directory.
    • Install the package with pip install -e .

  • Step 2: Install this package! I recommend moving out of the diffusers folder if you haven't already (eg. cd ..)

    • git clone https://github.com/nousr/koi.git, then cd koi and pip install -e .

Note 🙋

Before continuing, make sure you accept the terms of service for the diffusers model link to do so here.

Next, inside your terminal run the huggingface-cli login command and paste a token generated from here. If you don't want to repeat this step in the future you can then run git config --global credential.helper store. (note: only do this on a computer you trust)

  • Step 3: Run the server by typing python server.py
    • If you did everything correctly you should see an address spit out after some time (eg. 127.0.0.1:8888)
  • Step 4: Open Krita, if you haven't already, and paste your address into the endpoint field of the plugin
    • You will also need to append the actual API endpoint you are using. By default this is /api/img2img
    • If you are using all of the default settings your endpoint field will look something like this http://127.0.0.1:8888/api/img2img

Inference 🖌️

This part is easy!

  • Step 1: Create a new canvas that is 512 x 512 (px) in size and make a single-layer sketch (note: these are temporary restrictions).
  • Step 2: Fill out the prompt field in the koi panel (default location is somewhere on the right of your screen).
  • Step 3: Make any additional changes you would like to the inference parameters (strength, steps, etc.)
  • Step 4: Copy and paste your server's endpoint to the associated field
  • Step 5: Click dream!

FAQ ❔

  • What does koi stand for?
    • Krita Open(source) Img2Img: While support for StableDiffusion is first, the goal is to have this plug-in be compatible with any model!
  • Why the client/server setup?
    • The goal is to make this as widely available as possible. The server can be run anywhere with a GPU (i.e. colab) and allow those with low-powered hardware to still use the plug-in!
  • I'm getting an error with "set-size"
    • This usually happens when you either forget "/api/img2img" at the end of your endpoint when copying it into the plugin OR you have some issue with your backend server (check the output on your server for more information).

TODO

  • Add colab backend example
  • Flesh out UI
  • Move to CompVis repo
  • Add CI
  • Abstract away drop-in models for the server
  • Improve documentation
  • Add DreamStudio API support
  • Add support for arbitrary canvas size & selection-based img2img
  • Add support for masks
  • Offer more configuration options