Merge branch 'dev' into main

Author: gadicc

CHANGELOG.md

@@ -1,5 +1,70 @@
 # CHANGELOG
 
+* **NEXT MAIN**
+
+  * **Diffusers v0.9.0, Stable Diffusion v2.0**.  Models:
+      * `"stabilityai/stable-diffusion-2"` - trained on 768x768
+      * `"stabilityai/stable-diffusion-2-base"` - trained on 512x512
+      * `"stabilityai/stable-diffusion-2-inpainting"` - untested
+      * `""stabilityai/stable-diffusion-x4-upscaler"` - untested
+
+    > https://github.com/huggingface/diffusers/releases
+
+  * **DPMSolverMultistepScheduler**.  Docker-diffusers-api is simply a wrapper
+    around diffusers.  We support all the included schedulers out of the box,
+    as long as they can init themselves with default arguments.  So, the above
+    scheduler was already working, but we didn't mention it before.  I'll just
+    quote diffusers:
+
+    > DPMSolverMultistepScheduler is the firecracker diffusers implementation
+    of DPM-Solver++, a state-of-the-art scheduler that was contributed by one
+    of the authors of the paper. This scheduler is able to achieve great
+    quality in as few as 20 steps. It's a drop-in replacement for the default
+    Stable Diffusion scheduler, so you can use it to essentially half
+    generation times.
+
+  * **Storage Class / S3 support**.  We now have a generic storage class, which
+    allows for special URLs anywhere anywhere you can usually specify a URL,
+    e.g. `CHECKPOINT_URL`, `dest_url` (after dreambooth training), and the new
+    `MODEL_URL` (see below).  URLs like "s3:///bucket/filename" will work how
+    you expect, but definitely read [docs/storage.md](./docs/storage.md)
+    to understand the format better.  Note in particular the triple forwardslash
+    ("///") in the beginning to use the default S3 endpoint.
+
+  * **Dreambooth training**, working but still in development.  See
+    [this forum post](https://banana-forums.dev/t/dreambooth-training-first-look/36)
+    for more info.
+
+  * **`PRECISION`** build var, defaults to `"fp16"`, set to `""` to use the model
+    defaults (generally fp32).
+
+  * **`CHECKPOINT_URL` conversion**:
+    * Crash / stop build if conversion fails (rather than unclear errors later on)
+    * Force `cpu` loading even for models that would otherwise default to GPU.
+      This fixes certain models that previously crashed in build stage (where GPU
+      is not available).
+    * `--extract-ema` on conversion since these are the more important weights for
+      inference.
+    * `CHECKPOINT_CONFIG_URL` now let's to specify a specific config file for 
+      conversion, to use instead of SD's default `v1-inference.yaml`.
+
+  * **`MODEL_URL`**.  If your model is already in diffusers format, but you don't
+    host it on HuggingFace, you can now have it downloaded at build time.  At
+    this stage, it should be a `.tar.zst` file.  This is an *alternative* to
+    `CHECKPOINT_URL` which downloads a `.ckpt` file and converts to diffusers.
+
+  * **`test.py`**:
+    * New `--banana` arg to run the test on banana.  Set environment variables
+      `BANANA_API_KEY` and `BANANA_MODEL_KEY` first.
+    * You can now add to and override a test's default json payload with:
+      * `--model-arg prompt="hello"`
+      * `--call-arg MODEL_ID="my-model"`
+    * Support for extra timing data (e.g. dreambooth sends `train`
+      and `upload` timings).
+
+  * **Dev: better caching solution**.  No more unruly `root-cache` directory.  See
+    [CONTRIBUTING.md](./CONTRIBUTING.md) for more info.
+
 * **2022-11-08**
 
   * **Much faster `init()` times!**  For `runwayml/stable-diffusion-v1-5`:

CONTRIBUTING.md

@@ -0,0 +1,90 @@
+# CONTRIBUTING
+
+*Tips for development*
+
+1. [Using Buildkit](#buildkit)
+1. [Local HTTP(S) Caching Proxy](#caching)
+1. [Local S3 Server](#local-s3-server)
+
+<a name="buildkit"></a>
+## Using BuildKit
+
+Buildkit is a docker extension that can really improve build speeds through
+caching and parallelization.  You can enable and tweak it by adding:
+
+  `DOCKER_BUILDKIT=1 BUILDKIT_PROGRESS=plain`
+
+vars before `docker build` (the `PROGRESS` var shows much more detailed
+build logs, which can be useful, but are much more verbose).  This is
+already all setup in the the [build](./build) script.
+
+<a name="caching"></a>
+## Local HTTP(S) Caching Proxy
+
+If you're only editing e.g. `app.py`, there's no need to worry about caching
+and the docker layers work amazingly.  But, if you're constantly changing
+installed packages (apt, `requirements.txt`), `download.py`, etc, it's VERY
+helpful to have a local cache:
+
+```bash
+# See all options at https://hub.docker.com/r/gadicc/squid-ssl-zero
+$ docker run -d -p 3128:3128 -p 3129:80 \
+  --name squid --restart=always \
+  -v /usr/local/squid:/usr/local/squid \
+  gadicc/squid-ssl-zero
+```
+
+and then set the docker build args `proxy=1`, and `http_proxy` / `https_proxy`
+with their respective values.
+This is already all set up in the [build](./build) script.
+
+**You probably want to fine-tune /usr/local/squid/etc/squid.conf**.
+
+It will be created after you first run `gadicc/squid-ssl-zero`.  You can then
+stop the container (`docker ps`, `docker stop container_id`), edit the file,
+and re-start (`docker start container_id`).  For now, try something like:
+
+```conf
+cache_dir ufs /usr/local/squid/cache 50000 16 256 # 50GB
+maximum_object_size 20 GB
+refresh_pattern .  52034400 50% 52034400 store-stale override-expire ignore-no-cache ignore-no-store ignore-private
+```
+
+but ideally we can as a community create some rules that don't so
+aggressively catch every single request.
+
+<a name="local-s3"></a>
+## Local S3 server
+
+If you're doing development around the S3 handling, it can be very useful to
+have a local S3 server, especially due to the large size of models.  You
+can set one up like this:
+
+```bash
+$ docker run -p 9000:9000 -p 9001:9001 \
+  -v /usr/local/minio:/data quay.io/minio/minio \
+  server /data --console-address ":9001"
+```
+
+Now point a web browser to http://localhost:9001/, login with the default
+root credentials `minioadmin:minioadmin` and create a bucket and credentials
+for testing.  More info at https://hub.docker.com/r/minio/minio/.
+
+Typical policy:
+
+```json
+{
+    "Version": "2012-10-17",
+    "Statement": [
+        {
+            "Sid": "VisualEditor0",
+            "Effect": "Allow",
+            "Action": [
+                "s3:PutObject",
+                "s3:GetObject"
+            ],
+            "Resource": "arn:aws:s3:::BUCKET_NAME/*"
+        }
+    ]
+}
+```

Dockerfile

@@ -7,6 +7,23 @@
 FROM pytorch/pytorch:1.12.1-cuda11.3-cudnn8-runtime as base
 # Below: our ideal image, but Optimization fails with it.
 #FROM continuumio/miniconda3:4.12.0 as base
+
+# Note, docker uses HTTP_PROXY and HTTPS_PROXY (uppercase)
+# We purposefully want those managed independently, as we want docker
+# to manage its own cache.  This is just for pip, models, etc.
+ARG http_proxy
+ENV http_proxy=${http_proxy}
+ARG https_proxy
+ENV https_proxy=${https_proxy}
+RUN if [ -n "$http_proxy" ] ; then \
+    echo quit \
+    | openssl s_client -proxy $(echo ${https_proxy} | cut -b 8-) -servername google.com -connect google.com:443 -showcerts \
+    | sed 'H;1h;$!d;x; s/^.*\(-----BEGIN CERTIFICATE-----.*-----END CERTIFICATE-----\)\n---\nServer certificate.*$/\1/' \
+    > /usr/local/share/ca-certificates/squid-self-signed.crt ; \
+    update-ca-certificates ; \
+  fi
+ENV REQUESTS_CA_BUNDLE=${http_proxy:+/usr/local/share/ca-certificates/squid-self-signed.crt}
+
 ENV DEBIAN_FRONTEND=noninteractive
 #RUN apt-get install gnupg2
 #RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys A4B469963BF863CC
@@ -31,31 +48,35 @@ FROM base as output
 RUN mkdir /api
 WORKDIR /api
 
+## XXXX playing around a lot.
+# pip installs pytorch 1.13 and uninstalls 1.12 (needed by xformers)
+# recomment conda update; didn't help.  need to solve above issue.
+
+RUN conda update -n base -c defaults conda
 # We need python 3.9 or 3.10 for xformers
 # Yes, we install pytorch twice... will switch base image in future
-# RUN conda update -n base -c defaults conda
 RUN conda create -n xformers python=3.10
 SHELL ["/opt/conda/bin/conda", "run", "--no-capture-output", "-n", "xformers", "/bin/bash", "-c"]
 RUN python --version
 RUN conda install -c pytorch -c conda-forge cudatoolkit=11.6 pytorch=1.12.1
 RUN conda install xformers -c xformers/label/dev
 
 # Install python packages
-RUN mkdir -p /root/.cache/pip
-COPY root-cache/pip /root/.cache/pip
-RUN pip3 install --upgrade pip
+# RUN pip3 install --upgrade pip
+RUN https_proxy="" REQUESTS_CA_BUNDLE="" conda install pip
 ADD requirements.txt requirements.txt
-RUN pip3 install -r requirements.txt
+RUN pip install -r requirements.txt
 
-# Required to build flash attention
+# Not needed anymore, but, may be needed again in the future :D
 # Turing: 7.5 (RTX 20s, Quadro), Ampere: 8.0 (A100), 8.6 (RTX 30s)
 # https://arnon.dk/matching-sm-architectures-arch-and-gencode-for-various-nvidia-cards/
-# ENV FLASH_ATTENTION=0
 # ENV TORCH_CUDA_ARCH_LIST="7.5 8.0 8.6"
-# this is built it into memory efficient attention now ! ^_^
 
-ADD install.sh .
-RUN bash install.sh
+RUN git clone https://github.com/huggingface/diffusers
+WORKDIR /api/diffusers
+RUN git checkout v0.9.0
+WORKDIR /api
+RUN pip install -e diffusers
 
 # We add the banana boilerplate here
 ADD server.py .
@@ -76,20 +97,44 @@ ENV HF_AUTH_TOKEN=${HF_AUTH_TOKEN}
 ARG MODEL_ID="runwayml/stable-diffusion-v1-5"
 ENV MODEL_ID=${MODEL_ID}
 
+# "" = model default.
+ARG PRECISION="fp16"
+ENV PRECISION=${PRECISION}
+ADD precision.py .
+
 # ARG PIPELINE="StableDiffusionInpaintPipeline"
 ARG PIPELINE="ALL"
 ENV PIPELINE=${PIPELINE}
 
-COPY root-cache/huggingface /root/.cache/huggingface
-COPY root-cache/checkpoints /root/.cache/checkpoints
-RUN du -sh /root/.cache/*
+ARG USE_DREAMBOOTH=0
+ENV USE_DREAMBOOTH=${USE_DREAMBOOTH}
 
+ARG AWS_ACCESS_KEY_ID
+ARG AWS_SECRET_ACCESS_KEY
+ARG AWS_DEFAULT_REGION
+ARG AWS_S3_ENDPOINT_URL
+ENV AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
+ENV AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
+ENV AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION}
+ENV AWS_S3_ENDPOINT_URL=${AWS_S3_ENDPOINT_URL}
+
+COPY utils utils
+
+# Download diffusers model from somewhere else (see Storage docs)
+# Don't use this for checkpoints (.ckpt)!  Use CHECKPOINT_URL for that.
+ARG MODEL_URL=""
+ENV MODEL_URL=${MODEL_URL}
 # If set, it will be downloaded and converted to diffusers format, and
 # saved in a directory with same MODEL_ID name to be loaded by diffusers.
 ARG CHECKPOINT_URL=""
 ENV CHECKPOINT_URL=${CHECKPOINT_URL}
+ARG CHECKPOINT_CONFIG_URL=""
+ENV CHECKPOINT_CONFIG_URL=${CHECKPOINT_CONFIG_URL}
+
 ADD download-checkpoint.py .
 RUN python3 download-checkpoint.py
+ARG _CONVERT_SPECIAL
+ENV _CONVERT_SPECIAL=${_CONVERT_SPECIAL}
 ADD convert-to-diffusers.py .
 RUN python3 convert-to-diffusers.py
 # RUN rm -rf checkpoints
@@ -106,7 +151,15 @@ ARG USE_PATCHMATCH=0
 RUN if [ "$USE_PATCHMATCH" = "1" ] ; then apt-get install -yqq python3-opencv ; fi
 COPY --from=patchmatch /tmp/PyPatchMatch PyPatchMatch
 
+RUN if [ "$USE_DREAMBOOTH" = "1" ] ; then \
+    # By specifying the same torch version as conda, it won't download again.
+    # Without this, it will upgrade torch, break xformers, make bigger image.
+    pip install -r diffusers/examples/dreambooth/requirements.txt bitsandbytes torch==1.12.1 ; \
+  fi
+RUN if [ "$USE_DREAMBOOTH" = "1" ] ; then apt-get install git-lfs ; fi
+
 # Add your custom app code, init() and inference()
+ADD train_dreambooth.py .
 ADD send.py .
 ADD app.py .
 

README.md

@@ -18,7 +18,14 @@ Note: This image was created for [kiri.art](https://kiri.art/).
 Everything is open source but there may be certain request / response
 assumptions.  If anything is unclear, please open an issue.
 
-## [Read the CHANGELOG for Important Updates.](./CHANGELOG.md)
+## Updates and Help
+
+* [Official `docker-diffusers-api` Forum](https://banana-forums.dev/c/open-source/docker-diffusers-api/16):
+  help, updates, discussion.
+* Subscribe ("watch") these forum topics for:
+  * [notable **`main`** branch updates](https://banana-forums.dev/t/official-releases-main-branch/35)
+  * [notable **`dev`** branch updates](https://banana-forums.dev/t/development-releases-dev-branch/53)
+* Always [check the CHANGELOG](./CHANGELOG.md) for important updates when upgrading.
 
 **Official help in our dedicated forum https://banana-forums.dev/c/open-source/docker-diffusers-api/16.**
 
@@ -48,11 +55,8 @@ serverless).
 
 **Building**
 
-1. Set `HF_AUTH_TOKEN` environment var if you haven't set it elsewhere.
 1. `docker build -t banana-sd --build-arg HF_AUTH_TOKEN=$HF_AUTH_TOKEN .`
-1. Optionally add `DOCKER_BUILDKIT=1 BUILDKIT_PROGRESS=plain` to
-    start of the line, depending on your preferences.  (Recommended if
-    you're using the `root-cache` feature.)
+1. See [CONTRIBUTING.md](./CONTRIBUTING.md) for more helpful hints.
 1. Note: your first build can take a really long time, depending on
     your PC & network speed, and *especially when using the `CHECKPOINT_URL`
     feature*.  Great time to grab a coffee or take a walk.
@@ -101,11 +105,14 @@ explicitly name `modelInputs` above, and send a bigger object (with
 
 If provided, `init_image` and `mask_image` should be base64 encoded.
 
-Available schedulers: `LMSDiscreteScheduler`, `DDIMScheduler`, `PNDMScheduler`,
-`EulerAncestralDiscreteScheduler`, `EulerDiscreteScheduler`.  These cover the
-most commonly used / requested schedulers, but we already have code in place to
-support every scheduler provided by diffusers, which will work in a later
-diffusers release when they have better defaults.
+**Schedulers**: docker-diffusers-api is simply a wrapper around diffusers,
+literally any scheduler included in diffusers will work out of the box,
+provided it can loaded with its default config and without requiring
+any other explicit arguments at init time.  In any event, the following
+schedulers are the most common and most well tested:
+`DPMSolverMultistepScheduler` (fast!  only needs 20 steps!),
+`LMSDiscreteScheduler`, `DDIMScheduler`, `PNDMScheduler`,
+`EulerAncestralDiscreteScheduler`, `EulerDiscreteScheduler`.
 
 <a name="testing"></a>
 ## Examples and testing