GPSDiffusion-Object-Shadow-Generation

This is the official repository for the following paper:

Shadow Generation Using Diffusion Model with Geometry Prior

Haonan Zhao, Qingyang Liu, Xinhao Tao, Li Niu, Guangtao Zhai
Accepted by CVPR 2025.

Object shadow generation aims to generate plausible shadow for the inserted object in the composite image. We propose Geometry Prior guided Shadow generation Diffusion model (GPSDifffusion), which significantly improves the shadow geometry. The visual comparision between SGDiffusion and our GPSDiffusion is shown below. From left to right, we show the composite image, foreground mask, the result of SGDiffusion, the result of our GPSDiffusion, and ground-truth.

We also provide the visual comparison with other baselines below.

GPSDiffusion has been integrated into our image composition toolbox libcom. Note that the performance of GPSDiffusion is unstable, so you need to generate multiple results and pick the most satisfactory one.

Installation

• Clone this repo: git clone https://github.com/bcmi/GPSDiffusion-Object-Shadow-Generation.git
• Download the DESOBAv2 dataset from [Baidu Cloud] (access code: bcmi) or [One Drive]. Unzip desobav2-256x256.rar to ./data/, and rename it to desobav2.
• Download the checkpoints from [Baidu Cloud] (access code: bcmi). Unzip pretrained_models.zip to ./models/.

Environment

conda create -n GPSDiffusion python=3.8
conda activate GPSDiffusion
pip install -r requirements.txt

Training

python train_GPSDiffusion.py

Inference

python infer_GPSDiffusion.py

Post-processing

python post_processing.py

Evaluation

python eval.py

Other Resources

• Awesome-Object-Shadow-Generation
• Awesome-Image-Composition