FID, sFID, IS, Precision & Recall evaluation in PyTorch

This is a PyTorch implementation of Fréchet Inception Distance (FID), spatial Fréchet Inception Distance (sFID), Inception Score (IS), Precision & Recall evalution for image generation task.

A widely acknowledged TensorFlow implementation is guided-diffusion/evaluations

Usage

1. Download InceptionV3 wights provided by pytorch-fid

2. Prepare reference statistics:

   • for ImageNet, you can download reference npz file (ref_npz) from guided-diffusion/evaluations
   • for customized datasets, you need to prepare a sets of reference images (ref_imageset)
   • (optional) you can compress your ref_imageset into ref_npz by

   python metrics.py --weights path/to/inception_weights \
       --src-path path/to/ref_imageset --save-src-npz-path path/to/ref_npz
   

3. Generate source images with your model (src_imageset), e.g. 50000 images typically. Then calculate FID, sFID, IS, precision & recall:

python metrics.py --weights path/to/inception_weights \
    --src-path path/to/src_imageset --ref-path path/to/ref_npz
# or
python metrics.py --weights path/to/inception_weights \
    --src-path path/to/src_imageset --ref-path path/to/ref_imageset

Acknowledgements

This project is based on following projects:

• pytorch-fid
• guided-diffusion
• TTUR

License

This implementation is licensed under the Apache License 2.0.