🌟NNSmith🌟 is a random DNN generator and a fuzzing infrastructure, primarily designed for automatically validating deep-learning frameworks and compilers.
| Models | tvm |
pt2 |
torchjit |
tensorrt |
onnxruntime |
xla |
tflite |
|---|---|---|---|---|---|---|---|
| ONNX | ✅ | ✅ | ✅ | ||||
| PyTorch | 🔨 | ✅ | ✅ | ||||
| TensorFlow | 🔨 | ✅ | ✅ |
✅: Supported; 🔨: Coming soon;
Install latest code (GitHub HEAD):
pip install "git+https://github.com/ise-uiuc/nnsmith@main#egg=nnsmith[torch,onnx]" --upgrade
# [optional] add more front- and back-ends such as [tensorflow] and [tvm,onnxruntime,xla,...] in "[...]"Install latest stable release [click]
pip install "nnsmith[torch,onnx]" --upgradeInstall latest pre-release [click]
pip install "nnsmith[torch,onnx]" --upgrade --preSetting up graphviz for debugging [click]
Graphviz provides dot for visualizing graphs in nice pictures. But it needs to be installed via the following methods:
sudo apt-get install graphviz graphviz-dev # Linux
brew install graphviz # MacOS
conda install --channel conda-forge pygraphviz # Conda
choco install graphviz # Windows
pip install pygraphviz # Final step.Also see pygraphviz install guidance.
# Generate a random model in "nnsmith_outputs/*"
nnsmith.model_gen model.type=onnx debug.viz=true- Bugs: links to reports.
- Documentation: CLI, concept, logging, and known issues.
- Contributions:
doc/CONTRIBUTING.md - We use hydra to manage configurations. See
nnsmith/config/main.yaml.
📜 NNSmith: Generating Diverse and Valid Test Cases for Deep Learning Compilers. [click :: citation]
@inproceedings{liu2023nnsmith,
title={Nnsmith: Generating diverse and valid test cases for deep learning compilers},
author={Liu, Jiawei and Lin, Jinkun and Ruffy, Fabian and Tan, Cheng and Li, Jinyang and Panda, Aurojit and Zhang, Lingming},
booktitle={Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2},
pages={530--543},
year={2023}
}
