Introduction

Physical AI Bench (PAI-Bench) is a comprehensive benchmark suite for evaluating physical AI generation and understanding. PAI-Bench covers physical scenarios including autonomous vehicle (AV) driving, robotics, industry (smart space) and ego-centric everyday. PAI-Bench contains three subtasks:

• PAI-Bench-G (Video Generation): Evaluates world foundation models' ability to predict future states given current states and control signals
• PAI-Bench-C (Conditional Video Generation): Focuses on world model generation capabilities with more complex control signals such as edges, segmentation masks, depth, etc.
• PAI-Bench-U (Video Understanding): Evaluates understanding of physical scenes.

Datasets

Tasks	Data	Usage
PAI-Bench-G	🤗 physical-ai-bench-generation	Link
PAI-Bench-C	🤗 physical-ai-bench-conditional-generation	Link
PAI-Bench-U	🤗 physical-ai-bench-understanding	Link

Leaderboard

Leaderboard is available on 🤗 physical-ai-bench-leaderboard.

Citation

If you use Physical AI Bench in your research, please cite:

@misc{zhou2025paibenchcomprehensivebenchmarkphysical,
      title={PAI-Bench: A Comprehensive Benchmark For Physical AI}, 
      author={Fengzhe Zhou and Jiannan Huang and Jialuo Li and Deva Ramanan and Humphrey Shi},
      year={2025},
      eprint={2512.01989},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2512.01989}, 
}

Acknowledgements

We would like to thank NVIDIA Research, especially the Cosmos team for their support which led to the creation of PAI-Bench. We also thank Yin Cui, Jinwei Gu, Heng Wang, Prithvijit Chattopadhyay, Andrew Z. Wang, Imad El Hanafi, and Ming-Yu Liu for their valuable feedback and collaboration that helped shaped the project. This research was supported in part by National Science Foundation under Award #2427478 - CAREER Program, and by National Science Foundation and the Institute of Education Sciences, U.S. Department of Education under Award #2229873 - National AI Institute for Exceptional Education. This project was also partially supported by cyberinfrastructure resources and services provided Georgia Institute of Technology.