[sushmanthreddy]

What does this PR do?

Support Ovis2.5 multimodal models from AIDC-AI.

• models:
  • https://huggingface.co/ovis-community/Ovis2.5-2B
  • https://huggingface.co/ovis-community/ovis-2.5-9B
• technical report: https://huggingface.co/papers/2508.11737
• tracking issue: Add Support for Ovis2.5 Multi-Modal Model​ #40841

This PR adds native model support, checkpoint conversion, image and video processing, Auto class registrations, documentation, and tests. Converted checkpoints can be loaded without trust_remote_code.

This continues the work from the closed WIP PR #40321.

Fixes #40841

Who can review?

@zucchini-nlp @Cyrilvallez

Ovis2.5 is available in 2B and 9B variants and supports variable-resolution images, multiple images, video inputs, and multimodal reasoning. Looking forward to your feedback!

[sushmanthreddy]

cc @ArthurZucker @zucchini-nlp

[zucchini-nlp]

Thanks for working on it! We'll pass by a bit later since there are a few other model releases atm 🤗

[sushmanthreddy]

@zucchini-nlp I’ve completed an end-to-end review and validation of the changes. Could you please review the PR and let me know if any further changes are needed?
I compared the implementation with existing multimodal models in Transformers and followed the same code structure where applicable. I mainly used VideoLLaMA3 as a reference for the packed vision and image/video flow, and Qwen2-VL for the preprocessing structure.
Ovis2.5 uses a Qwen3 language backbone together with its own vision tower and visual-tokenizer mechanism.
The weight conversion follows the Transformers v5 weight-renaming and native save_pretrained flow. I validated both official 2B and 9B checkpoints with native loading using trust_remote_code=False, followed by text, image, and video inference.
I also removed redundant tests and retained the tests covering Ovis2.5-specific behavior and the standard Transformers model, processor, image, and video contracts. The complete Ovis2.5 test suite and style checks pass locally.

[zucchini-nlp]

thanks a lot, reviewing tonight or tmrw morning

[zucchini-nlp]

Oke, overall looks fine if we can push for even more modular. Especially, I didn't get why the vision block is re-inventing the wheel for some things we already have in QwenVL-family
See my comments 👇🏻 for details, that should help guide you where to copy from

Also, please lmk why we need a conversion script since it looks like we can load from existing remote-code repo

[github-actions]

[For maintainers] Suggested jobs to run (before merge)

run-slow: auto, ovis2_5

[sushmanthreddy]

@vasqu Thanks for the detailed review. I’ve gone through your comments and pushed the updates, including the merge conflict fix. Could you take another look when you have time? I’m still a bit unsure about the Ovis test files, so I’d especially appreciate a close review of their structure and coverage. Please let me know if anything else needs changing.

[vasqu]

@sushmanthreddy could we potentially first fix the tests / fast CI? Will try to check when I can but expect a bit of delays

[sushmanthreddy]

@sushmanthreddy could we potentially first fix the tests / fast CI? Will try to check when I can but expect a bit of delays

Fixed the tests @vasqu

[github-actions]

[For maintainers] Suggested jobs to run (before merge)

run-slow: auto, ovis2_5

[github-actions]

---

CI recap

Dashboard: View test results in Grafana
Latest run: 38338025774:2
Result: success | Jobs: 16 | Tests: 199,920 | Failures: 1 | Duration: 13h 47m

[zucchini-nlp]

Anton is off for a week so we will pass by in 1-2 weeks. IPing us agai then pls @sushmanthreddy

[sushmanthreddy]

sure @zucchini-nlp

[github-actions]

[For maintainers] Suggested jobs to run (before merge)

run-slow: auto, ovis2_5