fix(cv2): turn rotated videos upright without OpenCV

[kevin9327]

Description

Without OpenCV installed, a portrait video recorded on a phone is read sideways. With OpenCV installed, the same calls return it upright:

import supervision as sv

info = sv.VideoInfo.from_video_path("portrait_phone_clip.mp4")
frame = next(sv.get_video_frames_generator("portrait_phone_clip.mp4"))
# with OpenCV:    info.resolution_wh == (1080, 1920), frame.shape == (1920, 1080, 3)
# without OpenCV: info.resolution_wh == (1920, 1080), frame.shape == (1080, 1920, 3), turned 90 degrees

Phones usually store a portrait clip as landscape frames and record the turn in the container's display matrix. OpenCV's FFmpeg backend reads that matrix (CAP_PROP_ORIENTATION_AUTO is on by default), turns every decoded frame upright, and swaps CAP_PROP_FRAME_WIDTH and CAP_PROP_FRAME_HEIGHT to match. The PyAV fallback in supervision/_cv2/_video.py returned frame.to_ndarray() and the stream's coded size as they are, so it ignored the matrix.

This PR applies the rotation in the fallback the way OpenCV does:

• _VideoCapture.read turns each frame by VideoFrame.rotation, which PyAV reads from the same display matrix (available since PyAV 14.2, the minimum Supervision requires).
• _VideoCapture.get swaps the width and height for a quarter turn. PyAV only exposes the matrix on decoded frames, so _rotation decodes the first frame from a second handle and caches the angle. _frame_count already opens a second handle in the same way, and this leaves the capture's position untouched.
• Like OpenCV, only multiples of 90 degrees are applied. Any other angle leaves the frame alone.

Motivation and Context

The fallback is what runs for anyone without opencv-python: sv.VideoInfo.from_video_path, sv.get_video_frames_generator and sv.process_video all go through it. Phone footage is a common input for these, and without this fix models ran on sideways frames and sv.VideoSink wrote the annotated output sideways. The image path already handles the equivalent case: the fallback imread applies EXIF orientation as OpenCV does.

Changes Made

• src/supervision/_cv2/_video.py: new _quarter_turns helper, _VideoCapture._rotation, and rotation handling in _VideoCapture.get and _VideoCapture.read.
• tests/cv2/test_video.py: test_fallback_capture_turns_rotated_video_upright writes a 32×16 clip with a display rotation of 90, -90, 180 or 0 degrees (VideoStream.set_display_rotation) and expects the upright frame and size. test_fallback_capture_matches_opencv_display_rotation compares the size and first frame with cv2.VideoCapture, and is skipped without OpenCV.
• tests/utils/test_video.py: test_video_info_and_frames_follow_display_rotation checks VideoInfo.from_video_path and get_video_frames_generator on a rotated clip. It passes with OpenCV today, and it runs against the fallback in the OpenCV-free CI jobs, where tests/cv2 is skipped.
• docs/changelog.md: entry under Unreleased.

Out of scope: the writer. OpenCV's VideoWriter doesn't write a display matrix either, so the output of both backends is already the same.

Testing

I checked parity against cv2.VideoCapture (opencv-python 5.0.0) for rotations of 90, -90, 180 and 270 degrees, for 45 degrees, and for matrices with a horizontal or vertical flip. The fallback's size and first frame matched OpenCV's exactly in every case.

New tests on develop, with OpenCV installed:

E       assert (32.0, 16.0) == (16.0, 32.0)
E         At index 0 diff: 32.0 != 16.0
FAILED tests/cv2/test_video.py::test_fallback_capture_turns_rotated_video_upright[counterclockwise-quarter-turn]
FAILED tests/cv2/test_video.py::test_fallback_capture_turns_rotated_video_upright[clockwise-quarter-turn]
FAILED tests/cv2/test_video.py::test_fallback_capture_turns_rotated_video_upright[half-turn]
FAILED tests/cv2/test_video.py::test_fallback_capture_matches_opencv_display_rotation[90]
FAILED tests/cv2/test_video.py::test_fallback_capture_matches_opencv_display_rotation[-90]
FAILED tests/cv2/test_video.py::test_fallback_capture_matches_opencv_display_rotation[180]
6 failed, 2 passed

Without OpenCV (sys.modules["cv2"] = None, as in the OpenCV-free CI jobs):

E       assert (32, 16) == (16, 32)
E         At index 0 diff: 32 != 16
FAILED tests/utils/test_video.py::test_video_info_and_frames_follow_display_rotation
4 failed, 1 passed, 3 skipped

With this change:

$ pytest tests/cv2/test_video.py tests/utils/test_video.py -k rotat
8 passed
$ pytest tests/cv2 tests/utils          # without OpenCV
281 passed, 22 skipped      (develop: 276 passed, 19 skipped)
$ pytest            # src doctests + tests, OpenCV installed
4113 passed, 1 skipped      (develop: 4105 passed, 1 skipped)
$ ruff check && ruff format --check && docformatter --check   # changed files
All checks passed!
3 files already formatted

mypy on the changed source and test files reports the same errors on develop and on this branch.

• Added/updated tests, and the full suite passes locally
• Updated docs (docstrings / mkdocs entry) for new or changed public API
• Added a changelog entry in docs/changelog.md under Unreleased (skip for lint/type/format-only or pure doc changes)

Additional Notes

The test fixtures use VideoStream.set_display_rotation, which PyAV added in 17.1. uv.lock pins 17.1.0 and 18.0.0, so CI has it. The fix itself only needs VideoFrame.rotation, which PyAV has had since 14.2.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91%. Comparing base (eb58b7b) to head (dc1ff44).
⚠️ Report is 1 commits behind head on develop.

Additional details and impacted files

@@           Coverage Diff           @@
##           develop   #2601   +/-   ##
=======================================
  Coverage       91%     91%           
=======================================
  Files           78      78           
  Lines        11388   11408   +20     
=======================================
+ Hits         10387   10407   +20     
  Misses        1001    1001           

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

🟡 Changes recommended

The non-quarter-turn behavior lacks regression coverage, and the changelog modification date is stale.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds PyAV fallback support for video display rotation, matching OpenCV behavior.

Changes:

• Rotates decoded frames and reports upright dimensions.
• Adds fallback and integration tests.
• Documents the fix in the changelog.

Review scores: Code 5/5 · Tests 4/5 · Docs 4/5

File summaries

File	Description
src/supervision/_cv2/_video.py	Applies and caches display rotation metadata.
tests/cv2/test_video.py	Tests fallback rotation and OpenCV parity.
tests/utils/test_video.py	Tests public video APIs with rotated input.
docs/changelog.md	Adds the user-facing fix entry.

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 2
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.