Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change how frames are skipped in the generator #1348

Merged
merged 4 commits into from
Aug 23, 2024

Conversation

LinasKo
Copy link
Collaborator

@LinasKo LinasKo commented Jul 12, 2024

Description

For some videos, e.g. the one in #1345, the frame generator generates no frames. This happens when video.get(cv2.CAP_PROP_POS_FRAMES, x) is called, even when x = video.get(cv2.CAP_PROP_FRAME_COUNT).

Since we use this to skip the first N frames, we can do it manually instead by retrieving and dropping them.

This has the potential to affect many codebases, so thorough tests are needed.

At the very least. this should work on:

  1. Videos in supervision.assets
  2. Videos in Bug: get_video_frames_generator does not produce any frames. #1345 Already tested.
  3. 5 videos from different online sources, ideally with different video formats.
  4. Bonus: I had some GoPro videos that had invalid frames at the start. I should test with those.

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)

How has this change been tested, please provide a testcase or example of how you tested the change?

Tested locally on Videos of #1345

Any specific deployment considerations

Docs

  • Docs updated? What were the changes:

@LinasKo
Copy link
Collaborator Author

LinasKo commented Jul 12, 2024

We'd like some help testing this! If you have the time, please run it on videos you have (any video will do!) and tell us if there are any issues. This will help us merge faster.

It's pretty simple:

  1. Install the library
pip install git+https://github.com/roboflow/supervision.git@fix/no-frames-generated-for-some-videos
  1. Run it
import supervision as sv
import cv2

path_to_video = "test2.mp4"

cv2.namedWindow("Frame", cv2.WINDOW_NORMAL)
for frame in sv.get_video_frames_generator(path_to_video):
    cv2.imshow("Frame", frame)
    
    # Press Q to stop
    if cv2.waitKey(1) == ord("q"):
        break

I'd like to know:

  1. What video format did you use?
  2. Did it play successfully?
  3. Did the playback start from the beginning, or did it skip some frames?
  4. Did you have to wait before it started?
  5. Did it reach the end?
  6. Google Drive link to video (Optional, but useful)

@LinasKo
Copy link
Collaborator Author

LinasKo commented Jul 12, 2024

Test on videos of #1345:

  1. What video format did you use? mp4
  2. Did it play successfully? Yes
  3. Did the playback start from the beginning, or did it skip some frames? Started frrom the beginning
  4. Did you have to wait before it started? No
  5. Did it reach the end? Yes
  6. Google Drive link to video: See videos in Bug: get_video_frames_generator does not produce any frames. #1345

@LinasKo LinasKo linked an issue Jul 12, 2024 that may be closed by this pull request
@LinasKo LinasKo marked this pull request as ready for review July 12, 2024 09:36
@Bhavay-2001
Copy link
Contributor

Hi @LinasKo, one question off the topic. How do you generate this pip of the branch(Step 1) that you are currently working on to install it on the Colab notebook? I am working on some other project and I wanted to test it on Colab soo...

@LinasKo
Copy link
Collaborator Author

LinasKo commented Jul 15, 2024

Here's how we do it:
!pip install git+https://github.com/roboflow/supervision.git@fix/no-frames-generated-for-some-videos

You may also find it at the very bottom of the cheatsheet

Does that answer your question?

@roboflow roboflow deleted a comment from Bhavay-2001 Jul 15, 2024
@SkalskiP
Copy link
Collaborator

SkalskiP commented Aug 7, 2024

@LinasKo Just one comment, this will make skipping large amounts of frames really slow.

@LinasKo
Copy link
Collaborator Author

LinasKo commented Aug 13, 2024

@piotr, requesting your review.
Confirming that it was terribly slow. Seeking through 1 hour of a movie took 18 sec.
This PR now avoids calling video.set(cv2.CAP_PROP_POS_FRAMES, ...) when not necessary and adds a param to enable manual seeking mode. I couldn't get the video mentioned in the Colab to work otherwise.

Colab: https://colab.research.google.com/drive/1MGoj7Imgen9zYnMqzGDcP20hRmK76DNr?usp=sharing

@LinasKo LinasKo requested a review from SkalskiP August 13, 2024 13:43
@LinasKo LinasKo merged commit 207260a into develop Aug 23, 2024
9 checks passed
stride: int = 1,
start: int = 0,
end: Optional[int] = None,
manual_seek: bool = False,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd probably call it iterative_seek.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created PR #1481 to address this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bug: get_video_frames_generator does not produce any frames.
3 participants