feat(previews): allow ffmpeg to connect direct for AWS S3 buckets

[invario]

• Resolves: fix(previews): avoid large file downloads for remote movie storage #52079

Summary

Allow preview generation on AWS S3 buckets by creating a presigned URL for the file and then passing it directly to ffmpeg which is able to handle http/https already. It can locate the moov atom even if it's at the end of the video file and it does not need to retrieve the whole file to do it.

The presigned URL expires after 10 minute which should be more than enough for this purpose.

TODO

• Need more testing.
• Open to suggestions, critiques, criticisms.

Checklist

• Code is properly formatted
• Sign-off message is added to all commits
• Tests (unit, integration, api and/or acceptance) are included
• Screenshots before/after for front-end changes
• Documentation (manuals or wiki) has been updated or is not required
• Backports requested where applicable (ex: critical bugfixes)

[invario]

Updated with a test to see if encryption is enabled on the file, and if so, do not attempt to connect direct.

[come-nc]

Looks good codewise, but needs tests.
Not sure why there is the str_starts_with test for http?

[invario]

Looks good codewise, but needs tests. Not sure why there is the str_starts_with test for http?

The test is needed because getDirectDownload varies by the external mount provider. SMB, for example, doesn't currently return anything but in the future it might return something like smb://[email protected]/foo.bar

ffmpeg doesn't support smb out of the box and I'm not even sure it would work without passing credentials directly on the command line. Same thing for ftp and potentially others.

[invario]

Fixed my commit message and force pushed.

[come-nc]

Looks good codewise, but needs tests. Not sure why there is the str_starts_with test for http?

The test is needed because getDirectDownload varies by the external mount provider. SMB, for example, doesn't currently return anything but in the future it might return something like smb://[email protected]/foo.bar

ffmpeg doesn't support smb out of the box and I'm not even sure it would work without passing credentials directly on the command line. Same thing for ftp and potentially others.

Hmm, yeah but a provider may also give an http link that does not work without credentials then? It feels a bit arbitrary to me that http* is fine and all else is not.

[invario]

Looks good codewise, but needs tests. Not sure why there is the str_starts_with test for http?

The test is needed because getDirectDownload varies by the external mount provider. SMB, for example, doesn't currently return anything but in the future it might return something like smb://[email protected]/foo.bar
ffmpeg doesn't support smb out of the box and I'm not even sure it would work without passing credentials directly on the command line. Same thing for ftp and potentially others.

Hmm, yeah but a provider may also give an http link that does not work without credentials then? It feels a bit arbitrary to me that http* is fine and all else is not.

Very true. I can add a parameter in addition to the url one. Perhaps a presigned one and check for its value?

edit: Added.

[github-actions]

Hello there,
Thank you so much for taking the time and effort to create a pull request to our Nextcloud project.

We hope that the review process is going smooth and is helpful for you. We want to ensure your pull request is reviewed to your satisfaction. If you have a moment, our community management team would very much appreciate your feedback on your experience with this PR review process.

Your feedback is valuable to us as we continuously strive to improve our community developer experience. Please take a moment to complete our short survey by clicking on the following link: https://cloud.nextcloud.com/apps/forms/s/i9Ago4EQRZ7TWxjfmeEpPkf6

Thank you for contributing to Nextcloud and we hope to hear from you soon!

(If you believe you should not receive this message, you can add yourself to the blocklist.)

[invario]

Updated to ensure the stream is seekable as supposedly not all AWS S3 streams are,

So even though I have PR #53952 in the works using a sparse file generation method, I think having this one put in place also would be worthwhile, as it allows ffmpeg to handle operations itself directly.

[invario]

Not sure if this will ever get merged but rebased and pushed just in case.

[CarlSchwan]

Not sure if this will ever get merged but rebased and pushed just in case.

Sorry got sidetracked, but it's in my personal todo list to get that and the change in #54436 in 33.

[CarlSchwan]

Rebased and updated to the new interface :)

Tested on both an s3 server mounted as external storage and an s3 server mounted as main storage

[invario]

Rebased and updated to the new interface :)

Tested on both an s3 server mounted as external storage and an s3 server mounted as main storage

Tested, worked on my dev instance with external S3 storage, thank you!

[printminion-co]

/backport to stable31

[printminion-co]

/backport to stable32