Skip to content

Prototype viewport-sized image decoding with full-resolution upgrades on demand #90

Description

@TheRealShek

What problem would this solve?

The viewer decodes full-size raster images even when displaying them in a much smaller fitted viewport. The 12 MP benchmark exceeds the documented 100 MB PSS target.

Scenario Settled PSS Sampled peak
Full 4000×3000 JPEG 123.2 MiB 147.0 MiB
Full 8000×6000 JPEG 226.9 MiB 295.5 MiB
Separately resized 1224×918 JPEG, same GL renderer 91.8 MiB 91.8 MiB

The resized probe used three six-second launches and the final 15 samples for steady PSS. It is a proxy for retaining fewer pixels, not an implemented progressive decoder or proof of reduced decoder-side peaks.

What should open-mpv do?

Prototype decoding to the physical pixels needed for fit view, with full-resolution upgrades when zoom or clipboard operations require them. Verify scaling support per raster loader before committing to the design.

Acceptance checks:

  • Preserve original dimensions, orientation, fit/zoom/pan behavior and fractional scaling.
  • Preserve full-resolution Copy and Quick Markup semantics.
  • Cancel/reject stale upgrades and bound simultaneous low/full-resolution buffers.
  • Measure first-frame latency, upgrade latency, steady PSS and peak memory on representative JPEG/PNG/high-bit-depth inputs.
  • Verify actual decoder-side savings: resizing only after full decoding cannot remove the initial peak.

Is there another way to solve it?

Keep full-resolution loading for formats without efficient scale support. Lower cache budgets help neighboring images but do not reduce the single-image baseline. The exploratory software-renderer result is not sufficient reason to change the renderer default.

Anything else?

Code: first-frame decode. Glycin exposes FrameRequest::scale; presence was also checked in the pinned local 3.1.0 source. The online latest API is newer, and API availability alone does not establish per-format support.

Benchmark: 2026-09-13, optimized release commit ba3cd06 (cargo build --release --locked passed), Fedora 44 GNOME/Wayland, GTK 4.22.4, GStreamer 1.28.7, GSK_RENDERER=gl. Intel Alder Lake iGPU + NVIDIA RTX 3060 Mobile; video selected Intel vavp8dec. PSS includes the app and recursive decoder children, sampled from /proc/PID/smaps_rollup about every 100 ms. Fresh-launch values are medians of three 8-second runs, using seconds 5–8; peaks are sampled maxima. Transition results use one process per budget and are exploratory. MiB means 1,048,576 bytes. Synthetic gradient JPEGs and a 1080p30 VP8 test video were used; filesystem caches were not flushed. PSS does not fully account for GPU memory. No optimization has been implemented yet.

Before submitting

  • I searched for an existing issue about this idea.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    improvementImprove existing behavior, performance, or resource use

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions