Skip to content

Fix: Fix Multi-Video Stream Playback and Internal Frame Stealing - #139

Open
sqrrrl wants to merge 4 commits into
mainfrom
fix-multi-video
Open

Fix: Fix Multi-Video Stream Playback and Internal Frame Stealing#139
sqrrrl wants to merge 4 commits into
mainfrom
fix-multi-video

Conversation

@sqrrrl

@sqrrrl sqrrrl commented Mar 26, 2026

Copy link
Copy Markdown
Member

This PR addresses a bug where only one video stream would play at a time, even when multiple streams were requested. The fix involves resolving a "frame stealing" issue in the internal library and implementing
automatic layout repetition to simplify client-side configuration.

Changes

Internal Library Logic

  • Resolved "Frame Stealing": Removed the use of MediaStreamTrackProcessor in InternalMeetStreamTrackImpl. The previous implementation was consuming and closing frames to detect SSRCs, which prevented those frames
    from reaching the DOM
  • Non-Intrusive SSRC Polling: Replaced the processor with a non-intrusive polling mechanism that uses RTCRtpReceiver.getSynchronizationSources() and getContributingSources(). This allows the library to identify and
    assign tracks to media entries without interrupting the media flow.
  • Automatic Layout Repetition: Updated MeetMediaApiClientImpl.applyLayout to automatically repeat the last provided MediaLayoutRequest if the number of requests is fewer than the numberOfVideoStreams configured at
    session start. This maintains backward compatibility with simpler client implementations while ensuring all requested streams are active.
  • Assignment Optimization: Refined VideoAssignmentChannelHandler and MediaEntriesChannelHandler to only trigger the assignment search for tracks that do not already have an associated media entry, reducing redundant
    processing.

Sample Application

  • Stability: Added explicit .play() calls to media elements upon assignment to ensure reliable playback across different browser autoplay policies.

Verification Results

  • Build: Successfully verified with npx webpack in the samples directory.
  • Functionality: Verified that multiple video streams now play simultaneously even when only a single media layout is explicitly provided by the client.

sqrrrl and others added 3 commits March 26, 2026 01:09
…uts are provided than requested streams. Reverted sample script to only request one layout.
Removed unused variable 'requestedVideoStreamCount' from script.ts.
@sqrrrl sqrrrl changed the title Fix Multi-Video Stream Playback and Internal Frame Stealing FixL Fix Multi-Video Stream Playback and Internal Frame Stealing Mar 26, 2026
@sqrrrl sqrrrl changed the title FixL Fix Multi-Video Stream Playback and Internal Frame Stealing Fix: Fix Multi-Video Stream Playback and Internal Frame Stealing Mar 26, 2026
@sqrrrl
sqrrrl requested a review from koumiGoogle April 1, 2026 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant