[2.46][GST][MSE] Skip playback state update while EOS

[asurdej-comcast]

Playback state change notification syncs Media Client (HTMLMediaElement) state with current player state:
commit fb0730e:

The platform backend may change state, for example as a result
of an external plugin controlling the backend, so we need to
react to this situation by syncing up the WebCore state with the
platform backend.

We call playInternal()/pauseInternal() depending on the backend
state, to trigger the corresponding DOM events to match the state.

Sending that notification on EOS condition may effectively pause the pipeline without user request becasue the GST player reports it is paused m_isEndReached is set.

PS. We already had a bunch of problems with that playback state change notification and the way of how HTMLMediaElement handles mediaPlayerPlaybackStateChanged(). In 2.46 that part was added because of sleep disable while it was completely missing in 2.38. Maybe we should handle sleep disable separately and skip that mediaPlayerPlaybackStateChanged() at all
46455f0

Build-Tests	Layout-Tests
✅ 🛠 wpe-246-amd64-build	✅ 🧪 wpe-246-amd64-layout
✅ 🛠 wpe-246-arm32-build	✅ 🧪 wpe-246-arm32-layout

[asurdej-comcast]

Here is a test case for that https://asurdej-comcast.github.io/tr/tests/RDKTV-38576_mse_seek_at_eos.html

The flow is:

			video.addEventListener('ended', async function () {
				video.pause()

				// Reset timestampOffset - that will reopen MediaSource for appending (even after EOS)
				audioSb.timestampOffset = 1;
				videoSb.timestampOffset = 1;
				await appendData()

				video.currentTime = 1;
				video.play()
				// Mark end of stream to recalculate buffering state and readyState
				// at new position. Seek request may not be fully handled yet at this point.
				// This should make readyState = HAVE_ENOUGH_DATA and allow playback to start truely.
				ms.endOfStream();
			}

With current impl the playback gets stuck in paused state and never restarts

[asurdej-comcast]

Here is a sequence of events from WebKit perspective that leads to video stall:

• Ad playback progress and EOS is set on Media Source
• Upon hitting the end webkit sends ended event and timeupdate
• The app pauses the video that is reflected in real GST pipeline pause
• Ready state drops to HaveCurrentData
• The same MediaSouce and SourceBuffers are reused. Samples replacement happened.
• The app does seek(20.0) and play() -> WebKit marks seek position and enqueues seek request without affecting media player. Then the play() request is handled but it doesn't put GST pipeline in playing state because of ready state is not sufficient (have meta data at this point)
• Now GST Media player receives ready state change from MediaSource, for new seek position it already have data to play. That calls updateStates() in MSE media player which results in changing pipeline state to PLAYING and send playbackStateChanged() to HTMLMediaElement()
• Media Player has m_isEndReached flag set still and reports that the player is paused - ::paused() returns true -> HTMLMediaElement() see that playback is paused while it was playing before so it calls pauseInternal() that reverts the last play() request and marks that video should be paused. The pipeline is still in PLAYING
• Now we have a seek task executed on HTMLMediaElement side that triggers a seek in MediaPlayer -> m_isEndReached flag is reset. MediaPlayer looses its state PAUSED-> PAUSED transition
• HTMLMediaElement::updateStates() is called (not sure why, probably becasue of seek). HTMLMediaElement thinks that MediaPlayer should be paused while it is playing so it calls MediaPlayer::pause() again
• GST pipeline doesn't get paused really bacause it is in async state transition and it will still report to HTMLMediaElement that it is playing (m_isPipelinePlaying is set still). That makes HTMLMediaElement to enter playing state again - from playbackStateChanged notification - but that play() is never passed to the MediaPlayer as it reports it is playing still.
• Seek is completed and and MediaPlayer notices the pause request -> the playback gets paused and HTMLMeidaElement is notified about it -> it accepts the pause and both HTMLMediaElement and MediaPlayer are paused

My proposition fixes the bold bullet, While m_isEndReached is set, it doesn't send playbackStateChanged() notification to HTMLMediaElement so it will never accept the paused state and won't force it later on MediaPlayer

[eocanha]

I'm having a look at this next Monday Oct 6th.

[eocanha]

I'm still trying to understand all the steps, but I wonder (to myself) if, instead of ignoring the m_isEndReached flag as a sort of hacky workaround, there's any way of resetting the flag ahead of time (even before the seek starts to be processed asynchronously), so that it's false already at the right time.

I'm trying to analyze if this is possible, for instance by using willSeekToTarget(), which sets m_pendingSeekTime. I still haven't checked if that call happens soon enough, tho. I'll continue analyzing it.

[eocanha]

I'm having a hard time to get your test case to work properly (at least on a Raspberry Pi, wpe-2.46), even after having mirrored it on a local server and added extra logs. First, I needed to increase the timeouts (the data load was slow for some reason). Then, I'm having spureous "can't push buffer because the pad is on EOS" failures in one of the WebKitMediaSrc pads. All this applying only your suggested fix, instead of mine.

I think I only saw it working (green) one time after multiple tries. I'm still debugging what happens.

[asurdej-comcast]

@eocanha Thanks for looking into this.
Here are my original logs from Disney+ app, maybe that would help to understand the flow:
eos_restart.txt

Disney plus uses the same MediaSource object and the same SourceBuffers to play both main content and advertisments. That is a bit tricky as MediaSource isn't accepting more data after endOfStream() so it has to be reopened somehow, I use timestampOffset of SourceBuffer that reopens MediaSouce.
Regarding the test case, there are 5 retries of the same, for me the playback gets paused on the very first seek/replay.

Resetting m_isEndReached, e.g. in MediaPlayerPrivate::play() would help but the GST is still at eos position so there are no more data to play so it will be reported by the sinks again.

Another "problem" could be the ready state, that inside MediaSource it is calculated based on the new position (current position is taken from m_pendingSeekTime set from willSeekToTarget()) and propagated to player while the player is still at the old position